DE102019106729A1 - Drive device and machine learning device - Google Patents

Abstract

A drive device which controls a speed or torque of a servomotor of a processing machine in accordance with a command obtained by converting an external command input from an external input has a machine learning device which learns the appropriate conversion from the external command to the command. The machine learning apparatus includes: a state observation unit that monitors the external command, the command, and a state of the processing machine or the servomotor as state variables representing a current state of an environment; a determination data acquisition unit that acquires determination data indicating an evaluation result of the processing by the processing machine; and a learning unit that learns the relationship between the external command and the state of the processing machine or the servomotor and the command using the state variables and the determination data.

Description

BACKGROUND OF THE INVENTION

Field of the invention

The present invention relates to a machine learning drive apparatus and apparatus and, more particularly, to a machine learning drive apparatus and apparatus which optimize a command generated in response to an external command.

Description of the Related Art

There are driving devices which receive a command issued from a control device (hereinafter referred to as an external command) and generate a command for controlling a device (hereinafter referred to simply as a command). In a typical example, as in 10 A control device outputs an analog voltage command as an external command, a servomotor drive device applies the analog voltage command to an AD conversion and generates a speed command and a torque command, and a servo motor, after being converted by the commands (the speed command and the torque command). driven.

While a command value is transmitted in the form of an analog voltage value in the case of an analog voltage command, an error may arise between one of the control device due to input offset voltage, noise, and the like which may be caused by the semiconducting characteristics and the effect of temperature drift (temperature drift) provided command value and interpreted by the drive device based on the received analog voltage value command value occur. Such an error affects the controllability of the servomotor and may cause phenomena such as an increase in cycle time, a loss of reaction speed, a loss of machining quality, and an increase in power consumption.

1. (1) dass, wenn eine Steuervorrichtung einen 0-V-Befehl als einen externen Befehl ausgibt, zur selben Zeit ein Servoabschaltsignal und ein Achsenstoppsignal ausgegeben werden;
2. (2) dass die Steuervorrichtung ein Verhalten des Servomotors überwacht und einen externen Befehl ausgibt, der das Verhalten des Servomotors (in einer entgegengesetzten Richtung) aufhebt (eine Schleife einbaut); und (3) dass eine tote Zone oder ein Filter bereitgestellt ist (zum Beispiel werden Spannungsschwankungen von ±0,001 V ungültig gemacht).

In order to suppress occurrence of such phenomena, a malfunction of a servomotor is usually prevented by measures such as, for example, even if an error has occurred in a received analog voltage command value.

1. (1) that when a control device outputs a 0V command as an external command, a power-off signal and an axis stop signal are output at the same time;
2. (2) that the control device monitors a behavior of the servomotor and outputs an external command that overrides (loops) the behavior of the servomotor (in an opposite direction); and (3) that a dead zone or filter is provided (for example, voltage variations of ± 0.001V are invalidated).

As a related technique, the disclosed Japanese Patent Application No. 2017-102613 a motor control device that calculates, by machine learning, correction amounts of a position command, a speed command, and the like for controlling an engine.

That in the disclosed Japanese Patent Application No. 2017-102613 The described method aims at correcting a deterioration of the smoothness of the movement of a feed axis that occurs due to various mechanical factors, and is not intended to cause an error occurring between a command value provided by a controller and a command value interpreted by the drive device. and that can cause adverse phenomena such as those described above to correct.

The measures normally taken ( 1 ) and ( 2 ) have a problem that a complicated mechanism including the addition of hardware and complex control must be developed by a controller. Measure ( 3 ) has a problem that fluctuations in command values falling below a threshold of a dead zone or a filter as well as errors exceeding the threshold can not be accommodated. Therefore, the measures are insufficient and therefore additional measures are needed.

[SUMMARY OF THE INVENTION]

The present invention has been made to solve such problems, and an object thereof is to provide a driving apparatus and a machine learning apparatus which optimize a command generated in response to an external command.

A driving device according to an embodiment of the present invention is a driving device that controls a speed and a torque of a servomotor of a processing machine according to an instruction obtained by converting an externally input external command, the driving device having a machine learning device that performs an appropriate conversion from learning external command to the command, the machine learning apparatus comprising: a state observation unit which monitors the external command, the command and a state of the processing machine or the servomotor as state variables representing a current state of an environment; a determination data acquisition unit that acquires determination data indicating an evaluation result of the processing by the processing machine; and a learning unit which learns the relationship between the external command and the state of the processing machine and the command using the state variables and the designation data.

In the drive apparatus according to an embodiment of the present invention, the state variables, as the state of the processing machine or the servomotor, include at least one of the group consisting of speed command value, torque command value, position feedback, speed feedback, torque feedback, motor current value and motor temperature of each of the shafts of the processing machine Cycle time of the processing, temperature of each of the units of the processing machine, shape and raw material of a material used in processing, and shape and raw material of a mold used in processing.

In the drive device according to an embodiment of the present invention, the determination data includes at least one of the group consisting of a precision and a quality of a processed product, a vibration of the processing machine, a mold or a ball screw, a load on the servomotor, a power consumption by the Processing machine and a cycle time of processing.

In the drive apparatus according to an embodiment of the present invention, the learning unit includes: an impact calculation unit that obtains an effect related to the evaluation result; and a value-function updating unit that, using the effect, updates a function representing a value of the command with respect to the external command and the state of the processing machine or the servomotor.

In the drive apparatus according to an embodiment of the present invention, the learning unit calculates the state variables and the determination data in a multi-layered structure.

The driving device according to an embodiment of the present invention further comprises a decision making unit that outputs a value of the command based on a learning result of the learning unit.

In the driving apparatus according to an embodiment of the present invention, the learning unit executes the learning using the state variables and the determination data obtained from a plurality of the driving devices.

In the drive apparatus according to an embodiment of the present invention, the machine learning apparatus is realized by a cloud computing environment, a fog computing environment (edge computing of the cloud), or an edge computing environment (edge computing of the network).

A machine learning apparatus according to an embodiment of the present invention is a machine learning apparatus that learns the appropriate conversion from an external command input to a drive device from the outside to a command for controlling a speed or torque of a servomotor of a processing machine , the machine learning apparatus comprising: a state observation unit that observes the external command, the command, and a state of the processing machine or the servomotor as state variables representing a current state of an environment; a determination data acquisition unit that acquires determination data indicating an evaluation result of processing by the processing machine; and a learning unit that learns the relationship between the external command and the state of the processing machine or the servomotor and the command using the state variables and the determination data.

In accordance with the present invention, a drive apparatus and a machine learning apparatus that optimize a command generated in response to an external command may be provided.

list of figures

• 1 ein schematisches Hardwarekonfigurationsdiagramm einer Antriebsvorrichtung gemäß einer ersten Ausführungsform ist;
• 2 ein schematisches Funktionsblockdiagramm der Antriebsvorrichtung gemäß der ersten Ausführungsform ist;
• 3 ein schematisches Funktionsblockdiagramm ist, das einen Aspekt einer Antriebsvorrichtung zeigt;
• 4 ein schematisches Ablaufdiagramm ist, das einen Aspekt eines maschinellen Lernverfahrens zeigt;
• 5A ein Diagramm ist, das ein Neuron darstellt;
• 5B ein Diagramm ist, das ein neuronales Netzwerk darstellt;
• 6 ein schematisches Funktionsblockdiagramm einer Antriebsvorrichtung gemäß einer zweiten Ausführungsform ist;
• 7 ein schematisches Funktionsblockdiagramm ist, das einen Aspekt eines Systems zeigt, welches eine Antriebsvorrichtung umfasst;
• 8 ein schematisches Funktionsblockdiagramm ist, das einen weiteren Aspekt des Systems zeigt, welches eine Antriebsvorrichtung umfasst;
• 9 ein schematisches Ablaufdiagramm ist, das einen Aspekt eines maschinellen Lernverfahrens zeigt; und
• 10 ein Diagramm ist, das schematisch einen Betrieb einer Antriebsvorrichtung darstellt.

The above and other objects and features of the present invention will become more apparent from the following description of embodiments when taken in conjunction with the accompanying drawings, in which:

• 1 FIG. 3 is a schematic hardware configuration diagram of a drive apparatus according to a first embodiment; FIG.
• 2 is a schematic functional block diagram of the drive device according to the first embodiment;
• 3 Fig. 10 is a schematic functional block diagram showing an aspect of a drive device;
• 4 Fig. 10 is a schematic flowchart showing an aspect of a machine learning method;
• 5A is a diagram representing a neuron;
• 5B is a diagram representing a neural network;
• 6 FIG. 3 is a schematic functional block diagram of a drive device according to a second embodiment; FIG.
• 7 Fig. 12 is a schematic functional block diagram showing an aspect of a system including a drive device;
• 8th Figure 12 is a schematic functional block diagram showing another aspect of the system including a drive device;
• 9 Fig. 10 is a schematic flowchart showing an aspect of a machine learning method; and
• 10 Fig. 3 is a diagram schematically illustrating an operation of a drive device.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A drive device 1 According to an embodiment of the present invention, receiving an external command issued from a controller and having an error converts the received external command and generates a command that is typically used to drive a device (a servomotor or the like) ), and outputs this. The command drives the device and performs a machining. The drive device 1 is able to evaluate a correctness of a result of the processing and to optimize the conversion from the external command to the command to improve the processing results.

The control device is typically a device that controls a processing machine. For example, the control device of an injection molding machine is capable of controlling a plurality of axes, such as an injection axis, a measurement axis, a mold closure axis, a projection axis, a plastication axis, and an additional axis. The drive device 1 For example, a device which drives a servomotor of one of these axes and which is subordinate to the control device. The drive device 1 receives an analog voltage command output from the controller, A / D conversion the analog voltage command, and generates a speed command value and a torque command value, and outputs the command values to actually drive a servomotor.

1 FIG. 13 is a schematic hardware configuration diagram including the drive device. FIG 1 according to the first embodiment of the present invention and a servomotor driven by the driving device 1 is driven. The drive device 1 has a CPU 11 , a ROM 12 , a ram 13 , a non-volatile memory 14 , an interface 21 , an interface 22 , a bus 20 and a servo control unit 30 on. A servomotor 50 and a control device 80 are with the drive device 1 connected.

The CPU 11 is a processor that drives the entire drive 1 controls. The CPU 11 reads in the ROM 12 stored system program via the interface 22 and the bus 20 and controls the entire drive device 1 in line with the system program.

The ROM 12 stores in advance various system programs (including a system program for controlling communication with a machine learning device 100 which will be described later) required for the drive control of the servomotor 50 and the like.

The RAM 13 temporarily stores calculation data and the like.

The non-volatile memory 14 is buffered by a battery (not shown), for example, and is capable of maintaining its memory status even when power is supplied to the driving device 1 is off. The non-volatile memory 14 stores various programs, data and the like. The non-volatile memory 14 stored programs and data may be in the execution or use of the RAM 13 to be provided.

The servo control unit 30 receives a speed command and a torque command issued by the CPU 11 have been issued, and drives the servomotor 50 at.

The servomotor 50 is through the servo control unit 30 driven and moved in the processing machine provided axes. Typically, the servomotor can 50 have a built-in position / speed detector. The position / velocity detector outputs a position / velocity feedback signal and feedback control of position and velocity is performed when the position / velocity feedback signal is sent to the controller 80 is returned.

It should be noted that - although the control device 80 and the drive device 1 in 1 one-to-one connected - in fact a number of drive devices 1 that corresponds to the number of axes provided in the processing machine representing a control object with the control device 80 are connected. If the control device 80 For example, controlling a machining machine provided with six axes is a total of six driving devices 1 corresponding to the respective axes with the control device 80 connected.

The control device 80 is a device for controlling a processing machine, and is formed of, for example, a numerical control device. The control device 80 generates an external command to control an axis in accordance with a machining program, and passes the external command through the interface 22 to the CPU 11 further. While an external command is typically input as an analog voltage value, the analog voltage value may include an error attributable to input offset voltage, noise, or the like. In addition, the control device directs 80 various data information required for machine learning (to be described later) through the interface 22 to the CPU 11 further.

the interface 21 is an interface for connecting the drive device 1 with the machine learning device 100 , The machine learning device 100 has a processor 101 , a ROM 102 , a ram 103 and a nonvolatile memory 104 on.

The processor 101 controls the entire machine learning device 100 , The ROM 102 saves system programs and the like. The RAM 103 performs temporary storage in corresponding processes related to machine learning. The non-volatile memory 104 stores learning models and the like.

The machine learning device 100 observed over the interface 21 various pieces of information (external commands, various data information and the like provided by the control device 80 entered), which by the drive device 1 can be obtained. The machine learning device 100 gives commands (a speed command and a torque command) to drive the servomotor 50 over the interface 21 to the drive device 1 out. The drive device 1 receives one through the machine learning device 100 output command and gives the command to the servo control unit 30 out.

2 is a schematic functional block diagram of the drive device 1 and the machine learning apparatus 100 according to the first embodiment. The machine learning device 100 has a state observation unit 106 , a determination data acquisition unit 108 and a learning unit 110 on. The state observation unit 106 , the determination data acquisition unit 108 and the learning unit 110 For example, as a function of the processor 101 be implemented. Alternatively, the state observation unit 106 , the determination data acquisition unit 108 and the learning unit 110 for example, be implemented by the processor 101 in the ROM 102 stored software executes.

The state observation unit 106 observes state variables S representing a current state of an environment. The state variables S include a command value S1 obtained by converting and correcting an analog voltage command value having an error, an analog voltage command value S2 that has an error, and various pieces of information indicating a state of a processing machine. For example, if the processing machine is an injection molding machine, the state variables S may be axis information S3 with respect to the states of all the axes of the injection molding machine, one cycle time / cycle time S4 the injection molding machine, a temperature S5 the injection molding machine, material information S6 in terms of the resin material and shape information S7 with respect to an injection mold.

The command value S1 is a command value that is generated by the analog voltage command value S2 (which will be described later) subjected to A / D conversion, and a correction thereof is applied. A process of A / D conversion and correction can be reversed. The correction process can be done by the processor 101 by adding a prescribed offset to an analog input voltage, a part (for example, a leading edge) of a waveform an analog voltage is multiplied by a prescribed coefficient for changing the waveform, and the like. A variety of command values S1 can be obtained by varying the offset, the coefficient, and the like.

The analog voltage command value S2 is an analog voltage value, which is the drive device 1 as an external command from the control device 80 is entered. The state observation unit 106 is capable of the analog voltage command value S2 from the drive device 1 capture.

The axis information S3 are a speed command value, a torque command value, a position feedback, a speed feedback, a torque feedback, a motor current value, an engine temperature, and the like of all or one or more arbitrary axes of the injection molding machine. The state observation unit 106 captures the axis information S3 from the drive device 1 , The drive device 1 is capable of this item of information from the controller 80 capture.

The cycle time S4 is a cycle time of machining in the injection molding machine. The state observation unit 106 records the cycle time S4 from the drive device 1 , The drive device 1 is capable of this information from the control device 80 capture.

The temperature S5 represents a temperature of each of the units of the injection molding machine. The state observation unit 106 detects the temperature S5 from the drive device 1 , The drive device 1 is capable of this information from the control device 80 capture.

The material information S6 That is, information indicating a shape (for example, an identifier uniquely indicating a shape) and a raw material (for example, an identifier uniquely indicating a raw material) of a resin material used in the injection molding machine. The state observation unit 106 captures the material information S6 from the drive device 1 , The drive device 1 is capable of this item of information from the controller 80 capture.

The shape information S7 For example, information indicating a shape (for example, an identifier uniquely indicating a shape) and a raw material (for example, an identifier uniquely indicating a raw material) of a shape used in the injection molding machine are. The state observation unit 106 captures the shape information S7 from the drive device 1 , The drive device 1 is capable of this item of information from the controller 80 capture.

The determination data acquisition unit 108 obtains determination data D indicative of an index indicating a result when the servo motor is driven among the state variables S. The determination data D show a precision D1 which is information regarding a precision and a quality of a processed product, and various pieces of information indicating a state of a processing machine. For example, if the processing machine is an injection molding machine, the information may be a vibration D2 the injection molding machine, an injection mold, a ball screw or the like, a load D3 on the servomotor 50 , Power consumption / power consumption D4 through the injection molding machine, a cycle time D5 the injection molding machine and a reaction time D6 of the servomotor 50 include.

As the precision D1 can the determination data acquisition unit 108 For example, use a value that indicates an error between the dimensions and design values of the processed product. As the precision D1 An evaluation value may also be used that indicates a quality (for example, a surface condition) of the processed product. Since a detection method, an evaluation method and the like of the error and the evaluation value described above are known methods, detailed descriptions are omitted herein. For example, the error and the evaluation value may be measured within a processing machine, or they may be measured by an inspection machine or the like outside the processing machine. A measurement result of the inspection machine is transferred to the control device 80 entered. The drive device 1 detects this individual information from the control device 80 , and the determination data acquiring unit 108 detects this individual information from the drive device 1 ,

As the vibration D2 can the determination data acquisition unit 108 Information (a displacement, a speed, an acceleration or the like) indicative of a vibration of the injection molding machine, a mold, a ball screw or the like from the driving device 1 to capture. The drive device 1 is capable of this item of information from the controller 80 capture.

As the burden D3 can the determination data acquisition unit 108 from the drive device 1 Information which is a load on the servomotor 50 specify, record. The load on the servomotor 50 can through the control device 80 be recorded. The drive device 1 detects this information from the control device 80 , and the determination data acquiring unit 108 detects this individual information from the drive device 1 ,

As the energy consumption D4 can the determination data acquisition unit 108 from the drive device 1 Collect information indicating the energy consumption by the injection molding machine. The drive device 1 is capable of this information from the control device 80 capture.

As the cycle time / cycle time D5 can the determination data acquisition unit 108 from the drive device 1 Information indicating a cycle time of the injection molding machine, capture. The drive device 1 is capable of this information from the control device 80 capture.

As the reaction time D6 can the determination data acquisition unit 108 from the drive device 1 a reaction time of the servomotor 50 , or in other words, a time from the issuance of a command value until the speed, torque or the like reaches the command value. The reaction time of the servomotor 50 can through the control device 80 be recorded. The drive device 1 detects this information from the control device 80 , and the determination data acquiring unit 108 detects this information from the drive device 1 ,

Using the state variable S and the destination data D learn the lesson 110 Correlations between the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ), and the command value S1 , In other words, the learning unit generates 110 Model structures showing a correlation ratio of the components S1 . S2 . S3 . S4 . S5 . S6 and S7 the state variable S Show.

Considered under the aspect of a learning period in the learning unit 110 they are in the learning unit 110 entered state variables S Data from a learning period before a learning period during which the determination data D have been recorded. While the machine learning device 100 learning takes place in the environment ( 1 ) the acquisition of the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 , ( 2 ) driving the servomotor 50 in accordance with the command value S1 or, in other words, the execution of the processing, and ( 3 ) the acquisition of the determination data D repeated. The determination data D in ( 3 ) are an evaluation result of precision or quality of a processed product, productivity, efficiency, or the like in one on the command value S1 based editing.

By repeating such a learning cycle, the learning unit becomes 110 capable of providing a feature which correlates between the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 automatically detect. Although the correlations between the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 Essentially unknown at the beginning of a learning algorithm identifies the learning unit 110 gradually a feature and interprets the correlation ratio as the learning progresses. Once the correlations between the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 have been interpreted to a reasonably reliable level, will be a learning outcome by the learning unit 110 used repeatedly to carry out a selection of an action (decision making) regarding the type of command value to be issued in relation to the current state S2 , or, in other words, the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ). In other words, the learning unit becomes 110 capable of outputting an optimum solution of an action according to the current state.

The state variables S are formed of data hardly affected by noise and the determination data D are obtained once by the drive device 1 is caused to different data information from the control device 80 capture. Therefore, according to the machine learning apparatus 100 by using a learning outcome from the lesson 110 an optimal command value S1 in Reference to the current state, or in other words, the analog voltage command value S2 that has an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) are automatically and accurately obtained without the use of calculations or estimates. In other words, an optimal command value S1 be quickly determined simply by the current state, or in other words, the analog voltage command value S2 that has an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) be included. Therefore, a suitable conversion of an external command into a command can be performed efficiently.

As a modification of the machine learning apparatus 100 can the lesson 110 using the state variables S and the determination data D obtains appropriate command values in the processing machines for each of a plurality of processing machines of the same type performing the same operation. According to this configuration, speed and reliability of learning can be improved because a set of data sets representing the state variables S and the determination data D that have been obtained, exhibited, increased, and records entered with greater variety over a particular period of time.

It should be noted that through the learning unit 110 is not specifically limited, and learning algorithms known as machine learning can be adopted. 3 shows an aspect of the drive device 1 , in the 1 is shown, in which the drive device 1 is designed to include the learning unit, which performs reinforcement learning as an example of the learning algorithm. Gain learning is a method comprising observing a present state (in other words, an input) of an environment in which a learning object exists, and performing a prescribed action (in other words, an output) in the present state, repeating, by trial and error, of cycles that provide some sort of impact to the action, and learning a strategy that maximizes an overall impact (in the present embodiment, an output of a command value) as an optimal solution.

In the machine learning device 100 which the drive device 1 , in the 3 is shown, instructs the learning unit 110 an impact calculation unit 112 and a value-function updating unit 114 on.

The impact calculation unit 112 obtains an impact R in connection with an evaluation result (which the determination data D in a learning period after that in which the state variables S have been detected) of a precision and a quality of a processed product, a productivity, an efficiency, or the like, when the machining conditions based on the state variables S be set.

The value-function update unit 114 updates a function Q representing a value of a command value using the impact R , By the value function updating unit 114 the function Q repeatedly updated, learns the lesson 110 Correlations between the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 ,

The following is an example of an algorithm for the lesson 110 described gain learning described. The algorithm in accordance with this example is known as Q-learning, which is a method involving a state s of an action agent and an action a that the action agent in the state s as independent variables to learn a function Q ( s, a) which represents a value of the action a when selected in the state s. Selecting action a, which maximizes value function Q in state s, provides an optimal solution. By starting Q-learning in a state in which a correlation between the state s and the action a is unknown, and repeating trial-and-error by selecting various actions a in any state s, the value function Q is repeatedly updated and the optimal solution approximated. In this case, by adopting a configuration in which the environment (in other words, the state s) changes as a result of selecting the action a in the state s, the value function Q may have an effect (in other words, a weighting of the Action a) r is provided in accordance with the change, and guiding the learning so that the action a is selected for which a higher impact r is provided will be approximated to the optimal solution within a relatively short period of time.

An update formula of the value function Q can generally be determined by the below reproduced Expression 1 are shown. In expression 1, st and a _{t denote} a state and an action at a time t, respectively, and the state changes to S _{t + 1} due to the action a _t . r _{t + 1} denotes an effect obtained when the state changes from s _t to s _{t + 1} . The expression maxQ denotes Q when an action a is executed which generates (or is considered to generate) a maximum value Q at time t _{+ 1} . α and γ denote a learning coefficient and a discount factor, respectively, and are arbitrarily set to values of 0 <α ≦ 1 and 0 <γ ≦ 1. $$Q\left(s_t.a_t\right)\leftarrow Q\left(s_t.a_t\right)+\alpha \left(r_{t+1}+\gamma \underset{a}{max}Q\left(s_{t+\mathrm{1,}}a\right)-Q\left(s_{t.}{a}_t\right)\right)$$

When the lesson 110 Q-learning, the state variables correspond S by the state observation unit 106 be observed, and the determination data D generated by the determination data acquisition unit 108 the state s of the update formula, the action, such as the command value S1 with respect to the current state, or in other words, the analog voltage command value S2 that has an error, and the condition of the processing machines (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) correspond to action a of the update formula, and the impact R generated by the impact calculation unit 112 is obtained corresponds to the impact r of the update formula. Therefore, the value-function updating unit updates 114 repeats the function Q which represents a value of the command value with respect to the current state by Q learning using the effect R ,

For example, when the servomotor is based on the determined command value S1 and the processing is performed and an evaluation result of a precision or a quality of a processed product, a productivity, an efficiency or the like is determined to be "adequate", the impact calculation unit 112 The effect R set to a positive value. On the other hand, when an evaluation result of precision or quality of a processed product, productivity, efficiency or the like is determined to be "inadequate", the impact calculation unit may 112 The effect R set to a negative value. Absolute values of positive or negative effects R may be the same or different.

Cases in which the evaluation result of a precision or a quality of a processed product, a productivity, an efficiency or the like is determined to be "adequate" may include, for example: when the precision D1 (an error between dimensions and design values of the processed product or the like) is smaller than a threshold value; if the vibration D2 is less than a threshold; when the load D3 is less than a threshold; when the energy consumption D4 is less than a threshold; if the cycle time D5 is shorter than a threshold; and when the reaction time D6 is shorter than a threshold. Cases in which the evaluation result of precision or quality of a processed product, productivity, efficiency or the like is determined to be "inadequate" may include, for example, cases in which D1 to D6 are equal to or exceed the prescribed thresholds. Alternatively, the impact calculation unit 112 determine the correctness by combining a plurality of values included in the determination data D are included.

The impact calculation unit 112 may determine an evaluation result of a precision or a quality of a processed product, a productivity, an efficiency, or the like, by a value (a change amount, a change rate or the like within a prescribed period or a prescribed cycle), a change with time the one described above D1 to D6 indicates used. For example, a determination of "adequate" may be made if the amount of change of those described above D1 to D6 0 or takes a negative value, while a determination of "inadequate" can be made if the amount of change takes a positive value.

An evaluation result of a precision or a quality of a processed product, a productivity, an efficiency, or the like can only be set to two values, namely, "adequate" and "inadequate," but may be set to a plurality of levels. For example, the impact calculation unit 112 The effect R so reduce that impact R the smaller the difference on a positive side between one of the values of D1 to D6 and a prescribed threshold, and the impact R so increase that impact R the larger the difference on a negative side, the greater the value. For example, the impact calculation unit 112 The effect R so reduce that impact R the smaller the difference is on a positive side between an amount of change or a rate of change of the above-described values of D1 to D6 and a prescribed threshold, and the impact R so increase that impact R the larger the difference on a negative side, the greater the value.

The value-function update unit 114 may have an action value table in which the state variables S , the determination data D and the impact R associated with an action value (for example, a numerical value) represented by the function Q , are organized. In this case, a process of updating the function Q by the value function updating unit 114 synonymous with a process of updating the action value table by the value function updating unit 114 be. Because the correlations between the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 are unknown at the beginning of Q-learning, there are various state variables in the action value table S , Determination data D and effects R in terms of their correlation with randomly determined values (the function Q ) of the action value. The impact calculation unit 112 is capable of doing so R to calculate immediately as soon as the determination data D are known, in which case the calculated value R is written to the action value table.

When Q learning using an impact R In accordance with an evaluation result of a precision or a quality of a processed product, a productivity, an efficiency, or the like, the learning is guided in a direction in which an action having a higher effect is selected R and the action value table is updated by adding a value (the function Q ) of an action value with respect to an action performed in the current state in accordance with a state (in other words, the state variables S and the determination data D ) of the environment which changes as a result of executing the selected action in the present state is overwritten. By repeating this update, the value (the function Q ) of the action value displayed in the action value table is overwritten such that the more correct the action is, the greater the value. In this way, the previously unknown correlation ratio between the current state of the environment or, in other words, the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 gradually becoming apparent. In other words, by updating the action value table, the correlations between the analog voltage command value become S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 gradually approaching an optimal solution.

Referring to 4 becomes a course of Q learning (in other words, an aspect of a machine learning method) that is executed by the learning unit 110 is executed, described in more detail.

step SA01 : The value function update unit 114 at this point in time refers to the action value table, to the command value S1 as an action, in the present state, by the state variable S observed by the state observation unit 106 is displayed, execute is to select at random.

step SA02 : The value function update unit 114 loads the state variables S by the state observation unit 106 observed present condition.

step SA03 : The value function update unit 114 loads the through the determination data acquisition unit 108 detected determination data D of the current state.

step SA04 : The value function update unit 114 determined based on the determination data D whether the command value S1 is appropriate or not. If the command value S1 is appropriate, a transition is made to step SA05 , If the command value S1 is inappropriate, a transition is made to step SA07 ,

step SA05 : The value function update unit 114 applies a positive impact R obtained by the impact calculation unit 112 to an update formula of the function Q at.

step SA06 : The value function update unit 114 updates the action value table using the state variables S and the determination data D in the present state, the impact R and the value of the action value (the function Q after the update).

step SA07 : The value function update unit 114 applies a negative impact R obtained by the impact calculation unit 112 to an update formula of the function Q at.

The learning unit 110 updates the action value table repeatedly by repeating the steps SA01 to SA07 , and continues learning.

It should be noted that the process of obtaining the impact R and the update operation of the value function in the steps SA04 to SA07 with respect to each of the data information containing the determination data D have executed. For example 9 a flowchart in a case in which the reaction time D6 and the energy consumption D4 as two items of destination data D are used.

step SB01 : The value function update unit 114 at this point in time refers to the action value table, to the command value S1 as an action, in the present state, by the state variables S observed by the state observation unit 106 is displayed, execute is to select at random.

step SB02 : The value function update unit 114 loads the state variables S by the state observation unit 106 observed present condition.

step SB03 : The value function update unit 114 loads the through the determination data acquisition unit 108 recorded determination data D of the current state.

step SB04 : The value function update unit 114 determined based on the reaction time D6 in the determination data D whether the command value S1 is appropriate or not. If the command value S1 is appropriate, a transition is made to step SB05 , If the command value S1 is inappropriate, a transition is made to step SB06 ,

step SB05 : The value function update unit 114 applies a positive impact R1 obtained by the impact calculation unit 112 to an update formula of the function Q at.

step SB06 : The value function update unit 114 applies a negative impact R1 obtained by the impact calculation unit 112 to an update formula of the function Q at.

step SB07 : The value function update unit 114 determined based on the energy consumption D4 in the determination data D whether the command value S1 is appropriate or not. If the command value S1 is appropriate, a transition is made to step SB08 , If the command value S1 is inappropriate, a transition is made to step SB09 ,

step SB08 : The value function update unit 114 applies a positive impact R2 obtained by the impact calculation unit 112 to an update formula of the function Q at.

step SB09 : The value function update unit 114 applies a negative impact R2 obtained by the impact calculation unit 112 to an update formula of the function Q at.

step SB10 : The value function update unit 114 updates the action value table using the state variables S and the individual determination data D4 and D6 in the present state, the effects R1 and R2 and the value of the action value (the function Q after the update).

As a result, a specific example of a learning process is performed in accordance with the 9 described sequence described.

example 1

• • In step SB01 becomes the following value as the command value S1 output. Speed command value S1 :
  • X = speed conversion value of 0.021V
• • In step SB02 For example, the following values are detected as the state variables S. Analog voltage command value S2 :
  • 0.021 V (having an error of 0.021 V)
  axis information 3 (Position feedback):
  • X = 0.001 mm continuous increase
  • Y = 5,000 mm continuous increase
  • Z = 10,000 mm continuous increase
  axis information 3 (Speed feedback):
  • X = 0.001 min-1
  • Y = 1,000 rpm
  • Z = 2,000 min-1
• • In step SB03 The following values are detected as the determination data D. reaction time D6 :
  • 100 ms increase
  power consumption D4 :
  • 1.0 kWh increase
• • Because the reaction time D6 has increased in step SB04 determines that the command value S1 is inappropriate.
• • In step SB06 the effect R is reduced.
• • As the energy consumption D4 has increased in step SB07 determines that the command value S1 is inappropriate.
• • In step SB09 becomes the impact R reduced.
• • In step SB10 becomes the value Q updated.

Example 2

• • In step SB01 becomes the following value as the command value S1 output. Speed command value S1 :
  • X = speed conversion value of 0.000V
• • In step SB02 For example, the following values are detected as the state variables S. Analog voltage command value S2 :
  • 0.021 V (having an error of 0.021 V)
  axis information 3 (Position feedback):
  • X = 0.000 mm stopped state
  • Y = 5,000 mm continuous increase
  • Z = 10,000 mm continuous increase
  axis information 3 (Speed feedback):
  • X = 0.000 min-1
  • Y = 1,000 rpm
  • Z = 2,000 min-1
• • In step SB03 The following values are detected as the determination data D. reaction time D6 :
  • 100 ms shorter
  power consumption D4 :
  • 1.5 kWh reduction
• • Because the reaction time D6 has decreased in step SB04 determines that the command value S1 is appropriate.
• • In step SB05 becomes the impact R elevated.
• • As the energy consumption D4 has decreased in step SB07 determines that the command value S1 is appropriate.
• • In step SB08 becomes the impact R elevated.
• • In step SB10 becomes the value Q updated.

In advancing gain learning, for example, a neural network may be used instead of Q learning. 5A schematically maps a model of a neuron. 5B Forms a model by combining the in 5A imaged neurons configured three-layer neural network. The neural network may be formed, for example, by a data processing device, a storage device, and the like that simulate a model of a neuron.

This in 5A The neuron shown outputs a result y with respect to a plurality of inputs x (in this case, for example, input x ₁ to input x ₃ ). Each of the inputs x ₁ to x ₃ is multiplied by a weight w (w ₁ to w ₃ ) corresponding to the input x. Accordingly, the neuron outputs the result y represented by the expression 2 shown below. It should be noted that in Expression 2, the input x, the output y, and the weight w are each vectors. In addition, θ denotes a distortion and f _k denotes an activation function. $$y=f_k\left({\displaystyle {\Sigma }_{i=1}^n{x}_i{w}_i-\theta }\right)$$

With the three-layered neural network, which in 5B is shown, a plurality of inputs x (in this case, for example, the inputs x1 to x3) are input from a left side, and results y (in this case, for example, a result y1 to y3) are output from a right side. In the example shown, the respective inputs x1, x2 and x3 are multiplied by corresponding weights (together represented by w1), and each of the inputs x1, x2 and x3 becomes three neurons N11 . N12 and N13 entered.

In 5B are the outputs of the respective neurons N11 to N13 represented together by z1. z1 may be regarded as a feature vector obtained by extracting a feature amount of an input vector. In the illustrated example, the respective feature vectors z1 are multiplied by corresponding weights (represented together by w2), and each of the feature vectors z1 becomes two neurons N21 and N22 entered. The feature vectors z1 represent a feature between a weight W1 and a weight W2 represents.

In 5B are the outputs of the respective neurons N21 to N22 represented together by z2. z2 may be regarded as a feature vector obtained by extracting a feature amount of the input vector z1. In the illustrated example, the respective feature vectors z2 are multiplied by corresponding weights (together represented by w3), and each of the feature vectors z2 becomes three neurons N31 . N32 and N33 entered. The feature vectors z2 represent a feature between a weight W2 and a weight W3 Finally, the neurons give N31 to N33 the corresponding results y1 to y3.

It should be noted that a method of so-called deep learning using a neural network having three or more levels can also be used.

In the machine learning device 100 can by doing that the lesson 110 a computation with a multi-layered structure in accordance with the neural network using the state variables S and the determination data D as the input x, the instruction value S1 are output as the result y. In addition, in the machine learning device 100 in that the learning unit 110 Uses a neural network as a value function in gain learning and performs a calculation having a multi-layered structure in accordance with the neural network using the state variable S and an action a as the input x, outputting a value (the result y) of the action in the state become. Operating modes of a neural network include a learning mode and a value prediction mode, and for example, a weight w can be learned using a learning data set in the learning mode, and a value determination of an action can be performed in the value prediction mode using the learned weight w. Acquisition, classification, inference and the like can also be performed in the value prediction mode.

The above-described configuration of the drive device 1 can be described as a machine learning process (or program), which is executed by the processor 101 is performed. The machine learning method is a machine learning method of learning an appropriate conversion of an external instruction into the instruction value S1 , the machine learning method comprising the steps performed by a CPU of a computer: observing the analog voltage command value S2 having an error and the state of a processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) as state variables S representing a current state of an environment; Acquisition of determination data D which is an evaluation result of a precision or a quality of a processed product, a productivity, an efficiency or the like in accordance with an output command value S1 specify executed processing; and learning the analog voltage command value S2 having an error and the state of a processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 in conjunction with each other using the state variables S and the determination data D ,

6 shows a drive device 2 according to a second embodiment of the present invention. The drive device 2 has a machine learning device 120 and a state data acquisition unit 3 on.

The state data acquisition unit 3 detects the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) and the command value S1 as the state data S0 and represents the state observation unit 106 the status data S0 ready. For example, the state data acquisition unit 3 the status data S0 from the drive device 2 to capture.

The machine learning device 120 indicates in addition to the state observation unit 106 , the determination data acquisition unit 108 and the learning unit 110 a decision-making unit 122 on. For example, the decision making unit 122 as a function of the processor 101 be implemented. Alternatively, the decision making unit 122 for example, be implemented by the processor 101 is caused in the ROM 102 to run stored software.

In addition to software (such as a learning algorithm) and hardware (the processor 101 or the like) for autonomously learning an appropriate conversion from an external instruction to the instruction value S1 , Includes the machine learning device 120 Software (such as an arithmetic algorithm) and hardware (the processor 101 or the like) for outputting the command value S1 obtains as a command to the servomotor based on a learning result 50 , Alternatively, the machine learning device may be used 120 assume a configuration in which a common processor executes all software, including the learning algorithm and the arithmetic algorithm.

Based on a through the lesson 110 The learned result generates the decision making unit 122 a command value C , which is a command value S1 according to the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , of the temperature S5 , the material information S6 and the shape information S7 ). Once the decision-making unit 122 the command value C to the servomotor 50 outputs, becomes the servomotor 50 through the drive device 2 in accordance with the command value C driven. A state of the environment changes in this way.

The state observation unit 106 observes the state variables in a next learning period S that is due to the output of the command value C to the environment through the decision making unit 122 have changed. The learning unit 110 learns an appropriate conversion from an external command to the command value S1 , for example by updating the value function Q (in other words, the action value table) using the changed state variables S , Instead of the command value S1 from the through the state data acquisition unit 3 acquired status data S0 to capture, the state observation unit 106 thereby the command value S1 from the RAM 103 the machine learning device 120 capture as described in the first embodiment.

In addition, there is the decision-making unit 122 again the command value C , which is the command value S1 obtained based on the learning outcome, to the servomotor 50 out. By repeating this learning period, the machine learning device performs 120 learning continues and gradually improves the reliability of the instruction value S1 determined by the machine learning device 120 even.

The machine learning device 120 produces a similar effect as the machine learning device 100 according to the first embodiment. In addition, the machine learning device 120 capable of displaying a state of the environment through an output of the decision making unit 122 to change. In the machine learning device 100 can be a learning outcome of the learning unit 110 in the environment, by providing an external device functions which the decision making unit 122 correspond.

7 shows a system 170 comprising a plurality of drive devices. The system 170 has a plurality of drive devices 160 and drive devices 160 ' on. All of the drive devices 160 and the drive devices 160 ' are interconnected by a wired or a wireless network 172 connected.

The drive devices 160 and the drive devices 160 ' have a mechanism that is required for operations with the same goal. On the other hand, the drive devices 160 ' the machine learning device 100 not on while the drive devices 160 the machine learning device 100 exhibit.

The drive devices 160 comprising the machine learning device 100 are capable of learning the learning unit 110 to use the command value S1 according to the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) automatically and accurately, without using calculations or estimates. In addition, a configuration may be adopted in which the machine learning apparatus 100 at least one drive device 160 an appropriate conversion of an external command into the command value S1 , the all drive devices 160 and the drive devices 160 ' is common, based on the state variables S and the determination data D for each of the plurality of other drive devices 160 and the drive devices 160 ' be learned, and a learning outcome from it will be from all propulsion devices 160 and the drive devices 160 ' used together. According to the system 170 can provide a speed and reliability of learning an appropriate conversion from an external command to the command value S1 be improved by using datasets (including state variables S and the determination data D ) with greater variation than input.

8th shows a system 170 ' comprising a plurality of drive devices 160 ' , The system 170 ' has the plurality of drive devices 160 ' having the same mechanical configuration and the machine learning device 120 (or the machine learning device 100 ) on. The majority of drive devices 160 ' and the machine learning device 120 (or the machine learning device 100 ) are through a wired or wireless network 172 connected with each other.

The machine learning device 120 (or the machine learning device 100 ) learns a proper conversion of an external command into the command value S1 , the all drive devices 160 ' is common, based on the state variables S and the determination data D attained for each of the plurality of drive devices 160 ' , The machine learning device 120 (or the machine learning device 100 ) is capable of using a learning result thereof to the command value S1 according to the analog voltage command value S2 having an error and the state of the processing machine (the axis information S3 , the cycle time S4 , the temperature S5 , the material information S6 and the shape information S7 ) automatically and accurately, without using calculations or estimates.

The machine learning device 120 (or the machine learning device 100 ) can be in a cloud server or the like running on a network 172 is provided exist. According to this configuration, regardless of a position or a period of existence, each of the plurality of driving devices 160 ' whenever necessary, a required number of drive devices 160 ' with the machine learning device 120 (or the machine learning device 100 ) get connected.

An operator working on the system 170 or the system 170 ' can operate at an appropriate time after the start of learning by the machine learning device 120 (or 100 ) determine whether a degree of satisfaction of learning the command value S1 (in other words, the reliability of the command value S1 which is outputted) by the machine learning apparatus 120 (or the machine learning device 100 ) has reached a required level or not.

While embodiments of the present invention have been described above, it should be understood that the present invention is not limited to the examples set forth in the above-described embodiments and can be implemented in various aspects by making appropriate modifications thereof.

For example, by the machine learning device 100 and the machine learning device 120 executed learning algorithm by the driving device 1 and the drive device 2 The control algorithm and the like are not limited to those described above, and various algorithms can be adopted.

While the drive device 1 (or the drive device 2 ) and the machine learning device 100 (or the machine learning device 120 ) in the above-described embodiments are explained as having various CPUs, the machine learning apparatus may be explained 100 (or the machine learning device 120 ) to be configured by the CPU 11 which the drive device 1 (or the drive device 2 ), and one in ROM 12 stored system program can be implemented.

Further, the machine learning apparatus 100 (or the machine learning device 120 ) in information processing environments referred to as cloud data processing, fog data processing and edge data processing, and the like.

Moreover, while in the above-described embodiments, an example in which an analog voltage command value is input as an external command is explained in the first place, the present invention is not limited to this, but is applicable to all driving devices comprising a device according to one of drive command converted to any kind of external command.

While embodiments of the present invention have been described above, it should be understood that the present invention is not limited to the examples set forth in the above-described embodiments, and in other aspects, may be practiced by making suitable modifications thereto.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• JP 2017102613 [0005, 0006]

Claims (9)

A driving apparatus that controls a speed or a torque of a servomotor of a processing machine according to a command obtained by converting an externally input external command, the driving device including a machine learning apparatus that learns an appropriate conversion from the external command to the command, wherein the machine learning apparatus has: a state observation unit that observes the external command, the command, and a state of the processing machine or the servomotor as state variables representing a current state of an environment; a determination data acquisition unit that acquires determination data indicating an evaluation result of the processing by the processing machine; and a learning unit which learns, by using the state variables and the designation data, the external command and the state of the processing machine or the servomotor and the command in association with each other. Drive device after Claim 1 wherein the state variables, as the state of the processing machine or servomotor, are at least one of speed command value, torque command value, position feedback, speed feedback, torque feedback, motor current value and motor temperature of each of the axes of the processing machine, cycle time of processing, temperature of each of the units of the processing machine, mold and raw material of a material used in the machining, and mold and raw material of a molding tool used in the machining. Drive device after Claim 1 or 2 wherein the determination data comprises at least one of the group consisting of precision and quality of a processed product, vibration of the processing machine, forming tool or ball screw, load on the servomotor, power consumption by the processing machine, and cycle time of processing. Drive device according to one of Claims 1 to 3 wherein the learning unit comprises: an impact calculation unit that obtains an effect related to the evaluation result; and a value-function updating unit that, using the effect, updates a function representing a value of the command with respect to the external command and the state of the processing machine or the servomotor. Drive device according to one of Claims 1 to 4 wherein the learning unit calculates the state variables and the determination data in a multi-layered structure. Drive device according to one of Claims 1 to 5 , further comprising a decision making unit that outputs a value of the command based on a learning result by the learning unit. Drive device according to one of Claims 1 to 6 wherein the learning unit executes the learning using the state variables and the determination data obtained from a plurality of the driving devices. Drive device according to one of Claims 1 to 7 wherein the machine learning device is implemented by a cloud computing environment, a fog computing environment (edge computing of the cloud), or an edge computing environment (edge computing of the network). A machine learning apparatus which learns an appropriate conversion from an external command input from outside to a drive device to a command for controlling a speed or a torque of a servomotor of a processing machine, comprising the machine learning device: a state observation unit that observes the external command, the command, and a state of the processing machine or the servomotor as state variables representing a current state of an environment; a determination data acquisition unit that acquires determination data indicating an evaluation result of the processing by the processing machine; and a learning unit which learns, by using the state variables and the designation data, the external command and the state of the processing machine or the servomotor and the command in association with each other.

Images