JP6663538B1 - Machine learning device - Google Patents

Abstract

The machine learning device (100) learns control parameters for controlling machining conditions in the electric discharge machine (1). A machine learning device (100) includes a state observation unit (30) for observing a plurality of state variables representing a machining state during electric discharge machining, and a learning unit (40) for learning a control parameter based on the plurality of state variables. Equipped with.

Description

  The present invention relates to a machine learning device for learning control parameters for controlling electric discharge machining, an electric discharge machine, and a machine learning method.

  Adaptive control function to automatically change machining conditions expressed as physical quantities, such as changing power supply voltage waveform and power supply current waveform, and changing the gap control operation, which is a servo operation, in order to perform stable machining in an electric discharge machine There is. The processing conditions are determined by several to more than ten types of processing parameters that can be changed by the user. Parameters for changing the magnitude of the voltage applied to process the workpiece or the shape of the processing current pulse, parameters for adjusting the relative distance between the workpiece and the processing electrode serving as a tool, and changing the feed rate of the processing electrode Parameters and the like correspond to the processing parameters.

  A combination of these machining parameters can be experimentally obtained as a set of appropriate values using representative machining shapes, workpiece materials and electrode materials, and a plurality of sets are set in advance in the electric discharge machine. In some cases, it is possible for the user to make a selection. However, the shape to be machined by the electric discharge machine is a three-dimensional complicated shape, and the material to be machined also varies in view of the characteristics of the electric discharge machine, which can be machined if it can be energized. Therefore, it is necessary to optimize the processing parameters. For example, Patent Literature 1 discloses that the processing parameters are automatically set using a processing state input by an operator.

Japanese Patent Application Laid-Open No. H2-212041

  However, in the automatic setting described in Patent Literature 1, only a part of processing parameters that can be set by a user is adjusted based on a single type of processing state. In addition, in order to finally realize the processing conditions expressed as physical quantities based on the processing parameters, there are countless various control parameters as the background, and the control parameters are not adjusted.

  Therefore, in the adaptive control of the electric discharge machine, there has been a demand for adaptive control capable of acquiring more appropriate machining conditions as physical quantities.

  The present invention has been made in view of the above, and an object of the present invention is to provide a machine learning device capable of automatically learning more appropriate machining conditions in electric discharge machining.

In order to solve the above-described problem and achieve the object, a machine learning device of the present invention learns control parameters for determining machining conditions in an electric discharge machine. The machine learning device of the present invention is a state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining, and compares the plurality of state variables with a previous state variable so that the machining state is stabilized. A learning unit that learns control parameters. The control parameter is a correspondence table between a processing condition set value set by a user and a processing condition which is a physical quantity.

  The machine learning device according to the present invention has an effect that a more appropriate machining condition can be automatically learned in electric discharge machining.

1 is a block diagram showing a configuration of an electric discharge machine according to a first embodiment of the present invention. FIG. 6 is a diagram in which processing conditions according to the first embodiment are classified for control purposes. FIG. 4 is a diagram for explaining control parameters of processing conditions related to voltage control according to the first embodiment. FIG. 4 is a diagram illustrating a relationship between processing conditions related to voltage control and a generation cycle of a current pulse according to the first embodiment. FIG. 4 is a diagram for explaining control parameters of processing conditions related to pulse control according to the first embodiment. FIG. 4 is a diagram showing a relationship between machining conditions related to pulse control and the shape of a current pulse according to the first embodiment. FIG. 3 is a diagram for explaining control parameters of machining conditions related to axis drive control according to the first embodiment. The figure which shows the relationship between the machining condition regarding shaft drive control concerning Embodiment 1, and gap control by shaft drive. FIG. 4 is a diagram illustrating states of a voltage pulse and a current pulse according to the first embodiment. FIG. 4 is a diagram showing a case where voltage pulses and current pulses according to the first embodiment are stable. FIG. 4 is a diagram illustrating a case where a voltage pulse and a current pulse according to the first embodiment are unstable. FIG. 3 is a diagram showing an ideal average voltage value distribution according to the first embodiment; FIG. 7 is a diagram showing a distribution of average voltage values when stable discharge according to the first embodiment continues. FIG. 4 is a diagram showing a distribution of average voltage values when unstable discharge according to the first embodiment continues. 4 is a flowchart for explaining an optimization process by learning control parameters of machining conditions related to voltage control according to the first embodiment; 5 is a flowchart for explaining an optimization process by learning control parameters of machining conditions related to pulse control according to the first embodiment; 4 is a flowchart for explaining an optimization process by learning control parameters of machining conditions related to axis drive control according to the first embodiment; FIG. 2 is a block diagram showing a configuration of an electric discharge machine according to a second embodiment of the present invention. 3 is a block diagram showing a configuration of an electric discharge machine according to a third embodiment of the present invention. FIG. 2 is a diagram illustrating a hardware configuration when the functions of the machine learning device according to the first to third embodiments are implemented by a computer system.

  Hereinafter, a machine learning device, an electric discharge machine, and a machine learning method according to an embodiment of the present invention will be described in detail with reference to the drawings. The present invention is not limited by the embodiment.

Embodiment 1 FIG.
FIG. 1 is a block diagram illustrating a configuration of the electric discharge machine 1 according to the first embodiment of the present invention. The electric discharge machine 1 includes a machining electrode 2 serving as a machining tool, a driving device 4 for controlling a distance between the machining electrode 2 and the workpiece 3, and a driving device 4 between the machining electrode 2 and the workpiece 3. And a control device 10 for controlling the driving device 4 and the machining power supply 5. The workpiece 3 is connected to a processing power supply 5. The driving device 4 can drive one or both of the processing electrode 2 and the workpiece 3.

  The control device 10 corresponds to an axis drive control unit 11 that controls the driving device 4, a processing power supply control unit 12 that controls the processing power supply 5, a processing condition setting unit 15 that sets processing condition setting values, and processing conditions. The control device includes a control parameter storage unit 13 for storing control parameters, and an initial parameter setting unit 14 for setting initial values of the control parameters. Note that the processing condition set value is a set value that specifies a processing condition.

  The control parameter is a parameter that defines a relationship between the processing condition set value and the processing condition, and a processing condition expressed by a specific physical quantity is determined based on the processing condition set value and the control parameter. Therefore, the processing conditions of the processing pattern when processing by generating an electric discharge between the processing electrode 2 and the workpiece 3 are stored in the processing condition setting value set by the processing condition setting unit 15 and the control parameter holding unit 13. The determination is made based on the held control parameters. That is, machining conditions expressed by physical quantities in the electric discharge machine 1 are controlled by the control parameters. The user can set the processing condition setting value, but cannot set or change the control parameter. The axis drive control unit 11 and the processing power supply control unit 12 issue a command corresponding to the processing pattern of the processing condition based on the information provided from the processing condition setting unit 15 and the control parameter holding unit 13. Although the control parameters are changed as described later, the initial values of the control parameters initially set in the control parameter holding unit 13 are set by the initial parameter setting unit 14. When the control parameter is represented by a correspondence table between the processing condition setting value and the processing condition, the initial value of the control parameter is a correspondence table serving as the initial value.

  The drive device 4 controls the relative distance and the relative speed between the processing electrode 2 and the workpiece 3 based on the command from the shaft drive control unit 11. The machining power supply 5 applies a voltage between the machining electrode 2 and the workpiece 3 based on the command from the machining power supply control unit 12 to control a current waveform at the time of discharge.

  The control device 10 further includes an input / output unit 20, a machine learning device 100, a parameter changing unit 50, and a learning result storage unit 80.

  The input / output unit 20 is an input / output interface that accepts a user's input and supports the user's confirmation work by display. The input / output unit 20 includes a processing condition input unit 21 that receives a processing condition setting value that the user wants the processing condition setting unit 15 to set, and a display unit 22 that allows the user to perform a checking operation for observing the processing state.

  The machine learning device 100 includes a state observation unit 30 and a learning unit 40. The state observation unit 30 includes an axis drive recognition unit 31, a pulse state recognition unit 32, and a machining state observation unit 33. The learning unit 40 includes a reward calculation unit 47 and a function update unit 48, and optimizes by learning control parameters.

  The reward calculator 47 calculates a reward for voltage control, a first reward calculator for calculating a reward for pulse control, and a third reward for calculating a reward for axis drive control. A calculation unit 45. The function updating unit 48 includes a first function updating unit 42 for updating a function relating to voltage control, a second function updating unit 44 for updating a function relating to pulse control, and a third function updating a function relating to axis driving control. And an updating unit 46.

  The parameter changing unit 50 includes: a first control parameter changing unit 51 that changes a control parameter of a machining condition related to voltage control; a second control parameter changing unit 52 that changes a control parameter of a machining condition related to pulse control; A third control parameter changing unit 53 that changes a control parameter of a processing condition related to control; The parameter changing unit 50 changes the control parameters held by the control parameter holding unit 13 based on the result learned by the learning unit 40.

  The learning result storage unit 80 stores the learning result of the machine learning device 100.

  When the electric discharge machine 1 starts electric discharge machining, the axis drive control unit 11 and the machining power supply control unit 12 issue commands based on the machining condition setting values output by the machining condition setting unit 15, and the drive unit 4 and the machining power supply 5 By the above operation, a discharge is generated between the machining electrode 2 and the workpiece 3.

  During the electric discharge machining, the driving device 4 adjusts the optimal relative distance at which electric discharge occurs while decreasing or increasing the relative distance between the machining electrode 2 and the workpiece 3 in accordance with a command from the shaft drive control unit 11. Explore. The information on the position of the drive shaft and the operation of the drive shaft at this time is acquired by the shaft drive recognition unit 31 and recorded in the machining state observation unit 33 as the shaft behavior history.

  During the electric discharge machining, the machining power supply 5 applies a voltage between the machining electrode 2 and the workpiece 3 at the same time as the operation of the driving device 4 according to a command from the machining power supply control unit 12. Then, a current pulse having a current waveform of a commanded shape is generated. The processing power supply control unit 12 controls the voltage of the processing power supply 5 based on the processing condition setting value from the processing condition setting unit 15 so as to generate a current pulse having a current waveform of a specified shape at a constant cycle. However, due to physical characteristics, it is impossible to reliably generate a current pulse at a constant cycle in electric discharge machining. In some cases, the shape of the current pulse may be different from the current waveform indicated by the theoretical value. The pulse state recognizing unit 32 acquires the information about the generation cycle of the current pulse and the shape of the current pulse, and the voltage waveform indicating the magnitude and the application cycle of the applied voltage and the voltage pulse shape that are the source of the pulse generation. Then, it is recorded as a pulse behavior history in the machining state observation unit 33.

  The machining state observation unit 33 obtains, from the pulse behavior history and the axis behavior history, the distribution of the voltage value in a certain period, the generation cycle of the current pulse, the position information of the axis when the current pulse is generated, the speed information, and the acceleration information. The machining state observation unit 33 compares the information obtained by the electric discharge machining executed under the currently used control parameters with the currently used control parameters set in the control parameter holding unit 13. It is provided to the learning unit 40 in association with the information.

  Hereinafter, the processing conditions and the relationship between the processing conditions and the control parameters will be described in detail. FIG. 2 is a diagram in which machining conditions according to the first embodiment are classified for control purposes. FIG. 3 is a diagram illustrating control parameters of machining conditions related to voltage control according to the first embodiment. FIG. 4 is a diagram illustrating a relationship between processing conditions related to voltage control according to the first embodiment and a generation cycle of a current pulse. FIG. 5 is a diagram for explaining control parameters of processing conditions related to the pulse control according to the first embodiment. FIG. 6 is a diagram illustrating a relationship between processing conditions related to pulse control and the shape of a current pulse according to the first embodiment. FIG. 7 is a diagram for explaining control parameters of machining conditions related to the axis drive control according to the first embodiment. FIG. 8 is a diagram illustrating a relationship between machining conditions related to the shaft drive control and the gap control by the shaft drive according to the first embodiment.

  In FIG. 2, (1) type of processing circuit, (2) circuit auxiliary setting, (3) current pulse peak value, (4) current pulse length, (5) pulse pause time, (6) gap adjustment value between poles , (7) jump speed, (8) jump height, (9) maximum depth duration, (10) axis responsiveness, (11) target voltage value, etc. In the case where the processing condition related to the above or the processing condition related to the axis drive control is satisfied, a black circle is attached to the corresponding column. The processing conditions related to the voltage control relate to the generation cycle of the current pulse, the processing conditions related to the pulse control relate to the shape of the current pulse, and the processing conditions related to the shaft drive control relate to the gap control.

  FIGS. 3, 5, and 7 show control parameters held by the control parameter holding unit 13 corresponding to the processing conditions for which the processing condition setting unit 15 sets the processing condition set values. Although each processing condition shown in FIG. 2 relates to the generation cycle of the current pulse, the shape of the current pulse, and the gap control, the processing conditions may be overlapped. Therefore, changing the associated processing parameters to change any of the current pulse generation period, the shape of the current pulse, or the gap control may also affect others.

  A notch is specified for each processing condition by a processing condition setting value, and a plurality of processing conditions are expressed as a notch pattern which is a combination of the notch specified for each processing condition. The notch is a step that discretely specifies a physical quantity indicating a processing condition. Usually, several or dozens of notch patterns are registered in the electric discharge machine 1 in advance. Each processing condition has a control parameter that cannot be changed by the user, separately from the selection of the notch based on the processing condition set value. As described above, the processing condition, which is a specific physical quantity, is determined based on the processing condition set value indicating the selection of the notch and the control parameter. Specific examples of the control parameters are the notch division number and the notch distribution value. The notch division number is the number of notches that can be selected under the processing conditions. The notch distribution value is a value of a physical quantity of a processing condition assigned to each notch. However, the control parameters are not limited to these. When the control parameters of each of the eleven types of machining conditions shown in FIG. 2 are taken as variables, the total number of the variables ranges from tens to hundreds.

  FIG. 3 illustrates control parameters of processing conditions related to voltage control. FIG. 4 shows machining conditions related to voltage control, (4) current pulse length, (5) pulse pause time, (6) gap gap adjustment value, (9) deepest value duration, (10) axis response. And (11) how the target voltage value relates to the current pulse generation cycle. The (4) current pulse length and (5) pulse pause time are processing conditions indicated by the width indicated by the arrow in FIG. 4, (6) the gap jump adjustment value, (9) the deepest value duration, and (10) ) The axis response and (11) the target voltage value are machining conditions related to the current pulse generation cycle.

  To give a specific example, (4) the control parameters of the current pulse length are the number of notch divisions and the notch distribution values which are the length control parameters. It is assumed that notch division numbers and notch distribution values, which are certain control parameters, are set for the current pulse length. At this time, the current pulse length = 2 μsec corresponds to the notch designated by the machining condition set value 0, and the current pulse length = 4 μsec corresponds to the notch designated by the machining condition set value 1. The corresponding relationship that the current pulse length = 8 μsec corresponds to the notch designated by the value 2 is defined by the control parameter. When the control parameter of the current pulse length is changed, the correspondence is changed, so that the current pulse length for the same processing condition set value is changed. However, even if the control parameters are changed, the values of the processing conditions for all the processing condition set values may not be changed depending on the changed notch distribution values.

  FIG. 5 illustrates control parameters of processing conditions related to pulse control. FIG. 6 shows processing conditions related to pulse control, (1) type of processing circuit, (2) circuit auxiliary setting, (3) current pulse peak value, (4) current pulse length, and (6) gap between poles. (11) An outline of how the adjustment value and the target voltage value relate to the shape of the current pulse is shown. (1) When the circuit call parameter of the type of the processing circuit is changed, the processing circuit is changed, so that the shape of the current pulse changes. (2) The circuit auxiliary setting defines the slope of the rise of the current pulse. (3) The current pulse peak value defines the peak value of the current pulse. (4) The current pulse length defines the pulse length of the current pulse. The (6) gap adjustment value and (11) target voltage value are processing conditions related to the interval between current pulses.

To give a specific example, (3) the control parameters of the current pulse peak value are the number of notch divisions and the notch distribution values which are peak control parameters. It is assumed that a certain control parameter, the notch division number and the notch distribution value, are set for the current pulse peak value _Ip . At this time, _Ip = 1A corresponds to the notch specified by the processing condition set value 0, _Ip = 2A corresponds to the notch specified by the processing condition set value 1, and the processing condition set value 2 is specified. The corresponding relationship such that I _p = 4A corresponds to the notch to be set is defined by the control parameters. When the control parameter of the current pulse peak value _Ip is changed, the correspondence is changed, so that the value of _Ip for the same processing condition setting value is changed. However, even if the control parameters are changed, the values of the processing conditions for all the processing condition set values may not be changed depending on the changed notch distribution values.

  FIG. 7 illustrates control parameters of machining conditions related to axis drive control. FIG. 8 shows machining conditions related to axis drive control, (6) gap adjustment value, (7) jump speed, (8) jump height, (9) deepest value duration, (10) axis response. And (11) schematically show how the target voltage value relates to axis drive control. The (6) gap adjustment value, (9) deepest value duration, (10) axis response, and (11) target voltage value are processing conditions related to the approach operation between the processing electrode 2 and the workpiece 3. is there. (7) Jump speed, (8) Jump height and (10) Axis response are processing conditions related to the retreat operation of the processing electrode 2 from the workpiece 3 including the jump operation of the drive shaft.

  Next, the stability or instability of the voltage pulse and the current pulse in electric discharge machining will be described. FIG. 9 is a diagram illustrating states of a voltage pulse and a current pulse according to the first embodiment. FIG. 10 is a diagram illustrating a case where the voltage pulse and the current pulse according to the first embodiment are stable. FIG. 11 is a diagram illustrating a case where the voltage pulse and the current pulse according to the first embodiment are unstable. 9 to 11, the upper part shows a voltage waveform and the lower part shows a current waveform.

  When a voltage is applied between the processing electrode 2 and the workpiece 3, dielectric breakdown occurs at an unexpected timing, and a current flows. When an ideal relationship between voltage and current for stable processing is generated, a current pulse close to a rectangular wave having a constant slope and formed by a transistor circuit or the like is generated. This current pulse is shown in FIG. 9 as a stable discharge. If such an ideal relationship between the voltage and the current is not satisfied, the current waveform of the current pulse becomes unstable as shown in FIG. It may be like the abnormal discharge shown in FIG. 9 in which an irregularly shaped current waveform is generated.

  In the control of the electric discharge machining, as one index for controlling the relative distance between the electrodes, the control is performed by observing the average voltage value per a certain time when electric discharge occurs. When the ideal relation between voltage and current is maintained, as shown in FIG. 10, the average voltage value is maintained at the theoretical value, and stable discharge continues. However, when the unstable discharge or the abnormal discharge shown in FIG. 9 is repeated, the average voltage value fluctuates from the theoretical value as shown in FIG. 11, and the unstable discharge continues. If the distance between the poles is lost and the machining electrode 2 and the workpiece 3 come into contact with each other, a short circuit occurs, and if the distance between the machining electrode 2 and the workpiece 3 is far enough to cause no discharge, it is opened. Because of the state, the fluctuation of the average voltage value with respect to the theoretical value does not immediately determine the stability or instability of the discharge pulse. Further, even when the stable current pulse pattern shown in FIG. 10 continues to be generated under ideal conditions, there is an unpredictable time interval called a no-load voltage time until dielectric breakdown occurs. The frequency of occurrence is not constant. Therefore, the increase / decrease in the discharge generation cycle is an index independent of the machining stability.

  FIG. 12 is a diagram illustrating an ideal average voltage value distribution according to the first embodiment. FIG. 13 is a diagram showing the distribution of the average voltage value when the stable discharge according to the first embodiment continues. FIG. 14 is a diagram showing the distribution of the average voltage value when the unstable discharge according to the first embodiment continues. 12 to 14, the horizontal axis indicates the average voltage value per fixed time when the discharge occurs, and the vertical axis indicates the number of pulses per fixed time.

  When the ideal relationship between the voltage and the current is maintained, the average voltage value is the number of pulses determined in the theoretical value, as shown in FIG. In actual processing, due to physical phenomena, an average voltage value and a pulse number are distributed around a target voltage value which is a processing condition indicating a target voltage value. The target voltage value does not need to be a theoretical value. When the machining is stable and the stable discharge continues as shown in FIG. 10, the variation in the average voltage value is small as shown in FIG. 13, and the pulse number becomes maximum at the average voltage value at the target voltage value. I have. When the machining is unstable and the unstable discharge continues as shown in FIG. 11, the average voltage value is dispersed around the target voltage value and greatly varies as shown in FIG. 14, and the number of pulses also varies. Would.

  The pulse state recognizing unit 32 determines whether the pulse is stable or unstable based on the distribution of the voltage when the discharge occurs during a certain period, and determines whether the pulse is stable or unstable. As an example, the pulse state recognition unit 32 determines whether the voltage pulse and the current pulse are stable or unstable based on the relationship between the average voltage value, the target voltage value, and the voltage threshold value obtained from the machining power control unit 12. Is determined. Specifically, based on a command from the machining power supply control unit 12, the pulse state recognition unit 32 determines that the absolute value of the deviation from the target voltage value of the average voltage value over a certain period of time when a discharge occurs is larger than the voltage threshold. In this case, a pulse unstable signal is generated, and a value obtained by accumulating the number of times the unstable signal is generated for a predetermined period longer than the above-mentioned predetermined time is obtained as a value of the first state. Further, the pulse state recognition unit 32 obtains, as a value of the second state, the number of pulses generated in a predetermined period obtained from a command from the machining power supply control unit 12. The predetermined period may be, for example, an operation time until a retreat operation called a jump operation is completed, a gap position control for generating a discharge is performed, and a next jump operation is performed again. it can. The shaft drive recognizing unit 31 obtains, as a value in the third state, a feed amount of the shaft of the drive device 4 obtained from the command of the shaft drive control unit 11. The value in the third state is set to a larger positive value as the feed amount of the shaft increases in the machining progress direction, and to a larger negative value as the feed amount of the shaft increases in the backward direction. The value of the first state, the value of the second state, and the value of the third state are state variables representing the machining state during electric discharge machining, and the machining state observation unit 33 determines the first state, which is a plurality of acquired state variables. , The value of the second state, and the value of the third state are displayed on the display unit 22 so that the user can visually observe them in the form of a histogram such as a distribution chart or a bar graph. In this way, the state observation unit 30 observes the values of the first state, the second state, and the third state, which are a plurality of state variables. Then, the first reward calculator 41, the second reward calculator 43, and the third reward calculator 45 of the learning unit 40 calculate the first state value, the second state value, and the third state value acquired by the machining state observation unit 33. Calculate rewards based on state values.

  The learning algorithm used by the machine learning device 100 including the state observation unit 30, the learning unit 40, and the parameter changing unit 50 may use any learning algorithm. As an example, a case where reinforcement learning (Reinforcement Learning) is applied will be described.

  In reinforcement learning, an agent acting as an agent in a certain environment observes the current state and determines an action to be taken. The agent obtains rewards from the environment by selecting an action, and learns a strategy to obtain the highest reward through a series of actions. As a typical method of reinforcement learning, Q learning (Q-learning) or TD learning (TD-learning) is known. For example, in the case of Q learning, a general update equation of the action value function Q (s, a) is represented by the following equation (1). The action value function Q (s, a) is also called an action value table.

In Equation (1), s _t represents the state at time t, a _t represents the behavior in time t. By the action a _t, the state is changed to s _{t + 1.} rt _{+ 1} represents a reward obtained by the change of the state, γ represents a discount rate, and α represents a learning coefficient.

Represented update equation in Equation (1) in the Q learning, action value of the best action a at time t + 1 is greater than the action value Q of the executed action a _t at time t, activation level at time t Q is increased, and conversely, the action value Q at time t is decreased. In other words, the action value Q action a _t at time t, as close to the best action value at time t + 1, action value function Q (s _t, a _t) Update. As a result, the best action value in a certain environment sequentially propagates to the action value in an earlier environment.

Accordingly, in the operation of the machine learning system 100 to be described below, the change behavior of the control parameters and behavior a _t at time t, the first, if the state s _t at the second and third states the time t, Q You can understand that you are learning.

  Hereinafter, the operation of optimizing the control parameters by the machine learning device 100 will be described.

  FIG. 15 is a flowchart illustrating an optimization process based on learning of control parameters of processing conditions related to voltage control according to the first embodiment. The control parameters of the processing conditions related to the voltage control are variable values for performing the voltage control that are the basis of the target voltage value set as the processing conditions, and thus, not only the magnitude of the voltage but also the shape of the voltage waveform , A voltage reference value called a reference voltage for detecting discharge. In addition, by optimizing the control parameters of the machining conditions related to the voltage control, processing such as changing the initial notch pattern of the no-load voltage time and the target voltage value set as the control parameters to another notch pattern is also performed. Will be

  The first reward calculating unit 41 that calculates the reward for the voltage control calculates the amount of change in the reward based on the value of the first state and the value of the second state, which are the state variables obtained by the pulse state recognition unit 32. If the first reward calculation unit 41 calculates the change amount of the reward so as to increase the reward when the value of the first state is small and the value of the second state is large, the first reward value and the second reward value are calculated. There is no restriction on how the state value is used to determine the amount of change in reward. Specifically, the reward is increased when the value in the first state is small, and the reward is reduced when the value in the first state is large. In addition, the reward is increased when the value of the second state is increased, and the reward is decreased when the value of the second state is decreased. In addition to the basic criterion of increasing the reward when the number of unstable pulses decreases and the number of stable pulses increases, the number of stable pulses decreases even if the number of unstable pulses decreases. In such a case, the method of calculating the reward may be determined so that the reward is reduced.

  Based on the reward calculated by the first reward calculation unit 41, the first function update unit 42 updates an action value function Q that is a function for determining a control parameter related to voltage control. Based on the updated action value function Q, the first control parameter changing unit 51 changes the control parameter of the processing condition related to the voltage control so that the control parameter is such that the reward is obtained most.

  Based on the above, the optimization of the six control parameters of the processing conditions related to the voltage control shown in FIG. 3 will be described with reference to FIG. FIG. 15 illustrates that the control is executed in a situation where the electric discharge machine 1 is continuously performing the electric discharge machining, and that the control parameters to be changed are set to priorities. You may make it.

  Before the flowchart of FIG. 15 is executed, it is assumed that the first reward calculation unit 41 already holds the initial value of the reward for voltage control. The initial value of the reward is not limited and may be 0 if it is a fixed value. First, the state observation unit 30 observes information of the machining power supply control unit 12 when machining is being performed with current machining conditions and control parameters (step S101). Specifically, the state observation unit 30 acquires a command from the machining power supply control unit 12 during machining. Then, the pulse state recognition unit 32 calculates the value of the first state and the value of the second state based on the command from the processing power supply control unit 12 (Step S102). Next, the value of the first state and the value of the second state, which are the state variables obtained by the pulse state recognition unit 32, are provided from the machining state observation unit 33 to the first reward calculation unit 41. Here, the value of the first state and the value of the second state are linked to the currently used control parameter set in the control parameter holding unit 13 and are transmitted from the processing state observation unit 33 to the first reward calculation unit 41. Given to.

  Then, the first reward calculation unit 41 compares the given value in the first state with the previous value in the first state (step S103). The first reward calculation unit 41 holds the value of the first state given last time and can compare it with the value of the first state given this time. When the value of the first state is smaller than the value of the previous first state (step S103: small), the first reward calculation unit 41 increases the reward (step S104). That is, if the value of the first state indicates a state that is more stable than the previous state, the reward is increased. The reward increase value here is a predetermined value. When the value of the first state is the same as the value of the previous first state (step S103: the same), the first reward calculation unit 41 does not change the reward (step S105). When the value of the first state is larger than the value of the previous first state (step S103: large), the first reward calculation unit 41 reduces the reward (step S106). That is, if the value of the first state indicates a more unstable state than the previous state, the reward is reduced. The reduction value of the reward here is a predetermined value. When step S103 is executed for the first time, there is no value of the first state given last time, so the process proceeds to step S105.

  Next, the first reward calculation unit 41 compares the given value in the second state with the previous value in the second state (step S107). The first reward calculation unit 41 holds the value of the second state given last time and can compare it with the value of the second state given this time. When the value of the second state is larger than the value of the previous second state (step S107: large), the first reward calculation unit 41 increases the reward (step S108). That is, if the value of the second state indicates a state that is more stable than the previous state, the reward is increased. The reward increase value here is a predetermined value. When the value of the second state is the same as the value of the previous second state (step S107: the same), the first reward calculation unit 41 does not change the reward (step S109). When the value in the second state is smaller than the value in the previous second state (step S107: small), the first reward calculation unit 41 reduces the reward (step S110). That is, when the value of the second state indicates a more unstable state than the previous state, the reward is reduced. The reduction value of the reward here is a predetermined value. When step S107 is executed for the first time, the process proceeds to step S109 because there is no second state value given last time.

  Then, the first function updating unit 42 updates the action value function Q according to the formula (1) based on the reward calculated by the first reward calculating unit 41 (Step S111). Furthermore, the first function update unit 42 determines whether or not the update is not performed in step S111 and the action value function Q has converged (step S112). When it is determined that the action value function Q has not converged (step S112: No), the first control parameter changing unit 51 determines the processing condition related to the voltage control based on the action value function Q updated in step S111. Is changed (step S113). After step S113, the process returns to step S101. When it is determined that the action value function Q has converged (Step S112: Yes), the learning unit 40 determines whether or not all the control parameters of the processing conditions related to the voltage control have been changed by the first control parameter changing unit 51. Is determined (step S114). If it is determined that all the control parameters of the processing conditions related to the voltage control have not been changed (step S114: No), the control parameter to be changed by the first control parameter changing unit 51 is changed to another control in step S113. The parameter is replaced (step S115). Another control parameter that has been newly changed in step S115 is a control parameter related to voltage control that has not been changed yet. After step S115, the process proceeds to step S113.

  The change of the control parameter of the processing condition related to the voltage control in step S113 will be described in detail below. As described above, the priority order to be changed is determined for the six control parameters of the processing conditions related to the voltage control shown in FIG. 3 which are changed in step S113. What is changed by the first control parameter changing unit 51 when the process first enters step S113 is a voltage control parameter that is a control parameter of a target voltage value. Each time it is determined in step S112 that the action value function Q has converged, the control parameters to be changed by the first control parameter changing unit 51 are the GAIN control parameter, which is the control parameter of the axial response, and the pulse pause time. Length control parameter which is a control parameter of the gap adjustment value, length control parameter which is a control parameter of the maximum depth duration, and length control which is a control parameter of the current pulse length The parameters are replaced in step S115 in the order of the parameters.

  When the learning unit 40 determines that the action value function Q has converged and all the control parameters of the processing conditions related to the voltage control have been changed (step S114: Yes), the learning unit 40 learns the control parameters of the processing conditions related to the voltage control. The optimization process ends, and the learning result is stored in the learning result storage unit 80 (Step S116). The learning result includes, in addition to the control parameters finally changed and determined in step S113, values of the process of changing each control parameter, and values of the first state and the second state corresponding to the control parameters. Is included. The learning result stored in the learning result storage unit 80 can be used to determine the quality before and after the control parameter is changed. In addition, the control parameters finally determined as described above are most rewarded in the learning, and are stored in the control parameter storage unit 13 as optimal control parameters at the given processing condition set value. By optimizing the control parameters of machining conditions related to voltage control by learning, it is possible to prevent the generation of unstable signals from the start to the end of machining and to maximize the number of stable signal pulses. become. As described above, when optimizing the six types of control parameters of the machining conditions related to the voltage control at the same time, in step S113, the first control parameter changing unit 51 sets the action value function Q updated in step S111. , Six types of control parameters are changed simultaneously. In this case, steps S114 and S115 are unnecessary, and if it is determined in step S112 that the action value function Q has converged (step S112: Yes), the process may proceed to step S116 immediately.

  FIG. 16 is a flowchart illustrating an optimization process based on learning of control parameters of machining conditions related to pulse control according to the first embodiment. The control parameters of the processing conditions related to the pulse control are variable values for performing the current pulse control, such as the pulse inclination and the abnormal discharge detection threshold value that is the basis of the theoretical value of the pulse generation period. The control parameters of the processing conditions related to the pulse control include not only the magnitude and width of the current pulse but also the shape of the current waveform and the relative distance between the processing electrode 2 and the workpiece 3 for making the current value closer to an ideal shape. The adjustment value of the gap between the poles for adjusting the gap is also included. Further, by optimizing the control parameters of the processing conditions related to the pulse control, processing such as changing the initial notch pattern of the magnitude and width of the current set as the control parameter to another notch pattern is also performed.

  The second reward calculating unit 43 that calculates the reward for the pulse control calculates the reward based on the value of the first state and the value of the second state, which are the state variables obtained by the pulse state recognition unit 32. The method of calculating the reward in the second reward calculator 43 is the same as that in the first reward calculator 41.

  Based on the reward calculated by the second reward calculation unit 43, the second function update unit 44 updates an action value function Q that is a function for determining a control parameter related to pulse control. Based on the updated action value function Q, the second control parameter changing unit 52 changes the control parameter of the processing condition related to the pulse control so that the control parameter is such that the reward is obtained most.

  Based on the above, the optimization of the six control parameters of the processing conditions related to the pulse control shown in FIG. 5 will be described with reference to FIG. FIG. 16 illustrates that the process is executed in a situation where the electric discharge machine 1 continuously performs the electric discharge machining, and the control parameters to be changed are set to the priority order. You may make it.

  Before the flowchart of FIG. 16 is executed, it is assumed that the second reward calculating unit 43 holds the initial value of the reward related to the pulse control. The initial value of the reward is not limited and may be 0 if it is a fixed value. First, the state observation unit 30 observes information of the machining power supply control unit 12 when machining is being performed with the current machining conditions and control parameters (step S201). Specifically, the state observation unit 30 acquires a command from the machining power supply control unit 12 during machining. Then, based on a command from the machining power control unit 12, the pulse state recognition unit 32 calculates a value in the first state and a value in the second state (Step S202). Next, the value of the first state and the value of the second state, which are the state variables obtained by the pulse state recognition unit 32, are provided from the processing state observation unit 33 to the second reward calculation unit 43. Here, the value of the first state and the value of the second state are linked to the currently used control parameter set in the control parameter holding unit 13 and are transmitted from the processing state observation unit 33 to the second reward calculation unit 43. Given to.

  Then, the second reward calculation unit 43 compares the given value in the first state with the previous value in the first state (step S203). The second reward calculation unit 43 holds the value of the first state given last time and can compare it with the value of the first state given this time. When the value of the first state is smaller than the value of the previous first state (Step S203: small), the second reward calculation unit 43 increases the reward (Step S204). The reward increase value here is a predetermined value. When the value of the first state is the same as the value of the previous first state (step S203: the same), the second reward calculation unit 43 does not change the reward (step S205). When the value of the first state is larger than the value of the previous first state (step S203: large), the second reward calculation unit 43 reduces the reward (step S206). The reduction value of the reward here is a predetermined value. When step S203 is executed for the first time, there is no value of the first state given last time, and therefore, the process proceeds to step S205.

  Next, the second reward calculation unit 43 compares the given value in the second state with the previous value in the second state (step S207). The second reward calculation unit 43 holds the value of the second state given last time and can compare it with the value of the second state given this time. When the value of the second state is larger than the value of the previous second state (step S207: large), the second reward calculation unit 43 increases the reward (step S208). The reward increase value here is a predetermined value. If the value of the second state is the same as the value of the previous second state (step S207: the same), the second reward calculation unit 43 does not change the reward (step S209). When the value of the second state is smaller than the value of the previous second state (step S207: small), the second reward calculation unit 43 reduces the reward (step S210). The reduction value of the reward here is a predetermined value. When step S207 is executed for the first time, the process proceeds to step S209 because there is no second state value given last time.

  Then, the second function updating unit 44 updates the action value function Q according to the formula (1) based on the reward calculated by the second reward calculating unit 43 (Step S211). Further, the second function update unit 44 determines whether or not the update is not performed in step S211 and the action value function Q has converged (step S212). When it is determined that the action value function Q has not converged (step S212: No), the second control parameter changing unit 52 performs processing conditions related to pulse control based on the action value function Q updated in step S211. Is changed (step S213). After step S213, the process returns to step S201. When it is determined that the action value function Q has converged (Step S212: Yes), the learning unit 40 determines whether or not all the control parameters of the processing conditions related to the pulse control have been changed by the second control parameter changing unit 52. Is determined (step S214). If it is determined that all of the control parameters of the processing conditions related to the pulse control have not been changed (step S214: No), the control parameter to be changed by the second control parameter changing unit 52 is changed to another control in step S213. The parameter is replaced (step S215). Another control parameter that has been newly changed in step S215 is a control parameter related to pulse control that has not been changed yet. After step S215, the process proceeds to step S213.

  The change of the control parameter of the processing condition related to the pulse control in step S213 will be described in detail below. As described above, the priority order to be changed is determined for the six control parameters of the processing conditions related to the pulse control shown in FIG. 5 that are changed in step S213. What is changed by the second control parameter changing unit 52 when the process first enters step S213 is a voltage control parameter that is a control parameter of a target voltage value. Each time it is determined that the action value function Q has converged in step S212, the control parameter to be changed by the second control parameter changing unit 52 is a gap control parameter or a circuit A pulse tilt control parameter which is a control parameter for auxiliary setting, a length control parameter which is a control parameter for a current pulse length, a peak control parameter which is a control parameter for a current pulse peak value, and a circuit call which is a control parameter for a type of machining circuit. The parameters are replaced in step S215 in the order of the parameters.

  When the learning unit 40 determines that the action value function Q has converged and all the control parameters of the processing conditions related to the pulse control have been changed (step S214: Yes), the learning unit 40 learns the control parameters of the processing conditions related to the pulse control. The optimization process ends, and the learning result is stored in the learning result storage unit 80 (Step S216). The learning result includes, in addition to the control parameters finally changed and determined in step S213, values of the process of changing the control parameters, and values of the first state and the second state corresponding to the control parameters. Is included. The learning result stored in the learning result storage unit 80 can be used to determine the quality before and after the control parameter is changed. In addition, the control parameters finally determined as described above are most rewarded in the learning, and are stored in the control parameter storage unit 13 as optimal control parameters at the given processing condition set value. By optimizing the control parameters of the processing conditions related to pulse control by learning, it is possible to prevent the generation of unstable signals from the start to the end of processing and to maximize the number of stable signal pulses. become. As described above, when simultaneously optimizing the six types of control parameters of the processing conditions related to the pulse control, in step S213, the second control parameter changing unit 52 sets the action value function Q updated in step S211. , Six types of control parameters are changed simultaneously. In this case, steps S214 and S215 are unnecessary, and if it is determined in step S212 that the action value function Q has converged (step S212: Yes), the process may proceed to step S216 immediately.

  FIG. 17 is a flowchart illustrating an optimization process by learning control parameters of machining conditions related to the axis drive control according to the first embodiment. The control parameter of the machining condition related to the axis drive control is also called a gap control parameter, a deceleration distance when the machining electrode 2 and the workpiece 3 are brought close to each other, a speed at which an instantaneous evacuation behavior called a jump operation is generated, and It is a variable value for changing the axis driving behavior of the electric discharge machine 1, such as an acceleration parameter. Changes in the gap control parameters include not only changes in the shaft responsiveness for generating stable discharge between the poles, but also changes in the jump operation for cleaning the gaps, and natural frequency oscillations due to overresponse of the shaft. This also includes changing parameters to prevent vibrations due to vibration.

  The third reward calculation unit 45 that calculates the reward for the axis drive control includes a second state value that is a state variable obtained by the pulse state recognition unit 32 and a third state that is a state variable obtained by the axis drive recognition unit 31. Calculate the reward based on the value of. The third reward calculation unit 45 calculates the change amount of the reward so as to increase the reward when the value of the second state is large and the value of the third state is large. There is no restriction on how the state value is used to determine the amount of change in reward. Specifically, the reward is increased when the value of the second state is large, and the reward is reduced when the value of the second state is small. In addition to this, the reward is increased when the value of the third state increases, and the reward is decreased when the value of the third state decreases. Also, if the number of discharge pulses increases even if there is no change in the feed amount of the shaft, the reward is increased, but if the number of discharge pulses decreases even if the feed amount of the shaft increases in the machining progress direction. May determine the method of calculating the reward so that the reward decreases.

  Based on the reward calculated by the third reward calculation unit 45, the third function update unit 46 updates an action value function Q that is a function for determining a control parameter related to axis drive control. Based on the updated action value function Q, the third control parameter changing unit 53 changes the control parameters of the machining conditions related to the axis drive control so that the control parameter is the one that gives the most reward.

  Based on the above, the optimization of the five types of control parameters of the machining conditions related to the shaft drive control shown in FIG. 7 will be described with reference to FIG. FIG. 17 illustrates that the process is performed in a situation where the electric discharge machine 1 is continuously performing the electric discharge machining, and that the control parameters to be changed have a priority order. You may make it.

  Before the flowchart in FIG. 17 is executed, it is assumed that the third reward calculation unit 45 has already held the initial value of the reward for the axis drive control. The initial value of the reward is not limited and may be 0 if it is a fixed value. First, the state observation unit 30 observes information of the machining power supply control unit 12 when machining is being performed with current machining conditions and control parameters (step S301). Specifically, the state observation unit 30 acquires a command from the machining power supply control unit 12 during machining. Then, based on the command from the machining power control unit 12, the pulse state recognition unit 32 calculates the value of the second state (Step S303). Further, the state observation unit 30 observes information of the axis drive control unit 11 when the machining is being performed with the current machining conditions and control parameters (step S302). Specifically, the state observation unit 30 acquires a command from the axis drive control unit 11 during processing. Then, based on a command from the shaft drive control unit 11, the shaft drive recognition unit 31 calculates a value in the third state (step S304). Next, the value of the second state obtained by the pulse state recognition unit 32 and the value of the third state obtained by the axis drive recognition unit 31 are provided from the machining state observation unit 33 to the third reward calculation unit 45. Here, the value of the second state and the value of the third state are linked to the currently used control parameter set in the control parameter holding unit 13 and are transmitted from the processing state observation unit 33 to the third reward calculation unit 45. Given to.

  Then, the third reward calculation unit 45 compares the given value of the second state with the value of the previous second state (step S305). The third reward calculation unit 45 holds the value of the second state given last time and can compare it with the value of the second state given this time. When the value in the second state is larger than the value in the previous second state (step S305: large), the third reward calculation unit 45 increases the reward (step S306). The reward increase value here is a predetermined value. If the value of the second state is the same as the value of the previous second state (step S305: the same), the third reward calculation unit 45 does not change the reward (step S307). If the value of the second state is smaller than the value of the previous second state (step S305: small), the third reward calculation unit 45 reduces the reward (step S308). The reduction value of the reward here is a predetermined value. When step S305 is executed for the first time, the process proceeds to step S307 because the value of the second state given last time does not exist.

  Next, the third reward calculation unit 45 compares the given value in the third state with the previous value in the third state (step S309). The third reward calculation unit 45 holds the value of the third state given last time and can compare it with the value of the third state given this time. When the value of the third state is larger than the value of the previous third state (step S309: large), the third reward calculation unit 45 increases the reward (step S310). That is, if the value of the third state indicates a more stable state than the previous state, the reward is increased. The reward increase value here is a predetermined value. If the value of the third state is the same as the value of the previous third state (step S309: the same), the third reward calculation unit 45 does not change the reward (step S311). When the value in the third state is smaller than the value in the previous third state (step S309: small), the third reward calculation unit 45 reduces the reward (step S312). That is, if the value of the third state indicates a more unstable state than the previous state, the reward is reduced. The reduction value of the reward here is a predetermined value. When step S309 is executed for the first time, there is no third state value given last time, and therefore, the process proceeds to step S311.

  Then, the third function updating unit 46 updates the action value function Q according to the formula (1) based on the reward calculated by the third reward calculating unit 45 (Step S313). Further, the third function update unit 46 determines whether or not the update is not performed in step S313 and the action value function Q has converged (step S314). If it is determined that the action value function Q has not converged (step S314: No), the third control parameter changing unit 53 performs processing related to axis drive control based on the action value function Q updated in step S313. The control parameter of the condition is changed (step S315). After step S315, the process returns to steps S301 and S302. When it is determined that the action value function Q has converged (step S314: Yes), the learning unit 40 determines whether or not all the control parameters of the machining conditions related to the axis drive control have been changed by the third control parameter changing unit 53. It is determined (step S316). If it is determined that all of the control parameters of the machining conditions related to the axis drive control have not been changed (step S316: No), the control parameter to be changed by the third control parameter changing unit 53 is changed to another in step S315. The control parameter is replaced (step S317). Another control parameter that has been newly changed in step S317 is a control parameter related to axis drive control that has not been changed yet. After step S317, the process proceeds to step S315.

  The change of the control parameter of the machining condition related to the axis drive control in step S315 will be described in detail below. As described above, the priority order to be changed is determined for the five control parameters of the machining conditions related to the axis drive control shown in FIG. 7 that are changed in step S315. What is changed by the third control parameter changing unit 53 when the process first enters step S315 is a voltage control parameter which is a control parameter of a target voltage value. Each time it is determined in step S314 that the action value function Q has converged, the control parameters to be changed by the third control parameter changing unit 53 are the GAIN control parameter, which is the control parameter of the axial response, and the deepest value duration. The length control parameter, which is a control parameter for time, the jump control parameter, which is a control parameter for the jump speed and the jump height, and the gap control parameter, which is a control parameter for the gap adjustment value between poles, are replaced in this order in step S317.

  When the learning unit 40 determines that the action value function Q has converged and all the control parameters of the machining conditions related to the axis drive control have been changed (step S316: Yes), the learning parameters of the machining parameters related to the axis drive control are changed. The optimization process based on the learning ends, and the learning result is stored in the learning result storage unit 80 (step S318). The learning result includes, in addition to the control parameters changed and finally determined in step S315, the value of the process of changing each control parameter, and the value of the second state and the value of the third state corresponding to the control parameter Is included. The learning result stored in the learning result storage unit 80 can be used to determine the quality before and after the control parameter is changed. In addition, the control parameters finally determined as described above are most rewarded in the learning, and are stored in the control parameter storage unit 13 as optimal control parameters at the given processing condition set value. By optimizing the control parameters of machining conditions related to axis drive control by learning, the retreat operation called jump operation is completed, the gap position control for generating electric discharge is performed, and the next jump operation is performed again. By increasing the number of pulses in one operation unit, such as until the operation is performed, it becomes possible to increase the feed amount of the axis for each observation in the processing progress direction to accelerate the processing. As described above, when simultaneously optimizing five types of control parameters of machining conditions related to axis drive control, in step S315, the third control parameter changing unit 53 sets the action value function updated in step S313. Based on Q, five kinds of control parameters are changed simultaneously. In this case, steps S316 and S317 are unnecessary, and if it is determined in step S314 that the action value function Q has converged (step S314: Yes), the process may proceed to step S318 immediately.

Further, in the description of FIG. 17 from FIG 15, the method of changing the control parameter is changed based on the updated action value function Q is action value in the current state s _t function Q (s _t, a _t) action value Q sought is not particularly limited as long as the manner for obtaining the action a _t or control parameters such as the maximum.

  Since the control parameters under the same machining conditions are the same, when the flowcharts of FIGS. 15 to 17 are executed in parallel, the same control parameters are changed by the respective flowcharts.

  The operation of optimizing the control parameters according to the flowcharts of FIGS. 15 to 17 is performed from the stage when the electric discharge machine 1 starts the machining operation by the electric discharge machine 1, and is continued until the electric discharge machining ends. That is, the state of the machining is observed by the state observation unit 30 at the same time as the start of the machining, and the learning unit 40 and the parameter changing unit 50 search for the optimal control parameters until the end of the machining. That is, the flowcharts of FIGS. 15 to 17 are executed in parallel by the machine learning device 100, and the update of the control parameters continues until all the end conditions of FIGS. 15 to 17 are satisfied. The change of the control parameter ends when all the end conditions are satisfied.

  The above learning behavior by the machine learning device 100 is continuously performed from the start of electric discharge machining to the end of machining. A reward in the learning action is obtained based on the first, second, and third states, and a control parameter changing action is performed. As a result of this learning action, the action value Q based on the optimal control parameters obtained after the end of the processing is higher than the action value Q based on the control parameters initially set. The action value Q is increased by the electric discharge machine 1 according to the first embodiment, so that the time required for the end of machining is reduced, and the machining accuracy and machining surface quality of the workpiece obtained by machining with stable electric discharge are improved. Improvement is obtained as an effect.

  In the conventional adaptive control, processing condition setting values are controlled according to rules determined to stabilize processing, but adaptive control for changing control parameters is not performed. On the other hand, according to the machine learning device 100, the optimization learning for adjusting the control parameters is executed while actually performing the electric discharge machining according to the workpiece shape and the machining material. Stable machining conditions can be learned automatically. That is, according to the machine learning device 100, optimization of control parameters without limiting the applicable range of adaptive control, even under conditions of adaptive control that are difficult to predict in advance, such as the shape of the workpiece, the electrode material, and the electrode shape. Can be performed, the processing stability can be enhanced, and the processing speed and processing accuracy can be improved.

Embodiment 2 FIG.
FIG. 18 is a block diagram showing a configuration of an electric discharge machine 1A according to a second embodiment of the present invention. In the electric discharge machine 1A, a machining result input unit 23, which is a configuration for performing additional learning using a machining result, is added to the input / output unit 20 in the electric discharge machine 1 according to the first embodiment.

  In the first embodiment, the learning behavior of the control parameter when performing a specific processing has been described. In the second embodiment, however, the processing is performed once in advance with the same material of the workpiece 3 and the same processing condition set value. It has been done. It is assumed that as a result of the processing once, the surface roughness of the workpiece 3 after processing and the consumption weight or the consumption length, which is the amount of electrode consumption after the processing of the processing electrode 2, are obtained.

  The processing result input unit 23 receives processing results such as the surface roughness of the workpiece 3 after processing and the amount of electrode consumption after processing of the processing electrode 2 input by the user. The processing result input format may be a format in which the display unit 22 displays selectable options and the processing result input unit 23 accepts the user's selection result. Further, the processing result input unit 23 may accept numerical data on the surface roughness of the workpiece 3 after processing and the amount of electrode consumption after processing of the processing electrode 2 input by the user, and the present invention is not limited thereto. Further, the method of evaluating the quality of the surface roughness of the workpiece 3 and the amount of electrode consumption after the processing of the processed electrode 2 received by the processing result input unit 23 is also a design matter and is not particularly limited. Alternatively, the machining result input unit 23 may receive the quality of the surface roughness of the workpiece 3 after machining and the amount of electrode consumption after machining of the machining electrode 2 itself.

  When the machining is performed again with the same machining condition setting as the once performed machining, the machining result in the previous machining received by the machining result input unit 23 is used to change the control parameters described in FIGS. 15 to 17. Can be added or canceled to the amount of change in. On the basis of the machining result received by the machining result input unit 23, the machining state observation unit 33 and the like cause the parameter changing unit 50 to add or cancel the limit on the change amount in the change of the control parameter.

  Specifically, if the evaluation of the quality of the surface roughness of the workpiece 3 after the processing, which is received by the processing result input unit 23, is determined to be poor, the change of the control parameter affecting the processed surface quality is limited. As an example, the parameter changing unit 50 limits the change width of the length control parameter so that the length control parameter, which is the control parameter of the current pulse length, is not changed beyond a certain value.

  If it is determined that the electrode consumption after machining of the machining electrode 2 received by the machining result input unit 23 is small and there is still room, the restriction on the change of the control parameter affecting the electrode consumption is set in the parameter changing unit. 50 is released. As an example, the change width of the pulse slope control parameter in the circuit auxiliary setting is increased to release the change restriction. Conversely, when the electrode consumption is large, the parameter changing unit 50 can further limit the change of the control parameter.

  According to the electric discharge machine 1 </ b> A according to the second embodiment, by accepting the machining result of the machining once performed, the limitation of the change of the control parameter is processed in the same material of the workpiece 3 and the same machining condition set value. It can depend on the result. Thus, in addition to the effects obtained in the first embodiment, an accuracy improvement effect such as an improvement in the surface quality of a processed surface after processing and a cost reduction effect such as a reduction in electrode consumption can be obtained.

Embodiment 3 FIG.
FIG. 19 is a block diagram showing a configuration of an electric discharge machine 1B according to the third embodiment of the present invention. In the electric discharge machine 1B, a communication unit 60 is added to the electric discharge machine 1A according to the second embodiment. The communication unit 60 includes a learning content filing unit 61 that converts the learning result stored in the learning result storage unit 80 into learning result data that can be transmitted, a receiving unit 62 that receives learning result data from outside, and a learning result data. And a transmission unit 63 for transmitting the data to the outside. The receiving unit 62 and the transmitting unit 63 are connected to and can communicate with a cloud server 300 existing outside the electric discharge machine 1B.

  The cloud server 300 is also connected to electric discharge machines 301 to 303 having the same learning function as the control device 10 of the electric discharge machine 1B. Therefore, the electric discharge machine 1 </ b> B can communicate with other electric discharge machines 301 to 303 via the communication unit 60. The cloud server 300 can store not only learning result data of the electric discharge machine 1B but also learning result data of the electric discharge machines 301 to 303. The communication method between the cloud server 300 and the electric discharge machines 1B, 301 to 303 is not particularly limited as long as a known technique is used.

  When the optimization learning of the control parameters as described in the first and second embodiments has already been executed, the learning results stored in the learning result storage unit 80 are transferred to the EDMs 301 to 303 existing outside. The learning content filing unit 61 can convert the learning result data into a format that can be used. The format of the learning result data is not limited as long as it is a data format that can be used as long as it is a control device similar to the control device 10.

  The learning result data created by the learning content filing unit 61 can be stored in the cloud server 300 via the transmission unit 63. The learning result data stored in the cloud server 300 is automatically or actively transmitted to the electric discharge machines 301 to 303, and it is determined whether the electric discharge machines 301 to 303 use the learning result data. It can be determined by the user's judgments 301 to 303.

  By taking the learning result data into the machine learning devices existing in the electric discharge machines 301 to 303, the contents learned by the machine learning device 100 can be used in the electric discharge machines 301 to 303 in the same manner.

  Conversely, learning result data created by learning in the electric discharge machines 301 to 303 can also be used by the control device 10 via the cloud server 300 and the receiving unit 62. At this time, the contents learned or the state of observation learned by the control devices of the electric discharge machines 301 to 303 via the receiving unit 62 can be displayed on the display unit 22.

  As a result, a learning result obtained by a control device of the electric discharge machines 301 to 303 existing outside, such as a remote place, is used by the electric discharge machine 1B, or the machining state of the electric discharge machines 301 to 303 is observed by the electric discharge machine 1B. it can. Further, it is also possible to use the learning result by the electric discharge machine 1B to the electric discharge machines 301 to 303 of the same specification. Therefore, not only the adjustment of one electric discharge machine but also the improvement of the mechanical performance of a plurality of electric discharge machines of the same specification can be performed more efficiently as the number of electric discharge machines of the same specification increases. .

  The machine learning device 100 according to the first to third embodiments is realized by a computer system such as a personal computer or a general-purpose computer. FIG. 20 is a diagram illustrating a hardware configuration when the functions of the machine learning device 100 according to the first to third embodiments are implemented by a computer system. When the functions of the machine learning device 100 are realized by a computer system, the functions of the machine learning device 100 include a CPU (Central Processing Unit) 201, a memory 202, a storage device 203, a display device 204, and an input device 205, as shown in FIG. Is realized by: The functions executed by the machine learning device 100 are realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the storage device 203. The CPU 201 realizes the functions of the machine learning device 100 by reading out software or firmware stored in the storage device 203 into the memory 202 and executing the software or firmware. That is, the computer system stores a program that, when the functions of the machine learning device 100 are executed by the CPU 201, the steps of executing the machine learning method according to the first to third embodiments are executed as a result. Storage device 203 is provided. In addition, it can be said that these programs cause a computer to execute processing realized by the functions of the machine learning device 100. The memory 202 corresponds to a volatile storage area such as a RAM (Random Access Memory). The storage device 203 corresponds to a nonvolatile or volatile semiconductor memory such as a ROM (Read Only Memory) or a flash memory, or a magnetic disk. Specific examples of the display device 204 are a monitor and a display. Specific examples of the input device 205 are a keyboard, a mouse, and a touch panel.

  The configurations described in the above embodiments are merely examples of the contents of the present invention, and can be combined with other known technologies, and can be combined with other known technologies without departing from the gist of the present invention. Parts can be omitted or changed.

  1, 1A, 1B, 301 to 303 EDM, 2 machining electrode, 3 workpiece, 4 drive device, 5 machining power supply, 10 control device, 11 axis drive control unit, 12 machining power control unit, 13 control parameter holding Unit, 14 initial parameter setting unit, 15 processing condition setting unit, 20 input / output unit, 21 processing condition input unit, 22 display unit, 23 processing result input unit, 30 state observation unit, 31 axis drive recognition unit, 32 pulse state recognition Part, 33 machining state observing part, 40 learning part, 41 first reward calculating part, 42 first function updating part, 43 second reward calculating part, 44 second function updating part, 45 third reward calculating part, 46 third Function update unit, 47 reward calculation unit, 48 function update unit, 50 parameter change unit, 51 first control parameter change unit, 52 second control parameter change unit, 53 third control parameter Change unit, 60 communication unit, 61 learning content filing unit, 62 receiving unit, 63 transmitting unit, 80 learning result storage unit, 100 machine learning device, 201 CPU, 202 memory, 203 storage device, 204 display device, 205 input device , 300 cloud server.

Claims (6)

A machine learning device for learning control parameters for controlling machining conditions in an electric discharge machine,
A state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining,
A learning unit that learns the control parameter based on a plurality of the state variables;
With
The state observation unit includes a first state value that is a value obtained by accumulating the number of times of generation of an unstable signal of a pulse during a predetermined period, and a second state value that is a number of pulses generated in the predetermined period. Observing the value and the value of the third state, which is the feed amount of the shaft in the driving device, as a plurality of the state variables,
The learning unit includes a reward calculation unit that calculates a reward based on the state variable, and a function update unit that updates a function for determining the control parameter based on the reward.
The reward calculator includes a first reward calculator that calculates a reward for voltage control, a second reward calculator that calculates a reward for pulse control, and a third reward calculator that calculates a reward for axis drive control. With
The function update unit includes a first function update unit that updates a function related to voltage control, a second function update unit that updates a function related to pulse control, and a third function update unit that updates a function related to axis drive control. With
The first reward calculation unit increases the reward when the value of the first state is smaller than the previous time, and decreases the reward when the value of the first state is larger than the previous time. Machine learning device. A machine learning device for learning control parameters for controlling machining conditions in an electric discharge machine,
A state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining,
A learning unit that learns the control parameter based on a plurality of the state variables;
With
The state observation unit includes a first state value that is a value obtained by accumulating the number of times of generation of an unstable signal of a pulse during a predetermined period, and a second state value that is a number of pulses generated in the predetermined period. Observing the value and the value of the third state, which is the feed amount of the shaft in the driving device, as a plurality of the state variables,
The learning unit includes a reward calculation unit that calculates a reward based on the state variable, and a function update unit that updates a function for determining the control parameter based on the reward.
The reward calculator includes a first reward calculator that calculates a reward for voltage control, a second reward calculator that calculates a reward for pulse control, and a third reward calculator that calculates a reward for axis drive control. With
The function update unit includes a first function update unit that updates a function related to voltage control, a second function update unit that updates a function related to pulse control, and a third function update unit that updates a function related to axis drive control. With
The first reward calculation unit increases the reward when the value of the second state is larger than the previous time, and decreases the reward when the value of the second state is smaller than the previous time. Machine learning device. A machine learning device for learning control parameters for controlling machining conditions in an electric discharge machine,
A state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining,
A learning unit that learns the control parameter based on a plurality of the state variables;
With
The state observation unit includes a first state value that is a value obtained by accumulating the number of times of generation of an unstable signal of a pulse during a predetermined period, and a second state value that is a number of pulses generated in the predetermined period. Observing the value and the value of the third state, which is the feed amount of the shaft in the driving device, as a plurality of the state variables,
The learning unit includes a reward calculation unit that calculates a reward based on the state variable, and a function update unit that updates a function for determining the control parameter based on the reward.
The reward calculator includes a first reward calculator that calculates a reward for voltage control, a second reward calculator that calculates a reward for pulse control, and a third reward calculator that calculates a reward for axis drive control. With
The function update unit includes a first function update unit that updates a function related to voltage control, a second function update unit that updates a function related to pulse control, and a third function update unit that updates a function related to axis drive control. With
The second reward calculation unit increases the reward when the value of the first state is smaller than the previous time, and decreases the reward when the value of the first state is larger than the previous time. Machine learning device. A machine learning device for learning control parameters for controlling machining conditions in an electric discharge machine,
A state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining,
A learning unit that learns the control parameter based on a plurality of the state variables;
With
The state observation unit includes a first state value that is a value obtained by accumulating the number of times of generation of an unstable signal of a pulse during a predetermined period, and a second state value that is a number of pulses generated in the predetermined period. Observing the value and the value of the third state, which is the feed amount of the shaft in the driving device, as a plurality of the state variables,
The learning unit includes a reward calculation unit that calculates a reward based on the state variable, and a function update unit that updates a function for determining the control parameter based on the reward.
The reward calculator includes a first reward calculator that calculates a reward for voltage control, a second reward calculator that calculates a reward for pulse control, and a third reward calculator that calculates a reward for axis drive control. With
The function update unit includes a first function update unit that updates a function related to voltage control, a second function update unit that updates a function related to pulse control, and a third function update unit that updates a function related to axis drive control. With
The second reward calculation unit increases the reward when the value of the second state is larger than the previous time, and decreases the reward when the value of the second state is smaller than the previous time. Machine learning device. A machine learning device for learning control parameters for controlling machining conditions in an electric discharge machine,
A state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining,
A learning unit that learns the control parameter based on a plurality of the state variables;
With
The state observation unit includes a first state value that is a value obtained by accumulating the number of times of generation of an unstable signal of a pulse during a predetermined period, and a second state value that is a number of pulses generated in the predetermined period. Observing the value and the value of the third state, which is the feed amount of the shaft in the driving device, as a plurality of the state variables,
The learning unit includes a reward calculation unit that calculates a reward based on the state variable, and a function update unit that updates a function for determining the control parameter based on the reward,
The reward calculator includes a first reward calculator that calculates a reward for voltage control, a second reward calculator that calculates a reward for pulse control, and a third reward calculator that calculates a reward for axis drive control. With
The function update unit includes a first function update unit that updates a function related to voltage control, a second function update unit that updates a function related to pulse control, and a third function update unit that updates a function related to axis drive control. With
The third reward calculation unit increases the reward when the value of the second state is larger than the previous time, and decreases the reward when the value of the second state is smaller than the previous time. Machine learning device. A machine learning device for learning control parameters for controlling machining conditions in an electric discharge machine,
A state observation unit that observes a plurality of state variables representing a machining state during electric discharge machining,
A learning unit that learns the control parameter based on a plurality of the state variables;
With
The state observation unit includes a first state value that is a value obtained by accumulating the number of times of generation of an unstable signal of a pulse during a predetermined period, and a second state value that is a number of pulses generated in the predetermined period. Observing the value and the value of the third state, which is the feed amount of the shaft in the driving device, as a plurality of the state variables,
The learning unit includes a reward calculation unit that calculates a reward based on the state variable, and a function update unit that updates a function for determining the control parameter based on the reward,
The reward calculator includes a first reward calculator that calculates a reward for voltage control, a second reward calculator that calculates a reward for pulse control, and a third reward calculator that calculates a reward for axis drive control. With
The function update unit includes a first function update unit that updates a function related to voltage control, a second function update unit that updates a function related to pulse control, and a third function update unit that updates a function related to axis drive control. With
The third reward calculation unit increases the reward when the value of the third state is larger than the previous time, and decreases the reward when the value of the third state is smaller than the previous time. Machine learning device.

Images