DE112018007687T5 - Machine learning device, numerical control device, machine tool, and machine learning method - Google Patents

Abstract

A machine learning device (20) which learns a position command (53) to a driver unit (37) which moves a loader chuck (32) when a workpiece (40) is between the loader chuck (32), which the workpiece (40 ) grips and forwards, and a lathe chuck (31), which grips in order to receive the workpiece (40), is transferred, comprises: a state monitoring unit (25) which sends the position command (53) to the driver unit (37) as state variables (56) ) and feedback data from the driver unit (37) observed; and a learning unit (21) which learns the position command (53), which reduces an offset of a transfer position of the workpiece (40) between the loader chuck (32) and the lathe chuck (31), as a function of a data set which is based on the state variables ( 56) is generated.

Description

Area

The present invention relates to a machine learning apparatus which learns operations of handing over a workpiece, a numerical control apparatus, a machine tool, and a machine learning method.

background

In a machine tool such as a lathe, during an operation of transferring a workpiece between a chuck that grips the workpiece and passes the workpiece, and a chuck that grips the workpiece to receive the workpiece, a loader that the workpiece conveys the workpiece, which is gripped by the transferring chuck, to a workpiece transfer position. In some cases, the workpiece cannot be moved to the center position of a chuck portion of the chuck receiving the workpiece because, when the workpiece is an elongated workpiece, the workpiece is curved or, for example, because the chuck fails to grip the workpiece. Because, when handing over a workpiece, as described above, the handover may fail if the handover position is offset from a suitable position, a technology for reducing or preventing an offset of a handover position is desired.

A loader control apparatus described in Patent Literature 1 generates a function of the correlation between an offset value of a workpiece transfer position and a motor torque of a servomotor driving a loader, estimates an offset value of the transfer position based on the correlation and a measured motor torque, and corrects the transfer position based on the estimated offset value.

Citation list

Patent literature

Patent Literature 1: Japanese Patent Application, Publication Number 2002-187040

Summary

Technical problem

However, in the above-mentioned Patent Literature 1, since the function expressing the correlation is a fixed function, the possibility of failure of the transfer of a workpiece does not decrease even after repetitions of the operation of transferring the workpiece.

The present invention has been made in view of the above, and an object thereof is to provide a machine learning apparatus capable of reducing the possibility of failing to handover a workpiece by learning operations of handing over a workpiece.

the solution of the problem

In order to solve the aforementioned problems and achieve the object, one aspect of the present invention provides a machine learning device that learns a position command to a driving device that moves a first chuck when a workpiece is between the first chuck having the Workpiece grips and passes on, and a second chuck, which grips in order to receive the workpiece, is transferred. The machine learning apparatus includes: a state observation unit for observing, as a state variable, the position command to the driving device and feedback data from the driving device; and a learning unit for learning the position command, which reduces an offset of a transfer position of the workpiece between the first chuck and the second chuck, depending on a data set which is generated based on the state variable.

Advantageous Effects of the Invention

A machine learning apparatus according to the present invention has the effect of reducing the likelihood of failure of the handover of a workpiece by learning operations of handing over a workpiece.

Figure list

• 1 Fig. 13 is a diagram showing a configuration of a machining system according to an embodiment.
• 2 Fig. 13 is a diagram showing a configuration of a control system including a numerical control device according to the embodiment.
• 3 Fig. 13 is a flow chart showing operation procedures of the machining system according to the embodiment.
• 4th Fig. 13 is a diagram for explaining a first learning example performed by a machine learning device according to the embodiment.
• 5 Fig. 13 is a diagram for explaining a second learning example performed by the machine learning apparatus according to the embodiment.
• 6th Fig. 13 is a diagram for explaining relative positions of a lathe chuck of a machine tool according to the embodiment and a workpiece.
• 7th Fig. 13 is a diagram showing an example of a hardware configuration of the numerical control apparatus according to the embodiment.

Description of embodiments

A machine learning device, a numerical control device, a machine tool, and a machine learning method according to embodiments of the present invention will be described below in detail with reference to the drawings. It should be noted that the present invention is not limited to the embodiments.

Embodiment

1 Fig. 13 is a diagram showing a configuration of a machining system according to an embodiment. 1 shows a machining system 1 seen in the vertical direction. In the present embodiment, a case will be described in which the vertical direction is a Y-axis direction and the horizontal directions which are the directions of movement of a workpiece 40 are an X-axis direction and a Z-axis direction.

The machine processing system 1 comprises a machine tool 2, which is a workpiece 40 and a control system 3 that controls the operation of the machine tool 2. Examples of the machine tool 2 include a lathe and a machining center. The following describes a case where the machine tool 2 is a lathe.

The machine tool 2 comprises a rotating unit 35 , a loader chuck 32 , which is a first chuck, a lathe chuck 31 which is a second chuck, and a loader 36 , which is a conveying device for conveying a workpiece 40 is. The operation of the loader 36 is controlled by the control system 3. The loader chuck 32 is with the loader 36 connected and moves together with the loader 36 . The loader chuck 32 is capable of a workpiece 40 to grasp what is an object to be machined. Examples of the loader chucks 32 include a three-jaw chuck and a collet. When machining a workpiece 40 is started, the loader chuck transfers 32 the workpiece 40 to the lathe chuck 31 , and after finishing the machining of the workpiece 40 takes the loader chuck 32 the workpiece 40 from the lathe chuck 31 opposite.

The turntable 35 rotates around the Z axis, which is a major axis, as a rotation axis. The lathe chuck 31 is with the turntable 35 connected and rotates together with the turntable 35 . The lathe chuck 31 is capable of a workpiece 40 to grab. Examples of the lathe chuck 31 include a three-jaw chuck and a collet. When the workpiece 40 is being machined, the turntable rotates 35 with the lathe chuck 31 which the workpiece 40 reaches out to the workpiece 40 to turn. An example of the turntable 35 is a lathe device.

When the turntable 35 with a workpiece 40 is loaded, the machine tool 2 grips one end of the workpiece 40 with the loader chuck 32 . In this state the loader moves 36 along the negative X-axis direction and stops at a position that corresponds to the lathe chuck 31 opposite (s0). Then the loader moves 36 along the negative Z-axis direction. In this way the loader moves 36 of the workpiece 40 to a position where the lathe chuck 31 the workpiece 40 can grab (s1). The position of the loader 36 on which the lathe chuck 31 the workpiece 40 can grab is a desired transfer position. The machine tool 2 starts a closing operation of the lathe chuck 31 using an auxiliary function such as M codes (s2) and waits for the chuck closing operation 31 is complete (s3). By closing the lathe chuck 31 grips the lathe chuck 31 the other end of the workpiece 40 .

Thereafter, the machine tool 2 starts an opening operation of the loader chuck 32 (s4) and waits for the loader chuck opening operation 32 is complete (s5). The loader 36 then moves along the positive Z-axis direction. In this way the loader pulls itself 36 in the direction away from the workpiece 40 back (s6) and further moves along the positive X-axis direction.

When machining the workpiece 40 is completed, the machine tool 2 unloads the workpiece 40 from the turntable 35 by processes in the reverse order of the above-described processes s0 to s6. In this case, each of the processes s0 to s6 is a process in the reverse direction. Therefore, in each of the processes s0 to s6, the directions of movement are Loader 36 opposite to each other when loading and unloading. When unloading, the loading chuck closing operation 32 and the chuck opening operation 31 carried out.

The loader moves during unloading 36 in particular along the negative X-axis direction and further moves in the direction towards the workpiece 40 . The loader chuck 32 then starts a closing operation. When the closing operation is completed, the lathe chuck starts 31 an opening operation. When the opening operation is complete, the loader pulls 36 in the direction away from the workpiece 40 back and further moves along the positive X-axis direction.

Because the processes for loading the turntable 35 with a workpiece 40 and the processes for unloading a workpiece 40 from the turntable 35 When the processes are similar, the following will be processes for loading the turntable 35 with a workpiece 40 explained.

The mesh tool 2 has some properties that are specific to machines that produce a workpiece 40 grab, carry and the like. Therefore, there is a relationship caused by machine-specific characteristics between a position command to the loader chuck 32 and the actual position of the loader chuck 32 in the X-axis direction. Even if an appropriate position command to be used to load a workpiece 40 is provided, therefore the position of the workpiece 40 be offset in the X-axis direction when the loader chuck 32 the workpiece 40 to the lathe chuck 31 passes, and the workpiece 40 can end 30th of the lathe chuck 31 meet on the positive side in the Z-axis direction. In this case the loader chuck 32 the workpiece 40 not on the lathe chuck 31 to hand over.

In the present embodiment, a numerical control device learns 10 (NC device), which will be described later and included in the control system 3, the position of the loader chuck 32 in the X-axis direction when the loader chuck 32 a workpiece 40 to the lathe chuck 31 hands over. Therefore, the numerical control device reduces 10 the likelihood of failure to submit a workpiece 40 by learning position commands to the loader chuck 32 . When the position and shape of a workpiece 40 , which through the loader chuck 32 is gripped every time the workpiece is gripped 40 are the same, corresponds to the position of the workpiece 40 in the X-axis direction of the position of the loader chuck 32 in the X-axis direction. Therefore, in the present embodiment, an offset value of the loader chuck becomes 32 in the X-axis direction and an offset value of a workpiece 40 used synonymously in the X-axis direction.

Next is a configuration of the numerical control device 10 which controls the operation of the machine tool 2. 2 Fig. 13 is a diagram showing a configuration of the control system including the numerical control apparatus according to the embodiment. The control system 3 includes the numerical control device 10 , a driver unit 37 and a servo motor 38 .

The numerical control device 10 is a computer that shows the position of the loader 36 controls by a position command 53 to the driver unit 37 is sent. The position command 53 which the numerical control device 10 to the driver unit 37 is a command that determines the position of the loader 36 and includes a position command in the X-axis direction and a position command in the Z-axis direction. The numerical control device 10 controls the transfer of a workpiece 40 of the loader chuck 32 to the lathe chuck 31 and if the handover fails, change the position command 53 in the X-axis direction to the loader chuck 32 to control the handover again. The numerical control device 10 learns a suitable position command 53 in the X-axis direction to the loader chuck 32 based on a position command 53 in the X-axis direction to the loader chuck 32 and based on whether the result of a handover at which the position command 53 used in the X-axis direction, failure or success.

The driver unit 37 is a driving device for moving the loader 36 by driving the servo motor 38 . The driver unit 37 calculates a current value which is sent to the servo motor 38 to be transmitted based on the position command 53 from the numerical control device 10 . The driver unit 37 drives the servo motor 38 by transmitting one of the position commands 53 assigned current to the servomotor 38 . The driver unit 37 transmits a feedback (FB) stream 55 to the numerical control device 10 which data is which is sent to the servo motor 38 Specify the current to be transmitted. The FB stream 55 is an example of feedback data from the driver unit 37 to the numerical control device 10 .

If information about the speed of rotation of the servomotor 38 indicating from an encoder 39 is transferred, the driver unit calculates 37 an FB position 54 which data is which is the current position of the workpiece 40 based on the rotation speed and transmits the FB position 54 to the numerical Control device 10 . The FB position 54 is an example of the feedback data from the driver unit 37 to the numerical control device 10 .

The servo motor 38 is with the loader 36 connected and moves the loader 36 depending on the current from the driver unit 37 . The servo motor 38 includes a servo motor for moving the loader 36 in the X-axis direction and a servo motor for moving the loader 36 in the Z-axis direction. The measuring transducer 39 , which is the speed of rotation of the servomotor 38 detected is on every servo motor 38 for the X-axis direction and for the Z-axis direction. The measuring transducer 39 transmits to the driver unit 37 information indicating the detected rotational speed.

The numerical control device 10 comprises a machining control program storage unit 11 , an analysis unit 12 , a control unit 13 , a storage unit 14th and a machine learning device 20th . The machining control program storage unit 11 stores machining control programs that are used to machine a workpiece 40 be used. A machining control program includes a loading command 61 for loading the turntable 35 with a workpiece 40 , a machining command for machining the workpiece 40 and an unload command for unloading the workpiece 40 from the turntable 35 . In 2 is a loading order 61 shown. Among these commands are the loading command 61 and the unload command has dedicated commands for transferring a workpiece 40 . The loading order 61 is sent to the analysis unit 12 in the form of a G code 51 for positioning the loader 36 transfer.

The analysis unit 12 analyzes the machining control programs. The analysis unit 12 determines whether or not a parsed command is a dedicated command, and if the parsed command is a dedicated command, for example, the load command 61 , generates transfer position information based on the G code 51 52 which indicates the position at which a workpiece 40 is to be handed over. In particular, the analysis unit generates 12 the transfer position information 52 based on a command to position the loader 36 , which is contained in the G code 51. The transfer position information 52 is information about the position at which the workpiece 40 between the loader chuck 32 and the lathe chuck 31 is passed. In particular, the transfer position information 52 an end point of the loader 36 . When handing over a workpiece for the first time 40 is information used for the delivery operation set by arguments of the dedicated command.

• (A1) der Endpunkt des Beladers 36, welcher die Übergabepositionsinformation 52 des Werkstücks 40 ist;
• (A2) ein Referenzstromwert A, welcher eine Referenz zum Bestimmen ist, ob das Werkstück 40 bei der Übergabe anschlägt oder nicht;
• (A3) ein Rückzugswert Lz des Beladers 36, wenn bestimmt wurde, dass das Werkstück 40 angeschlagen ist;
• (A4) die Richtung und ein Bewegungswert Lx, in welche und mit welchem der Belader 36 während des Lernens bewegt wird; und
• (A5) eine maximale Bewegungsdistanz Lmax in eine Richtung während des Lernens.

The dedicated command includes the following command arguments (A1) through (A5):

• (A1) the end point of the loader 36 , which is the transfer position information 52 of the workpiece 40 is;
• (A2) a reference current value A, which is a reference for determining whether the workpiece is 40 hits or not at handover;
• (A3) a retraction value Lz of the loader 36 when it has been determined that the workpiece 40 is posted;
• (A4) the direction and a movement value Lx in which and with which the loader 36 is moved while learning; and
• (A5) a maximum moving distance Lmax in one direction during learning.

The transfer position information 52 of the aforementioned (A1) includes an X coordinate and a Z coordinate. The reference current value A of (A2) is a value for determining whether or not the handover is in an abnormal state, and is matched with the FB current 55 compared which is a current that is sent to the servo motor 38 is sent. A case in which an FB stream 55 to the servo motor 38 which is larger than the reference current value A is such an abnormal state in which the workpiece 40 the lathe chuck 31 meets at a position that is different from the transfer position of the transfer by the lathe chuck 31 is different. An example of the position at which the workpiece 40 something other than the transfer position of the transfer by the lathe chuck 31 hits, is the end 30th of the lathe chuck 31 , as mentioned above. The retraction value Lz of (A3) is a distance by which the workpiece 40 is moved back along the Z-axis direction when the workpiece 40 the end 30th has hit and the movement of the workpiece 40 stopped.

When a workpiece 40 the end 30th and has stopped moving, the machine learning device is learning 20th the position command 53 . The movement value Lx of (A4) is a distance that the workpiece 40 is moved in the X-axis direction during learning. The workpiece 40 is moved by the amount of movement Lx in the X-axis direction and then to the lathe chuck 31 moved in the Z-axis direction. The workpiece 40 is moved by the movement value Lx until the position to which the workpiece 40 was moved for the transfer is within an allowed range. Furthermore, the movement distance Lmax of (A5) is a distance limit by which the workpiece 40 while learning in the X-axis Direction is moved. Even while learning, the workpiece becomes 40 therefore does not move further than the moving distance Lmax.

The analysis unit 12 sends the transfer position information 52 from (A1) and the retraction value Lz from (A3) to the control unit 13 . The analysis unit 12 also sends the values of the command arguments of (A2), (A4) and (A5) described above to the machine learning device 20th . It should be noted that the analysis unit 12 is not limited to the case where the values of the command arguments are obtained from the dedicated command, and may obtain values associated with the command arguments from parameters. In this case, the values assigned to the command arguments are stored as parameters in the storage unit 14th saved.

The control unit 13 generates a position command 53 depending on the transfer position information 52 by the analysis unit 12 was sent, or depending on an action 58 which through the machine learning device 20th was provided. The action 58 is the next position command 53 in the X-axis direction. The control unit 13 sends the position command 53 to the driver unit 37 and the machine learning device 20th . After receiving a message from the machine learning device 20th , which indicates the position to which the workpiece 40 is within the permitted range, the control unit controls 13 the further operation of handing over the workpiece 40 .

After receiving a message from the machine learning device 20th which indicates that the position to which of the workpiece 40 is outside the permitted range, the control unit moves 13 the workpiece 40 by the retraction value Lz in the Z-axis direction and generates a position command 53 depending on the action 58 .

The machine learning device 20th comprises a condition monitoring unit 25th and a learning unit 21st . The condition observation unit 25th received from the analysis unit 12 the reference current value A of (A2) of the command arguments and the learning unit 21st received from the analysis unit 12 the movement value Lx of (A4) and the movement distance Lmax of (A5) of the command arguments.

The condition observation unit 25th receives the position commands 53 in the X-axis direction and in the Z-axis direction from the control unit 13 who have favourited FB positions 54 in the X-axis direction and in the Z-axis direction from the driver unit 37 and the FB currents 55 in the X-axis direction and in the Z-axis direction from the driver unit 37 .

To determine whether the position to which the workpiece 40 is moved within the allowable range or not, the state observation unit uses 25th the position commands 53 in the X-axis direction and in the Z-axis direction, the FB positions 54 in the X-axis direction and in the Z-axis direction and the FB currents 55 in the X-axis direction and in the Z-axis direction. Alternatively, the condition monitoring unit 25th based on the position command 53 in the X-axis direction, the FB position 54 in the X-axis direction and the FB current 55 in the X-axis direction determine whether the position at which the workpiece 40 moved is within the allowable range or not. As a further alternative, the condition monitoring unit 25th based on the position command 53 in the Z-axis direction, the FB position 54 in the Z-axis direction and the FB current 55 in the Z-axis direction determine whether the position at which the workpiece 40 moved is within the allowable range or not. As a further alternative, the condition monitoring unit 25th based on the FB position 54 in the Z-axis direction and the position command 53 in the Z-axis direction without using the FB current 55 in the Z-axis direction determine whether the position at which the workpiece 40 moved is within the allowable range or not.

In addition, if the learning unit 21st the position commands 53 to the driver unit 37 learns, observes the condition monitoring unit 25th the position command 53 in the X-axis direction, the FB position 54 in the X-axis direction and the FB current 55 in the X-axis direction as state variables 56 and sends the state variables 56 which results of the observation are to the learning unit 21st . Hence the state variables include 56 by the condition observation unit 25th to the learning unit 21st are transmitted, the position command 53 in the X-axis direction, the FB position 54 in the X-axis direction and the FB current 55 in the X-axis direction.

The transfer position of the workpiece 40 can opposite the center position of the lathe chuck 31 in the X-axis direction depending on the shape of the workpiece 40 be offset. In addition, in a case in which the transfer position at which the workpiece 40 through the loader chuck 32 is not suitable, the transfer position of the workpiece 40 from the center position of the lathe chuck 31 be offset in the X-axis direction. In these cases it is more likely that the workpiece 40 the lathe chuck 31 meets at a position different from the transfer position of the Transfer through the lathe chuck 31 is different, and stops at a position where the workpiece 40 through the lathe chuck 31 cannot be grasped.

In a case where the workpiece 40 the lathe chuck 31 hits at a position that is different from the transfer position is the position at which the workpiece 40 moved outside the permitted range. In a case where the workpiece 40 when moving to the transfer position on the lathe chuck 31 rubs is the position at which the workpiece 40 was moved, also outside the permitted area.

In an abnormal case in which the position to which the workpiece 40 moved is outside the permitted range, the load on the loader increases 36 in the X-axis direction and / or the Z-axis direction. In the case where the position to which the workpiece 40 moved is outside the permitted range, the servomotor therefore picks up 38 sent stream to and the FB stream 55 also increases. Therefore, the state observer determines 25th based on a result of comparison between the reference current value A and the FB current 55 whether the position to which the workpiece 40 moved is within the allowable range or not. In a normal case, in which the position to which the workpiece 40 is within the permitted range, the workpiece will stop 40 at the transfer position of the transfer by the lathe chuck 31 , and therefore takes the load on the loader 36 not too and the FB current 55 does not increase.

In addition, in an abnormal case in which the position to which the workpiece 40 is outside the permitted range, the position command is correct 53 assigned position of the loader 36 and that of the FB position 54 corresponding position of the loader 36 does not match within a certain period of time. Therefore, the state observer determines 25th based on a result of comparison between the position command 53 and the FB position 54 whether the position to which the workpiece 40 moved is within the allowable range or not. The condition observation unit 25th sends the result of the determination of whether the position to which the workpiece 40 is within the permitted range or not to the control unit 13 .

The learning unit 21st learns an action 58 which the next position command 53 is, depending on the state variables 56 . In other words, the unit learns 21st a position command 53 , at which the offset value of the transfer position of the transfer of the workpiece 40 by the loader 36 is reduced.

In particular, the learning unit learns 21st an action 58 depending on a data record, which is based on the state variables 56 was generated, which the position command 53 who have favourited FB position 54 and the FB stream 55 include. The learning unit 21st comprises a function update unit 22nd and a reward calculation unit 23 .

The reward calculation unit 23 calculates a reward 57 based on the state variables 56 . The reward calculation unit 23 calculates a difference between the position given by the position command 53 and the FB position 54 based on the state variables 56 and extracted from the state variables 56 the FB stream 55 . The reward calculation unit 23 increases the reward 57 when the difference between that specified by the position command 53 specified position and the FB position 54 is equal to or less than a limit value and the FB current 55 is equal to or less than the reference current value A. In this case, the reward calculation unit enlarges 23 the reward 57 as the difference between the position command becomes smaller 53 specified position and the FB position 54 and with decreasing FB current 55 . The learning unit 21st sends the calculated reward 57 to the function update unit 22nd . In the following description, the difference between the position command 53 specified position and the FB position 54 also be referred to as a positional difference.

The feature update unit 22nd stores a function for determining an action 58 and updates the function for determining an action 58 based on the reward 57 . An example of the function for determining an action 58 is an action value function Q (s _t , a _t ) which will be described later. The feature update unit 22nd of the embodiment updates the action value function Q (s, a) so that the offset value of the transfer position decreases every time the operation of transferring a workpiece 40 is repeated by the machine tool 2. The feature update unit 22nd calculates an action 58 using the updated action value function Q (s, a). The feature update unit 22nd sends the calculated action 58 to the control unit 13 and sends previously learned data, data used for learning and for controlling the loader 36 necessary data to the storage unit 14th . An example of the learned data is a position command 53 for the next handover that was calculated when a handover was successful, and an example of the data used for learning is the action value function Q (s, a) given by the Learning unit 21st was used for learning. An example for controlling the loader 36 used data is a retraction value Lz. The storage unit 14th saves the previously learned data, the data used for learning and the data used to control the loader 36 by the control unit 13 necessary data.

Next are operating procedures of the machining system 1 explained. 3 Fig. 13 is a flow chart showing the operation procedures of the machining system according to the embodiment. After reading a loading command 61 when handing over the workpiece 40 the first handover is, that is, in an unskilled case, the numerical control device starts 10 moving the loader 36 to the handover position indicated by the command argument of the above-described (A1) (step ST1). In this case the control unit transmits 13 a position command 53 to the driver unit 37 and the condition observation unit 25th . This causes the loader to move 36 in the X-axis direction and then moves in the Z-axis direction while engaging the loader chuck 32 the workpiece 40 .

When the workpiece 40 starts moving, the driver unit receives 37 the FB position 54 and the FB stream 55 at certain intervals and sends the FB position 54 and the FB stream 55 to the condition monitoring unit 25th . The condition observation unit 25th therefore monitors the position command 53 who have favourited FB position 54 and the FB stream 55 .

The condition observation unit 25th determines whether the FB current 55 of the X-axis and the Z-axis is equal to or smaller than the reference current value A, respectively, or not (step ST2). It should be noted that reference current values A of different values can be used for the X-axis and the Z-axis.

When the FB current 55 the X-axis or the Z-axis becomes larger than a reference current value A before the workpiece 40 reaches the transfer position (step ST2: no), the status monitoring unit sends 25th to the learning unit 21st State variables 56 which position commands 53 in the X-axis direction and in the Z-axis direction, FB positions 54 in the X-axis direction and in the Z-axis direction and FB currents 55 in the X-axis direction and in the Z-axis direction. The condition monitoring unit also provides information 25th the control unit 13 about that the position at which the workpiece 40 moved is outside the permitted range.

When the FB current 55 of an axis is greater than the reference current value A, ie if the movement to the transfer position fails, the learning unit stops 21st for the position command 53 used to deliver a reward 57 with a small value one. This is how the learning unit learns 21st a suitable position command 53 depending on the status variables 56 and determine an action 58 which the next position command 53 is, such that the reward 57 becomes maximum (step ST4a). The control unit 13 moves the workpiece 40 back in the Z-axis direction by the retraction amount Lz (step ST5). The control unit 13 then starts moving the loader 36 to the handover position (step ST1). The control unit 13 therefore moves the workpiece 40 in the X-axis direction using the position command 53 which depends on the action 58 and then moves the workpiece 40 in the Z-axis direction.

When in the process of step ST2 the FB stream 55 of the X-axis and the Z-axis each became equal to or smaller than the reference current value A (step ST2: Yes), the state observation unit determines 25th for the X-axis and the Z-axis, respectively, whether the position difference between that determined by the position command 53 specified position and the FB position 54 is equal to or less than the limit value or not (step ST3).

If the FB position 54 and the position command 53 for the X-axis and the Z-axis are different from one another (step ST3: No), the state monitoring unit sends 25th to the learning unit 21st State variables 56 , which the position command 53 in the X-axis direction, the FB position 54 in the X-axis direction and the FB current 55 include in the X-axis direction. The condition monitoring unit also provides information 25th the control unit 13 about that the position at which the workpiece 40 moved is outside the permitted range. In this way, the above-described processes in steps ST4a and ST5 are performed, and the process in step ST1 is further performed.

Now becomes a process of learning an action 58 through the learning unit 21st explained. A learning algorithm which is used for the learning unit 21st used can be any learning algorithm. A case is explained here in which reinforcement learning is applied to the learning algorithm. In reinforcement learning, an agent who is a subject of an action in an environment observes the current state, which is determined by state variables 56 is specified and determines an action to be taken based on the result of the observation 58 . The agent receives from the environment as a result of choosing the action 58 a reward 57 and learns an action, which through a series of actions 58 the greatest reward 57 can get. Q-learning and TD-learning are known as typical reinforcement learning techniques. For example, in the case of Q learning, a typical formula (action value table) for updating the action value function Q (s, a) is expressed by the following formula (1) below. Therefore, an example of the action value table is the action value function Q (s, a) of the formula (1).
[Math. 1] $$Q\left(s_t,a_t\right)\leftarrow Q\left(s_t,a_t\right)+\mathrm{\alpha }\left(r_{t+1}+\mathrm{\gamma }\underset{a}{{\displaystyle \text{Max}}}Q\left(s_{t+1},a\right)-Q\left(s_t,a_t\right)\right)$$

In the formula (1), s _{t represents} an environment at time t and a _t represents an action at time t. The action a _{t changes} the environment to s _{t + 1} . r _{t + 1} represents a reward 57 given as a result of the change in the environment, γ represents a discount rate and α represents a learning coefficient. When Q-learning is applied, action a _{t is} the next position command 53 the handover operation.

The update formula expressed by the formula (1) increases the action value Q when the action value of the best action a at time t + 1 is larger than the action value Q of the action a performed at time t, or decreases the action value Q im opposite case. In other words, the action value function Q (s, a) is updated so that the action value Q of the action a at time t is closer to the best action value at time t + 1. As a result, the best action value in one environment develops sequentially to action values in previous environments.

The reward calculation unit 23 calculates a reward 57 based on the position difference between the position command 53 specified position and the FB position 54 and based on the FB current 55 .

As described above, the reward calculation unit increases 23 the reward 57 when the positional difference between that specified by the position command 53 specified position and the FB position 54 is equal to or less than a limit value and the FB current 55 is equal to or less than the reference current value A. In this case, the reward calculation unit gives 23 for example a reward 57 from 1".

In contrast, the reward calculation unit downsizes 23 the reward 57 when the positional difference between that specified by the position command 53 specified position and the FB position 54 is greater than the limit or if the FB current 55 is greater than the reference current value A. In this case, the reward calculation unit gives 23 for example a reward 57 from 1".

When the position difference is zero and a value of the change in FB current 55 is zero, represents the reward calculation unit 23 the reward 57 for example, a maximum reward. When the position difference is equal to or less than the limit value and the FB current 55 is half of the reference current value A, the reward calculating unit provides 23 the reward 57 half of the maximum reward. An example of the case where the FB stream 55 is half of the reference current value A, is a case where the handover of the workpiece 40 is successful, but the transfer position is slightly offset from a desired position. In a case where the workpiece 40 before reaching the transfer position on the lathe chuck 31 rubs is the FB current 55 great during rubbing. In addition, in a case where the transfer position is slightly offset from a desired position when the lathe chuck 31 tried the workpiece 40 in the middle of the transfer position of the workpiece 40 to grab after the workpiece 40 has reached the transfer position, the workpiece 40 against the lathe chuck 31 pressed and the FB current 55 increases. In such cases, the handover position is within the permitted range, the machine learning device 20th however, gives a reward 57 with a value between a case in which the workpiece 40 the end 30th does not meet, and there is a case where the workpiece 40 the end 30th meets.

If the position difference is greater than the limit value or if the FB current 55 is greater than the reference current value A, the reward calculation unit 23 the reward 57 for a minimal reward. The reward calculation unit 23 sends the calculated reward 57 to the function update unit 22nd .

The feature update unit 22nd updates the function for determining an action 58 depending on the reward 57 by the reward calculation unit 23 was calculated. In the case of Q learning, the action _{value function Q (s t} , a _t ) expressed by the formula (1) is, for example, the function for calculating an action 58 , which by the function update unit 22nd is updated.

4th Fig. 13 is a diagram for explaining a first learning example performed by the machine learning device according to the embodiment. 4th shows positions P0 to P6 of the workpiece 40 in the X-axis direction. The position of the workpiece 40 in the X-axis direction when the workpiece 40 the end 30th of the lathe chuck 31 is assumed to be the position P0. In this case, the machine learning device repeats 20th a process of moving the workpiece 40 back along the positive Z-axis direction, a process of moving the workpiece 40 to the next position in the X-axis direction and a process of inserting the workpiece 40 along the negative Z-axis direction until the workpiece 40 no longer the end 30th of the lathe chuck 31 meets.

In particular, the machine learning device moves 20th the workpiece 40 in the X-axis direction in an order of position P1, position P2, position P3, position P4, position P5 and position P6, which are positions in the X-axis direction. The interval between the position P0 and the position P1 is the movement value Lx. In the same way are the intervals between the position P1 and the position P2, between the position P2 and the position P3, between the position P0 and the position P4, between the position P4 and the position P5 and between the position P5 and the position P6 the movement value Lx. In addition, the intervals between the position P0 and the position P3 and between the position P0 and the position P6 are the moving distance Lmax, respectively.

For example, to move the workpiece 40 the machine learning device sends to position P1 20th an action 58 to the control unit 13 to the workpiece 40 load to position P1. This initiates the action 58 assigned position command 53 the loader 36 to move in the X-axis direction.

When the workpiece 40 successfully to the lathe chuck 31 is moved without the end 30th to meet ends the machine learning device 20th the movement of the workpiece 40 in the X-axis direction. The machine learning device 20th gives a position command 53 a small reward 57 when the workpiece 40 the end 30th hits, and gives a position command 53 a great reward 57 when the workpiece 40 the end 30th did not hit.

It should be noted that the machine learning device 20th the workpiece 40 can move to positions P1 to P6 in any order. For example, the machine learning device 20th the workpiece 40 move in the X-axis direction in ascending order of distance from position P0, for example according to position P1, position P4, position P2, position P5, position P3 and position P6. In addition, the machine learning device is 20th not limited to the case where six positions P1 to P6 are set as the positions in the X-axis direction, and can set five or less or seven or more positions in the X-axis direction.

When the FB current 55 of the X-axis and the Z-axis respectively became equal to or smaller than the reference current value A (step ST2: Yes) and the positional difference between that determined by the position command 53 specified position and the FB position 54 for the X-axis and the Z-axis became equal to or smaller than the limit value (step ST3: Yes), the learning unit learns 21st an action 58 which the next position command 53 is, depending on the state variables 56 (Step ST4b). The learning unit 21st therefore provides for the position command 53 used to deliver a reward 57 with a large value and then determines the action 58 which the next position command 53 corresponds.

After the workpiece 40 has been moved to the transfer position, informs the state monitoring unit 25th also the control unit 13 about that the position at which the workpiece 40 moved is within the permitted range. The control unit 13 therefore performs the process of handing over the workpiece 40 between the chucks.

In particular, the control unit initiates 13 the machine tool 2 to perform the operations from steps s2 to s6 described above. In particular, the control unit starts 13 the operation of closing the lathe chuck 31 (Step ST6) and waits for the lathe chuck 31 is closed (step ST7). By closing the lathe chuck 31 grips the lathe chuck 31 the workpiece 40 .

Then the control unit starts 13 the operation of opening the loader chuck 32 (Step ST8) and waits for the loader chuck 32 is opened (step ST9). The control unit 13 then pulls the loader 36 back along the Z-axis positive direction (step ST10).

When the workpiece 40 from the lathe chuck 31 to the loader 36 is discharged, the machine learning device learns 20th also position commands 53 about processes similar to those of the case in which the workpiece 40 from the loader 36 on the lathe chuck 31 is loaded.

As described above, when a handover of the workpiece 40 fails, causes the machine learning device 20th that the handover of the workpiece 40 is carried out again as a function of the movement value Lx, whereby a correction of the transfer position is made possible. In addition, because the machine learning device 20th the transfer position of the workpiece 40 learns one can Failure of the handover can be prevented. In addition, because the machine learning device 20th the transfer position of the workpiece 40 learns is the machine learning device 20th also applicable to an environment in which the handover fails due to a slight offset, for example in the case of a collet. Furthermore, because the machine learning device 20th can cause the handover of the workpiece 40 is performed again and can prevent failure of the handover, the productivity of the machine tool 2 is increased.

In addition, because the machine learning device 20th the transfer position of the workpiece 40 based on the position difference between the position command 53 specified position and the FB position 54 certain, it is not necessary to use a special device or device, such as a camera, for checking the transfer of the workpiece 40 to provide. Therefore, the handover can be checked at a low cost.

In addition, when handing over the workpiece 40 failed, the machine learning device leads 20th the handover of the workpiece 40 again depending on the movement value Lx, eliminating the need for manual recovery even when the workpiece 40 the end 30th of the lathe chuck 31 meets. This can reduce downtime when the workpiece 40 and may prevent productivity from being lowered.

While, in the present embodiment, the case where the machine tool 2 is the workpiece is explained 40 Moving in the X-axis direction and in the Z-axis direction, the machine tool 2 can move the workpiece 40 move in the X-axis direction, the Y-axis direction, and the Z-axis direction. In this case, include position commands 53 which the numerical control device 10 to the driver unit 37 transmits a position command in the X-axis direction, a position command in the Y-axis direction, and a position command in the Z-axis direction. It also includes the servo motor 38 Servo motors for moving the loader 36 in the X-axis direction, in the Y-axis direction, and in the Z-axis direction. The machine learning device 20th then learns position commands in the X-axis direction, position commands in the Y-axis direction, and position commands in the Z-axis direction.

5 Fig. 13 is a diagram for explaining a second learning example performed by the machine learning apparatus according to the embodiment. 6th Fig. 13 is a diagram for explaining relative positions of the rotary chuck of the machine tool according to the embodiment and a workpiece. A case will be explained here in which the machine tool 2 is a workpiece 40 moves in the X-axis direction, the Y-axis direction, and the Z-axis direction. The 5 and 6th show positions of the workpiece 40 in an XY plane when the workpiece 40 is viewed in the Z-axis direction.

An in 6th shown lathe chuck 31A is an example of the lathe chuck 31 which related to 1 is described. The lathe chuck 31A is a three-jaw chuck. The three lathe chucks 31A grab the workpiece 40 by each moving to the center Q1 of a circular chuck portion 45.

The numerical control device 10 generates a position command 53 for inserting the workpiece 40 at the center Q1, however, the actual workpiece 40 at a position offset from the center Q1 such as a position P0 and the end 30th to meet. The machine learning device 20th therefore learns a position command 53 with which the workpiece 40 the end 30th does not meet.

It is assumed that the position of the workpiece 40 when the workpiece 40 the end 30th of the lathe chuck 31A hits, the position is P0. In this case, the machine learning device repeats 20th a process of moving the workpiece back 40 along the positive Z-axis direction, a process of moving the workpiece 40 to the next positions in the X-axis direction and in the Y-axis direction and a process of inserting the workpiece 40 along the negative Z-axis direction until the workpiece 40 the end 30th of the lathe chuck 31A no longer hits. In this case, the machine learning device calculates 20th the next position to which the workpiece 40 is moved based on the movement value Lx and the movement distance Lmax. While learning the movement positions, the workpiece is 40 therefore moved to a position which is limited by certain learned directions and certain learned movement values in the XY plane.

In particular, the machine learning device moves 20th the workpiece 40 in an order according to position P11, position P12, position P13, position P14, position P15, position P16, position P17, position P18 and position P19, which are positions in the XY plane. The intervals between the position P0 and the position P11, between the position P11 and the position P12, and between the position P12 and the position P13 are respectively the movement value Lx. In the same way, the intervals between the position P0 and the position P14, between the position P14 and the position P15, and between the position P15 and the position P16 are each the movement value Lx. In the same way, the intervals between the position P0 and the position P17, between the position P17 and the position P18, and between the position P18 and the position P19 are each the movement value Lx. In addition, the intervals between the position P0 and the position P13, between the position P0 and the position P16, and between the position P0 and the position P19 are the moving distance Lmax, respectively.

In addition, an angle between a direction from the position P0 to the position P13 and a direction from the position P0 to the position P16 is a learned angle θ, and an angle between a direction from the position P0 to the position P16 and a direction from the Position P0 to position P19 is the learned angle θ. The machine learning device 20th repeats a process of moving the workpiece at a time 40 from the position P0 in a first direction by the movement amount Lx and after the workpiece 40 is moved by the moving distance Lmax, each process of moving the workpiece 40 from the position P0 in a second direction, which is a direction with the learned angle θ to the first direction, by the movement amount Lx. When a first search for a suitable movement position of the workpiece 40 is performed, the machine learning device searches 20th therefore seek a suitable moving position by placing the workpiece 40 is moved in the first direction by the movement value Lx up to the maximum movement distance Lmax. If no suitable movement position is found in the first direction, the machine learning device leads 20th perform a search equal to that in the first direction in the second direction which has the learned angle θ with respect to the first direction. The machine learning device 20th each repeats a process of searching for a suitable moving position while the workpiece 40 by the movement amount Lx, and a process of rotating the learned angle θ until a sum of the learned angles θ within the chuck portion 45 exceeds 360 °.

For moving the workpiece 40 the machine learning device sends to position P11 20th for example an action 58 to the control unit 13 to the workpiece 40 to load at position P11. This triggers a position command assigned to action 85 53 the loader 36 to move in the X-axis direction and in the Y-axis direction.

When the workpiece 40 successfully to the lathe chuck 31A was moved without the end 30th to meet ends the machine learning device 20th the movements of the workpiece 40 in the X-axis direction and in the Y-axis direction.

It should be noted that the machine learning device 20th the workpiece 40 can move to positions P11 to P19 in any order. For example, the machine learning device 20th the workpiece 40 move in ascending order of distance from position P0, e.g. according to position P11, position P14, position P17, position P12, position P15, position P18, position P13, position P16 and position P19. In addition, the machine learning device is 20th not limited to the case where nine positions P11 to P19 are set as the positions in the XY plane, and can set eight or less or ten or more positions in the XY plane.

Now, a hardware configuration of the numerical control apparatus becomes 10 described. 7th Fig. 13 is a diagram showing an example of a hardware configuration of the numerical control apparatus according to the embodiment.

The numerical control device 10 can through an in 7th The control circuit 300 shown may be implemented, ie a processor 301 and a memory 302. Examples of the processor 301 include a central processing unit (CPU; also as a central processing device, as a processing device, as a computing device, as a microprocessor, as a microcomputer, as a processor or as a digital signal processor (DSP)) and a highly integrated system. Examples of the memory 302 include random access memory (RAM) and read only memory (ROM).

The numerical control device 10 is implemented by the processor 301 which has programs for performing operations of the numerical control apparatus 10 that are stored in the memory 302, reads and executes. In other words, these programs cause a computer to carry out the procedures or processes performed by the numerical control device 10 be performed. The memory 302 is also used as temporary storage when the processor 301 performs various processes.

It should be noted that some of the functions of the numerical control device 10 may be implemented by dedicated hardware and that others may be implemented by software or firmware. In addition, the machine learning device 20th through the in 7th control circuit 300 shown may be implemented.

While in the present embodiment based on the FB current 55 and the position difference is determined whether or not a transfer position is within the allowable range, the determination of whether or not a transfer position is within the allowable range can be can be determined based on the position difference without the FB current 55 to use.

Furthermore, while in the present embodiment, the position commands 53 for the transfer position based on the FB current 55 in the X-axis direction and the position difference in the X-axis direction can be learned, the position commands 53 for the transfer position based on the position difference in the X-axis direction can be learned without the FB current 55 to be used in the X-axis direction.

It should be noted that in a case where the lathe chuck 31 an electric chuck is the offset of the workpiece 40 also on the lathe chuck 31 can be detected. In this case, the machine learning device can 20th a position command 53 based on an offset, learn which on the lathe chuck 31 is detected.

While, in the present embodiment, the case in which the machine learning device 20th performs machine learning using reinforcement learning, the machine learning device can 20th perform machine learning using other known methods, for example using a neural network, genetic programming, functional logic programming, or a support vector machine.

In addition, while in the present embodiment the case is explained in which the control unit 13 the loader 36 based on the action 58 controls that of the learning unit 21st has been learned, the control unit 13 the loader 36 control without the action 58 to use. In this case, the control unit determines 13 based on the position command 53 , the FB position 54 and the FB stream 55 whether the position to which the workpiece 40 is within the allowable range or not, and if the offset value is not within the allowable range, issues a new position command 53 from which a position is opposite to that given by the position command 53 specified is offset. The control unit 13 therefore searches for a suitable moving position of the workpiece 40 by changing the moving position of the workpiece 40 Is moved piece by piece. In particular, the control unit provides 13 making the offset value fall within the allowable range by performing a process of determining the offset one or more times and, when the offset value is not within the allowable range, a process of issuing the new position command 53 are executed.

Because, as described above, according to the embodiment, a position command 53 , which is an offset of the transfer position of a workpiece 40 between the chucks is reduced or prevented, depending on a data set which is based on the state variables 56 is generated, the probability of failure of handing over the workpiece may increase 40 by repeating the operation of handing over the workpiece 40 be reduced.

The configurations shown in the embodiment described above are examples of the present invention and can be combined with other known technologies and / or can be partially omitted or modified without departing from the scope of the present invention.

List of reference symbols

1

Machining system; 2 machine tool; 3

10

numerical control device;

11

Machining control program storage unit;

12

Analysis unit;

13

Control unit;

14th

Storage unit;

20th

Machine learning device;

21st

Learning unit;

22nd

Function update unit;

23

Reward calculation unit;

25th

Condition monitoring unit;

30th

The End;

31, 31A

Lathe chucks; encoder;

32

Loader chucks;

35

Turntable;

36

Loader;

37

Driver unit;

38

Servo motor;

39

Measuring transducer;

40

Workpiece;

52

Transfer position information;

53

Position command;

54

FB position;

55

FB current;

56

State variable;

57

Reward;

58

Action;

61

Loading order.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• JP 2002187040 [0004]

Claims (11)

Machine learning device that learns a position command to a driver device that moves a first chuck when a workpiece is passed between the first chuck that grips and passes the workpiece and a second chuck that grips to receive the workpiece , wherein the machine learning device comprises: a state observation unit for observing, as a state variable, the position command to the driving device and feedback data from the driving device; and a learning unit for learning the position command, which reduces an offset of a transfer position of the workpiece between the first chuck and the second chuck, depending on a data set that is generated based on the state variable. Numerical control device comprising: the machine learning device according to Claim 1 ; and a control unit for outputting the position command learned by the learning unit to the driving device. Numerical control device according to Claim 2 wherein the feedback data includes a feedback position indicating a position of a conveying device for conveying the workpiece, and the learning unit includes: a reward calculating unit for calculating a reward based on a difference between a position indicated by the position command to the driving device and the Feedback position; and a function update unit for updating, based on the reward, a function for determining the position command. Numerical control device according to Claim 3 wherein the feedback data includes a feedback current, the feedback current being data indicating a current output from the driving device to make the driving device move the first chuck, and the reward calculating unit calculates the reward based on the difference and calculated from the feedback current. Numerical control device according to Claim 4 wherein the reward calculating unit increases the reward when the difference is equal to or smaller than a limit value and the feedback current is equal to or smaller than a reference current value, or decreases the reward when the difference is greater than the limit value or when the feedback current is greater than the reference current value. Numerical control device according to one of the Claims 3 to 5 wherein the function update unit updates an action value table expressing the function depending on the reward. Numerical control device according to Claim 4 wherein the state monitoring unit determines that the handover has failed when the difference is greater than a limit value or when the feedback current is greater than a reference current value, upon determining that the handover has failed, the learning unit learns the position command, and upon determination that the handover has failed, the control unit tries the handover again in accordance with the position command learned by the learning unit. Numerical control device according to one of the Claims 2 to 7th wherein the learning unit of position commands to the driving device learns a position command in a first direction which is perpendicular to a direction of transfer between the first chuck and the second chuck. Machine tool, which by the numerical control device according to one of the Claims 2 to 8th is controlled and driven by the driver device. A machine learning method for learning a position command to a driver device that moves a first chuck when a workpiece is learned between the first chuck that grips and relays the workpiece and a A second chuck which grips to receive the workpiece, the machine learning method comprising: a state observation step of observing, as a state variable, the position command to the driving device, and feedback data from the driving device; and a learning step of learning the position command, which reduces an offset of a transfer position of the workpiece between the first chuck and the second chuck, depending on a data set that is generated based on the state variable. Numerical control device that issues a position command to a driving device that moves a first chuck when a workpiece is passed between the first chuck that grips and transfers the workpiece and a second chuck that grips to receive the workpiece, the Numerical control device includes: a control unit for determining an offset of a transfer position of the workpiece between the first chuck and the second chuck based on the position command to the driving device and feedback data from the driving device and for issuing a new position command at which a position is opposite to a position specified by the position command is offset when an offset value is not within an allowed range, where, if the offset value is not within the allowable range, the control unit makes the offset value fall within the allowable range by performing a process of determining the offset and a process of issuing the new position command one or more times.

Images