JP6940425B2 - Control device and machine learning device - Google Patents

Description

The present invention relates to a control device and a machine learning device.

Conventionally, a 6-axis articulated robot has been used when a robot is used to load a work into a machine tool or sort products on a conveyor at a factory. In the 6-axis articulated robot, the positions of all the axes are uniquely determined according to the tip position of the robot hand. However, the route from the source to the destination can be freely selected. On the other hand, a 7-axis articulated robot in which one axis is further added to the 6-axis articulated robot is also generally used (FIG. 8, Patent Document 1, etc.). The 7-axis articulated robot is characterized in that the position of each axis is not uniquely determined with respect to the tip position of the robot hand, and the robot has a sufficient degree of freedom in operation.

FIG. 9 is a diagram showing how a 7-axis articulated robot is used to sort defective products among products flowing on a conveyor into defective product boxes. As illustrated in FIG. 9, the 7-axis articulated robot has axes J1 to J7, and the tip of the robot hand can be moved to a desired position by operating each axis. For example, in FIG. 9, when a defective product is transferred from the products on the conveyor to the defective product box by a 7-axis articulated robot, for example, the J4 and J5 axes are driven to transfer the product on the conveyor to the defective product box. This can be done (path 1), or the products on the conveyor can be moved to the defective box by driving the J2 and J3 axes (path 2). It is also possible to transfer the product on the conveyor to the defective product box by combining other shafts.

Japanese Unexamined Patent Publication No. 2011-011263

In this way, when a multi-axis articulated robot such as a 7-axis articulated robot is used, there are a plurality of combinations of axes for realizing one motion, but in general, a combination of axes capable of performing the same motion. A combination of axes that can be operated at high speed is selected from the inside. However, if only the same shaft is used, the operating temperature of the motor that drives the shaft rises. Then, if the motor is continuously used in a high temperature region, there arises a problem that the life of the motor is shortened at an accelerating rate. Further, as an additional problem, the speed reducer related to each motor also has a problem that the life is shortened if it is continuously used in a high temperature region.

Therefore, an object of the present invention is to provide a control device and a machine learning device capable of extending the operating life of each motor by operating each motor in a low temperature region as much as possible in consideration of the temperature of each motor. ..

In the present invention, using machine learning technology, a shaft that is driven when each motor is operated in a low temperature region as much as possible in consideration of the temperature of each motor and the robot is operated so as to extend the operating life of the motor. The above problem is solved by selecting the combination of the above and changing the moving speed and the distance.

Then, one aspect of the present invention is a control device that controls to extend the life of the motor that drives each axis of the robot, and includes a machine learning device that learns a movement command of the motor with respect to the operating environment of the robot. The machine learning device includes movement command data indicating a movement command of the motor, reference position / orientation data indicating the position / orientation of the moving source and moving destination of the tip of the robot, and a motor temperature indicating the temperature or estimated temperature of the motor. A state observation unit that observes data as a state variable representing the current state of the environment, a judgment data acquisition unit that acquires motor temperature judgment data indicating the suitability judgment result of the motor temperature or the estimated temperature as judgment data, and the above. Using the state variable and the determination data, a learning unit that learns by associating the position / orientation of the movement source / destination of the tip of the robot, the temperature or the estimated temperature of the motor, and the movement command of the motor with each other. It is a control device provided.

Another aspect of the present invention is a control device that controls to extend the life of a motor that drives each axis of the robot, and includes a machine learning device that learns a movement command of the motor with respect to the operating environment of the robot. The machine learning device has movement command data indicating a movement command of the motor, reference position / orientation data indicating the position / orientation of the moving source and moving destination of the tip of the robot, and motor temperature data indicating the temperature or estimated temperature of the motor. A state observation unit that observes the current state of the environment as a state variable, a judgment data acquisition unit that acquires motor temperature judgment data indicating the suitability judgment result of the motor temperature or the estimated temperature as judgment data, and the robot. A learning unit that learns by associating the position and orientation of the movement source / destination of the tip of the tip, the temperature or estimated temperature of the motor, and the movement command of the motor, the state variables observed by the state observation unit, and the learning unit. This is a control device including a decision-making unit that outputs a movement command of a motor that drives each axis of the robot based on the learning result of the above.

Another aspect of the present invention is a control device that controls to extend the life of a motor that drives each axis of the robot, and includes a machine learning device that learns a movement command of the motor with respect to the operating environment of the robot. The machine learning device uses reference position / orientation data indicating the position / orientation of the moving source and moving destination of the tip of the robot and motor temperature data indicating the temperature or estimated temperature of the motor as state variables representing the current state of the environment. Using the state observation unit to be observed, the label data acquisition unit that acquires the movement command data indicating the movement command of the motor as label data, the state variable, and the label data, the movement source of the tip of the robot It is a control device including a learning unit that learns by associating a position / orientation of a moving destination, a temperature or an estimated temperature of the motor, and a movement command of the motor.

Another aspect of the present invention is a control device that controls to extend the life of a motor that drives each axis of the robot, and includes a machine learning device that learns a movement command of the motor with respect to the operating environment of the robot. The machine learning device uses reference position / orientation data indicating the position / orientation of the moving source and moving destination of the tip of the robot and motor temperature data indicating the temperature or estimated temperature of the motor as state variables representing the current state of the environment. The state observing unit to be observed, the learning unit learned by associating the position / orientation of the moving source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the movement command of the motor, and the state observing unit It is a control device including an observed state variable and an estimation result output unit that estimates and outputs a movement command of a motor that drives each axis of the robot based on the learning result by the learning unit.

Another aspect of the present invention is a machine learning device that learns a movement command of the motor with respect to the operating environment of the robot in order to control so as to extend the life of the motor that drives each axis of the robot. The movement command data indicating the movement command of the robot, the reference position / attitude data indicating the position / orientation of the moving source and the moving destination of the tip of the robot, and the motor temperature data indicating the temperature or the estimated temperature of the motor represent the current state of the environment. Using the state observation unit for observing as a state variable, the judgment data acquisition unit for acquiring the motor temperature judgment data indicating the suitability judgment result of the motor temperature or the estimated temperature as judgment data, and the state variable and the judgment data. The machine learning device includes a learning unit that learns by associating the position / orientation of the movement source / destination of the tip of the robot, the temperature or the estimated temperature of the motor, and the movement command of the motor.

Another aspect of the present invention is a machine learning device that learns a movement command of the motor with respect to the operating environment of the robot in order to control so as to extend the life of the motor that drives each axis of the robot. The movement command data indicating the movement command of the robot, the reference position / attitude data indicating the position / orientation of the moving source and the moving destination of the tip of the robot, and the motor temperature data indicating the temperature or the estimated temperature of the motor represent the current state of the environment. A state observation unit that observes as a state variable, a judgment data acquisition unit that acquires motor temperature judgment data indicating the suitability judgment result of the motor temperature or the estimated temperature as judgment data, and a movement source / destination of the tip of the robot. Based on the learning unit learned by associating the position / orientation of the motor, the temperature or estimated temperature of the motor, and the movement command of the motor, the state variables observed by the state observation unit, and the learning result by the learning unit. It is a machine learning device including a decision-making unit that outputs a movement command of a motor that drives each axis of the robot.

According to the present invention, the robot can be controlled so as to extend the operating life of the motor that drives each axis of the robot by calculating the movement command of the motor that drives each axis of the robot by applying machine learning. become.

It is the schematic hardware block diagram of the control device by 1st Embodiment. It is a schematic functional block diagram of the control device by 1st Embodiment. It is a schematic functional block diagram which shows one form of a control device. It is a schematic flowchart which shows one form of the machine learning method. It is a figure explaining a neuron. It is a figure explaining a neural network. It is a schematic functional block diagram of the control device by 2nd Embodiment. It is a schematic functional block diagram which shows one form of the system which incorporated the control device. It is a figure which shows the example of a 7-axis articulated robot. It is a figure explaining the operation of a 7-axis articulated robot.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic hardware configuration diagram showing a main part of the control device according to the first embodiment. The control device 1 can be implemented as a control device that controls a manufacturing machine such as a robot. Further, the control device 1 includes a personal computer attached to a control device that controls a manufacturing machine such as a robot, a cell computer connected to the control device via a wired / wireless network, a host computer, an edge server, and a cloud server. It can be implemented as a computer such as. In this embodiment, an example in which the control device 1 is mounted as a control device for controlling a robot is shown.

The CPU 11 included in the control device 1 according to the present embodiment is a processor that controls the control device 1 as a whole. The CPU 11 reads the system program stored in the ROM 12 via the bus 20 and controls the entire control device 1 according to the system program. Temporary calculation data, display data, various data input by the operator via an input unit (not shown), and the like are temporarily stored in the RAM 13.

The non-volatile memory 14 is configured as a memory in which the storage state is maintained even when the power of the control device 1 is turned off, for example, by backing up with a battery (not shown). In the non-volatile memory 14, a program read from the external device 72 via the interface 15, a program input via the display / MDI unit 70, various data acquired from each part of the control device 1 and the robot (for example,). , The position and orientation of the robot (position of each axis), the temperature of the motor that drives each axis of the robot, etc.) are stored. The program and various data stored in the non-volatile memory 14 may be expanded in the RAM 13 at the time of execution / use. Further, various system programs such as a known analysis program (including a system program for controlling interaction with the machine learning device 100, which will be described later) are written in the ROM 12 in advance.

The interface 15 is an interface for connecting the control device 1 and an external device 72 such as an adapter. Programs, various parameters, etc. are read from the external device 72 side. Further, the programs and various parameters edited in the control device 1 can be stored in the external storage means via the external device 72. The PMC (programmable machine controller) 16 is a sequence program built in the control device 1 and outputs a signal to the robot and peripheral devices of the robot via the I / O unit 17 to control the robot. In addition, it receives signals from various switches on the operation panel installed in the main body of the robot, performs necessary signal processing, and then passes them to the CPU 11.

The display / MDI unit 70 is a manual data input device including a display, a keyboard, and the like, and the interface 18 receives commands and data from the keyboard of the display / MDI unit 70 and passes them to the CPU 11. The interface 19 is connected to an operation panel 71 provided with a manual pulse generator or the like used when manually driving each axis.

The axis control circuit 30 for controlling each axis that moves the joints of the robot receives the axis movement command amount from the CPU 11 and outputs the axis command to the servo amplifier 40. In response to this command, the servo amplifier 40 drives the servomotor 50 that moves the shaft included in the robot. The shaft servomotor 50 has a built-in position / speed detector, feeds back the position / speed feedback signal from the position / speed detector to the shaft control circuit 30, and performs position / speed feedback control. In the hardware configuration diagram of FIG. 1, only one axis control circuit 30, servo amplifier 40, and servo motor 50 are shown, but in reality, the number of axes provided in the robot to be controlled (for example). , If it is a 7-axis robot, only 7) will be prepared.

The interface 21 is an interface for connecting the control device 1 and the machine learning device 100. The machine learning device 100 stores a processor 101 that controls the entire machine learning device 100, a ROM 102 that stores a system program and the like, a RAM 103 that temporarily stores each process related to machine learning, a learning model, and the like. The non-volatile memory 104 used in the above is provided. The machine learning device 100 can observe each information that can be acquired by the control device 1 via the interface 21 (for example, the position and orientation of the robot (each axis position), the temperature of the motor that drives each axis of the robot, and the like). can. Further, the control device 1 receives the movement amount of each axis output from the machine learning device 100 and controls the movement of each axis of the robot.

FIG. 2 is a schematic functional block diagram of the control device 1 and the machine learning device 100 according to the first embodiment. In each functional block shown in FIG. 2, the CPU 11 included in the control device 1 shown in FIG. 1 and the processor 101 of the machine learning device 100 execute their respective system programs, and the control device 1 and the machine learning device 100 It is realized by controlling the operation of each part.

The control device 1 of the present embodiment includes a control unit 34 that controls the robot 2 based on a movement command of a motor that drives each axis of the robot 2 output from the machine learning device 100. The control unit 34 generally controls the operation of the robot according to the position and orientation of the moving destination of the tip of the robot hand of the robot 2 designated by a program or the like. At that time, the robot 2 moves from the machine learning device 100 to the moving destination. When the movement command of the motor that drives each axis when moving the tip of the robot hand of the robot 2 is output, the movement for moving the tip of the robot hand of the robot 2 to the position and orientation commanded by the program or the like is performed. , The movement command output from the machine learning device 100 is controlled.

On the other hand, the machine learning device 100 included in the control device 1 drives each axis of the robot 2 with respect to the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor that drives each axis of the robot 2. It includes software (learning algorithm, etc.) and hardware (processor 101, etc.) for self-learning a movement command of a motor by so-called machine learning. What the machine learning device 100 included in the control device 1 learns is the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2, the temperature of the motor that drives each axis of the robot 2, and each axis of the robot 2. It corresponds to a model structure that expresses the correlation with the movement command of the motor that drives the robot.

As shown by the functional blocks in FIG. 2, the machine learning device 100 included in the control device 1 has movement command data S1 indicating a movement command of a motor for driving each axis of the robot 2, and a movement source of the tip of the robot hand of the robot 2. A state observation unit 106 that observes as a state variable S that represents the current state of the environment including the reference position / orientation data S2 that indicates the position / orientation of the destination and the motor temperature data S3 that indicates the temperature of the motor that drives each axis of the robot 2. And the motor temperature determination data D1 for determining the change state of the motor temperature due to the movement of the tip of the robot hand of the robot 2 made based on the movement command of the motor driving each axis of the robot 2 determined. Using the judgment data acquisition unit 108 for acquiring the judgment data D, the state variable S, and the judgment data D, the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and each axis of the robot 2 are driven. It includes a learning unit 110 that learns by associating the temperature of the motor with the movement command of the motor that drives each axis of the robot 2.

Of the state variables S observed by the state observation unit 106, the movement command data S1 can be acquired as a movement command of the motor that drives each axis of the robot 2. Examples of the movement command of the motor that drives each axis of the robot 2 include an array of movement amounts (including positive and negative codes) output to the motor that drives each axis of the robot 2.

Based on the learning result of the learning unit 110, the movement command data S1 determines the position and orientation of the tip of the robot hand of the robot 2 and each axis of the robot 2 in the previous learning cycle. With respect to the temperature of the driving motor, the movement command of the motor driving each axis of the robot 2 determined in the learning cycle can be used as it is. When such a method is adopted, the machine learning device 100 temporarily stores the movement command of the motor driving each axis of the robot 2 in the RAM 103 for each learning cycle, and the state observing unit 106 temporarily stores the movement command from the RAM 103. The movement command of the motor that drives each axis of the robot 2 in the previous learning cycle may be acquired as the movement command data S1 of the current learning cycle.

Of the state variables S observed by the state observation unit 106, the reference position / orientation data S2 is the tip of the robot hand of the robot 2 acquired from the robot 2 or from a program used for controlling the robot 2 by the control device 1. Can be acquired as the current position / orientation (position / orientation of the movement source) and the position / orientation of the movement destination of the tip of the robot hand of the robot 2 acquired from the program used for controlling the robot 2 by the control device 1. .. The position and orientation of the movement source may be indicated by the position of each axis of the robot 2, but the position and orientation of the movement destination need not be indicated by the position of each axis of the robot 2. It may be indicated by the position and orientation of the tip of the robot hand.

Among the state variables S observed by the state observation unit 106, the motor temperature data S3 can be acquired as the temperature of the motor that drives each axis of the robot 2. For the temperature of the motor that drives each axis of the robot 2, when a temperature sensor is attached to the motor, the detection value of the temperature sensor can be used. Further, the temperature of the motor that drives each axis of the robot 2 is based on the drive history of each motor, the resistance value of the current flowing through the motor (winding), and the like even when the temperature sensor is not attached to the motor. It can be estimated based on. In the present specification, unless otherwise specified, the temperature of the motor includes the estimated temperature of the motor.

The determination data acquisition unit 108 determines the motor temperature after the tip of the robot hand of the robot 2 is moved, which is performed as the motor temperature determination data D1 based on the movement command of the motor that drives each axis of the robot 2. The results can be used. The motor temperature determination data D1 used by the determination data acquisition unit 108 includes, for example, the temperature of each motor acquired after the robot 2 actually operates based on the determined movement command of the motor that drives each axis of the robot 2. You may use the result judged by the judgment criteria set appropriately, such as whether it is within a predetermined predetermined threshold (appropriate) or exceeds the threshold (no). As the motor temperature determination data D1, for example, an evaluation value based on the temperature of the motor that drives each axis of the robot 2 may be calculated using a predetermined evaluation function, and the evaluation value may be used.

The determination data acquisition unit 108 has an indispensable configuration at the stage of learning by the learning unit 110, but the position and orientation of the moving source / destination of the tip of the robot hand of the robot 2 by the learning unit 110 and each axis of the robot 2. It is not always an indispensable configuration after the learning relating the temperature of the motor for driving the robot 2 and the movement command of the motor for driving each axis of the robot 2 is completed. For example, when the machine learning device 100 for which learning has been completed is shipped to the customer, the determination data acquisition unit 108 may be removed before shipping.

The state variable S simultaneously input to the learning unit 110 is based on the data one learning cycle before the determination data D was acquired, when considered in the learning cycle by the learning unit 110. In this way, while the machine learning device 100 included in the control device 1 advances learning, in the environment, the reference position / orientation data S2 and the motor temperature data S3 are acquired, and the movement command data S1 determined based on the acquired data is obtained. The movement of each axis of the robot 2 and the acquisition of the determination data D are repeatedly carried out based on the above.

The learning unit 110 follows the arbitrary learning algorithm collectively called machine learning, and the robot 2 has a movement source / destination position and orientation with respect to the position and orientation of the tip of the robot hand of the robot 2 and the temperature of the motor driving each axis of the robot 2. Learn the movement command of the motor that drives each axis. The learning unit 110 can iteratively execute learning based on the data set including the state variable S and the determination data D described above. The state during the repetition of the learning cycle of the movement command of the motor that drives each axis of the robot 2 with respect to the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor that drives each axis of the robot 2. As described above, the variable S is determined by the position and orientation of the moving source / destination of the tip of the robot hand of the robot 2 before one learning cycle, the temperature of the motor driving each axis of the robot 2, and one learning cycle before. The determination data D is acquired from the movement command of the motor that drives each axis of the robot 2 and is controlled based on the movement command of the motor that drives each axis of the determined robot 2 after the operation of the robot 2. It is the result of determining the suitability of the motor temperature.

By repeating such a learning cycle, the learning unit 110 determines the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2, the temperature of the motor that drives each axis of the robot 2, and each axis of the robot 2. It becomes possible to identify features that imply a correlation with the movement command of the motor that drives the robot. At the start of the learning algorithm, the correlation between the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor driving each axis of the robot 2 and the movement command of the motor driving each axis of the robot 2. Is substantially unknown, but the learning unit 110 gradually identifies the features and interprets the correlation as the learning progresses. The correlation between the position and orientation of the tip of the robot hand of the robot 2 and the temperature of the motor driving each axis of the robot 2 and the movement command of the motor driving each axis of the robot 2 is reliable to some extent. When interpreted to a level that can be interpreted, the learning result repeatedly output by the learning unit 110 is the current state (that is, the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the position and orientation of the motor that drives each axis of the robot 2). It can be used to select an action (that is, make a decision) as to how to determine a movement command of a motor that drives each axis of the robot 2 with respect to the temperature). That is, as the learning algorithm progresses, the learning unit 110 sets each axis of the robot 2 with respect to the position and orientation of the moving source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor driving each axis of the robot 2. The correlation with the behavior of how to set the movement command of the driving motor can be gradually approached to the optimum solution.

The decision-making unit 122 determines the movement command of the motor that drives each axis of the robot 2 based on the result learned by the learning unit 110, and controls the movement command of the motor that drives each of the determined axes of the robot 2. Output to 34. In the state where the learning by the learning unit 110 is completed, the decision-making unit 122 tells the machine learning device 100 the position and orientation of the moving source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor that drives each axis of the robot 2. Is input, a movement command of the motor that drives each axis of the robot 2 is output. The movement command of the motor that drives each axis of the robot 2 determined by the decision-making unit is obtained from the position / orientation of the movement source of the tip of the robot hand of the robot 2 included in the reference position / orientation data S2. It is a movement command that can be moved to the position command of the movement destination of the tip, and is expressed as a combination of the movement amounts of the motors that drive each axis of the robot 2. Since there can be a plurality of combinations of motor positions that can realize the position and orientation of the moving destination of the tip of the robot hand of the robot 2 included in the reference position and orientation data S2, the decision-making unit 122 sets the state variable S. Based on the result learned by the learning unit 110, the movement command of the motor that drives each axis of the appropriate robot 2 is determined.

As described above, the machine learning device 100 included in the control device 1 uses the state variable S observed by the state observation unit 106 and the judgment data D acquired by the judgment data acquisition unit 108, and the learning unit 110 uses the machine learning algorithm. According to this, the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the movement command of the motor driving each axis of the robot 2 with respect to the temperature of the motor driving each axis of the robot 2 are learned. The state variable S is composed of data such as movement command data S1, reference position / orientation data S2, and motor temperature data S3, and the determination data D is unique by analyzing the information acquired from the robot 2 by the control device 1. Is required. Therefore, according to the machine learning device 100 included in the control device 1, the learning result of the learning unit 110 is used to drive the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and each axis of the robot 2. It is possible to automatically and accurately issue a movement command of the motor that drives each axis of the robot 2 according to the temperature of the motor.

Then, if the movement command of the motor that drives each axis of the robot 2 can be automatically determined, the position / orientation (reference position / orientation data S2) of the movement source / destination of the tip of the robot hand of the robot 2 and By simply grasping the temperature of the motor that drives each axis of the robot 2 (motor temperature data S3), it is possible to quickly determine an appropriate value of the movement command of the motor that drives each axis of the robot 2. Therefore, it is possible to efficiently determine the movement command of the motor that drives each axis of the robot 2.

As a modification of the machine learning device 100 included in the control device 1 of the present embodiment, the state observation unit 106 further includes an environmental temperature in addition to the movement command data S1, the reference position / orientation data S2, and the motor temperature data S3. Environmental data S4 may be observed. For the ambient temperature, a value detected by a temperature sensor installed near the robot 2 or a value obtained by the operator looking at a thermometer or the like and input via the display / MDI unit 70 can be used. can.

According to the above modification, the machine learning device 100 can learn and determine the movement command of the motor that drives each axis of the robot 2 in consideration of the environmental temperature observed as the environmental data S4. When the temperature is low, the motor that drives each axis of the robot 2 that is appropriate according to the environmental temperature, such as determining a transfer command so as to apply a slight load to some high-speed moving motors. You will be able to determine the movement command.

As another modification of the machine learning device 100 included in the control device 1 of the present embodiment, the cycle time of the operation of the robot 2 is set as the determination data D for the determination data acquisition unit 108 to determine the suitability of the operation of the robot 2. The cycle time data D2 shown can also be used. The determination data acquisition unit 108 can use the time from the start to the end of the movement based on the movement command of the motor that drives each axis of the robot 2 as the cycle time of the operation of the robot 2.

According to the above modification, the machine learning device 100 determines the appropriate axes of the robot 2 while considering the trade-off between the temperature of each motor after the robot 2 operates and the cycle time of the operation of the robot 2. It becomes possible to determine the movement command of the driving motor. When the cycle time data D2 is introduced, it is preferable to make a judgment to shorten the cycle time of the operation of the robot 2 within a range in which the temperature of each motor does not rise too much when the robot 2 operates. ..

As another modification of the machine learning device 100 included in the control device 1 of the present embodiment, the state observation unit 106 adds the movement command data S1, the reference position / orientation data S2, and the motor temperature data S3 as state variables S. Further, the speed reducer temperature data S5 indicating the temperature of the speed reducer installed in the motor for driving each axis of the robot 2 may be observed. At that time, the determination data acquisition unit 108 uses the determination data D as the determination data D based on the movement command of the motor that drives each axis of the determined robot 2 in addition to the motor temperature determination data D1. The speed reducer temperature determination data D3 that determines the suitability of the speed reducer temperature due to the movement of the tip of the robot hand may be acquired. As the temperature of the speed reducer installed in the motor, a value detected by a temperature sensor installed in the vicinity of the speed reducer can be used.

According to the above modification, the machine learning device 100 considers the temperature of the speed reducer installed in the motor that drives each axis of the robot 2 by using the speed reducer temperature data S5 and the speed reducer temperature determination data D3. Therefore, it becomes possible to learn and determine the movement command of the motor that drives each axis of the robot 2, so that the movement command of the motor that drives each axis of the robot 2 is appropriate according to the temperature state of the reducer. Will be able to decide.

In the machine learning device 100 having the above configuration, the learning algorithm executed by the learning unit 110 is not particularly limited, and a learning algorithm known as machine learning can be adopted. FIG. 3 is a form of the control device 1 shown in FIG. 2, and shows a configuration including a learning unit 110 that executes reinforcement learning as an example of a learning algorithm. Reinforcement learning is a trial-and-error cycle of observing the current state (that is, input) of the environment in which the learning target exists, executing a predetermined action (that is, output) in the current state, and giving some reward to that action. It is a method of repeatedly learning a measure that maximizes the total reward (in the machine learning device of the present application, a movement command of a motor that drives each axis of the robot 2) as an optimum solution.

In the machine learning device 100 included in the control device 1 shown in FIG. 3, the learning unit 110 determines the movement command of the motor that drives each axis of the robot 2 based on the state variable S, and each of the determined robots 2. When the robot 2 operates based on the movement command of the motor that drives the axis, the suitability determination result of the operation of the robot 2 (corresponding to the determination data D used in the next learning cycle in which the state variable S is acquired). A reward calculation unit 112 that obtains a reward R related to the robot 2 and a value function update unit 114 that updates a function Q representing the value of a movement command of a motor that drives each axis of the robot 2 by using the reward R are provided. In the learning unit 110, the value function updating unit 114 repeatedly updates the function Q, so that the robot 2 has a position and orientation of the tip of the robot hand of the robot 2 and the temperature of the motor that drives each axis of the robot 2. Learn the movement command of the motor that drives each axis of.

An example of the reinforcement learning algorithm executed by the learning unit 110 will be described. The algorithm according to this example is known as Q-learning, and acts in the state s with the state s of the action subject and the action a that the action subject can select in the state s as independent variables. This is a method of learning a function Q (s, a) that represents the value of an action when a is selected. The optimum solution is to select the action a having the highest value function Q in the state s. By starting Q-learning in a state where the correlation between the state s and the action a is unknown and repeating trial and error to select various actions a in an arbitrary state s, the value function Q is repeatedly updated and optimized. Get closer to the solution. Here, when the environment (that is, the state s) changes as a result of selecting the action a in the state s, a reward (that is, a weighting of the action a) r corresponding to the change is obtained, and a higher reward is obtained. By inducing learning to select the action a in which r is obtained, the value function Q can be approached to the optimum solution in a relatively short time.

The update formula of the value function Q can be generally expressed as the following equation (2). In Equation 2, s _t and a _t is a state and behavior at each time t, the state by action a _t is changed to s _{t + 1.} r t _{+ 1} is a reward obtained by the state changes from s _t in s _{t + 1.} The term of maxQ means the Q when the action a which becomes the maximum value Q at the time t + 1 (which is considered at the time t) is performed. α and γ are learning coefficients and discount rates, respectively, and are arbitrarily set with 0 <α ≦ 1 and 0 <γ ≦ 1.

When the learning unit 110 executes Q learning, the state variable S observed by the state observation unit 106 and the judgment data D acquired by the judgment data acquisition unit 108 correspond to the update type state s, and correspond to the current state (that is, the robot). How to determine the movement command of the motor that drives each axis of the robot 2 with respect to the position and orientation of the movement source / destination of the tip of the robot hand 2 and the temperature of the motor that drives each axis of the robot 2. The action corresponds to the renewal type action a, and the reward R required by the reward calculation unit 112 corresponds to the renewal type reward r. Therefore, the value function update unit 114 repeatedly updates the function Q representing the value of the movement command of the motor driving each axis of the robot 2 with respect to the current state by Q learning using the reward R.

The reward R obtained by the reward calculation unit 112 is, for example, the operation of the robot 2 based on the movement command of the motor that drives each axis of the robot 2 after determining the movement command of the motor that drives each axis of the robot 2. When the suitability judgment result is judged to be "suitable" (for example, when the motor temperature threshold set for each motor is not exceeded after the operation of the robot 2, the cycle time of the operation is shortened, each speed reducer A positive (plus) reward R is set when the motor temperature threshold is not exceeded (for example, when the motor temperature threshold is not exceeded), and after determining the movement command of the motor that drives each axis of the robot 2, each of the determined robots 2 When the suitability determination result of the operation of the robot 2 based on the movement command of the motor driving the shaft is determined to be "No" (for example, when the motor temperature threshold set for each motor is exceeded after the operation of the robot 2). , When the motor temperature threshold set for each speed reducer is exceeded, etc.), a negative (minus) reward R can be set. The absolute values of the positive and negative rewards R may be the same or different from each other. Further, as a judgment condition, a plurality of values included in the judgment data D may be combined for judgment.

Further, the suitability determination result of the operation of the robot 2 based on the movement command of the motor that drives each axis of the set robot 2 can be set not only in two ways of "suitable" and "bad" but also in a plurality of stages. .. As an example, when the threshold value of the temperature of the motor that drives the axis of the robot 2 is T _max , and the temperature T of the motor after the operation of the robot 2 is 0 ≦ T <T _max / 5, the reward R = 5 is given. When T _max / 5 ≤ T <T _max / 2, the reward R = 3 is given, when T _max / 2 ≤ T <T _max , the reward R = 1, and when T _max ≤ T, the reward R = 3. It can be configured to give = -3 (negative reward).

Further, as a result of determining the suitability of the operation of the robot 2 based on the movement command of the motor that drives each axis of the set robot 2, it is possible to set a reward with a different weight for each motor. Motors have different windings, bearings, grease, structures, etc. for each type, and their durability also differs. Therefore, the robot 2 can be set by changing the weight of the reward for each motor, such as giving a reward so that the minus becomes larger when the temperature of the motor having low durability rises. It is possible to study in consideration of the structure of.

Further, the threshold value used for the determination may be set relatively large in the initial stage of learning, and the threshold value used for the determination may be reduced as the learning progresses.

The value function update unit 114 can have an action value table in which the state variable S, the determination data D, and the reward R are arranged in association with the action value (for example, a numerical value) represented by the function Q. In this case, the act of updating the function Q by the value function updating unit 114 is synonymous with the act of updating the action value table by the value function updating unit 114. At the start of Q-learning, the correlation between the current state of the environment and the movement command of the motor that drives each axis of the robot 2 is unknown. Therefore, in the action value table, various state variables S, judgment data D, and reward R are used. Is prepared in a form associated with a randomly determined action value value (function Q). If the determination data D is known, the reward calculation unit 112 can immediately calculate the reward R corresponding to the determination data D, and the calculated value R is written in the action value table.

When Q-learning is advanced using the reward R according to the suitability judgment result of the operation of the robot 2, the learning is guided in the direction of selecting the action that obtains a higher reward R, and the result of executing the selected action in the current state. The action value value (function Q) for the action performed in the current state is rewritten and the action value table is updated according to the state of the environment (that is, the state variable S and the determination data D) that changes as. By repeating this update, the value of the action value (function Q) displayed in the action value table is set to the proper action (in the case of the present invention, the temperature of the motor driving each axis of the robot 2 does not become high). It is rewritten so that the larger the value is, the larger the value is (the action of operating the robot 2). In this way, the current state of the unknown environment (the position and orientation of the moving source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor driving each axis of the robot 2) and the action for it (robot 2). The correlation with the movement command of the motor that drives each axis) is gradually clarified. That is, by updating the action value table, the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2, the temperature of the motor that drives each axis of the robot 2, and the movement command of the motor that drives each axis of the robot 2 The relationship with is gradually brought closer to the optimal solution.

With reference to FIG. 4, the above-mentioned Q-learning flow (that is, one form of the machine learning method) executed by the learning unit 110 will be further described. First, in step SA01, the value function update unit 114 drives each axis of the robot 2 as an action to be performed in the current state indicated by the state variable S observed by the state observation unit 106 while referring to the action value table at that time. Randomly select the movement command of the motor. Next, the value function update unit 114 takes in the state variable S of the current state observed by the state observation unit 106 in step SA02, and determines the current state acquired by the determination data acquisition unit 108 in step SA03. Capture D. Next, the value function update unit 114 determines in step SA04 whether or not the movement command of the motor driving each axis of the robot 2 is appropriate based on the determination data D, and if it is appropriate, in step SA05. , The positive reward R obtained by the reward calculation unit 112 is applied to the update formula of the function Q, and then in step SA06, the state variable S in the current state, the judgment data D, the reward R, and the value of the action value (the updated function). Update the action value table using Q) and. When it is determined in step SA04 that the movement command of the motor driving each axis of the robot 2 is not appropriate, the negative reward R obtained by the reward calculation unit 112 is applied to the update formula of the function Q in step SA07. Next, in step SA06, the action value table is updated using the state variable S in the current state, the judgment data D, the reward R, and the action value value (updated function Q). The learning unit 110 iteratively updates the action value table by repeating steps SA01 to SA07, and advances learning of the movement command of the motor that drives each axis of the robot 2. The process of obtaining the reward R and the process of updating the value function from step SA04 to step SA07 are executed for each data included in the determination data D.

For example, a neural network can be applied when advancing the reinforcement learning described above. FIG. 5A schematically shows a neuron model. FIG. 5B schematically shows a model of a three-layer neural network constructed by combining the neurons shown in FIG. 5A. The neural network can be configured by, for example, an arithmetic unit or a storage device that imitates a neuron model.

The neuron shown in FIG. 5A outputs the result y for a plurality of inputs x (here, as an example, inputs x ₁ to inputs x _3). Each input x ₁ ~x _3, the weight w corresponding to the input x (w ₁ ~w ₃₎ is multiplied. As a result, the neuron outputs the output y expressed by the following equation (3). In addition, in the equation 3, the input x, the output y, and the weight w are all vectors. Further, θ is a bias and f _k is an activation function.

In the three-layer neural network shown in FIG. 5B, a plurality of inputs x (here, as an example, inputs x1 to input x3) are input from the left side, and a result y (here, as an example, result y1 to result y3) is input from the right side. It is output. In the illustrated example, each of the inputs x1, x2, x3 is multiplied by the corresponding weight (collectively represented by w1), and the individual inputs x1, x2, x3 are all three neurons N11, N12, N13. It has been entered.

In FIG. 5B, the outputs of neurons N11 to N13 are collectively represented by z1. z1 can be regarded as a feature vector obtained by extracting the feature amount of the input vector. In the illustrated example, each of the feature vectors z1 is multiplied by a corresponding weight (collectively represented by w2), and each of the individual feature vectors z1 is input to the two neurons N21 and N22. The feature vector z1 represents a feature between the weights w1 and w2.

In FIG. 5B, the outputs of neurons N21 to N22 are collectively represented by z2. z2 can be regarded as a feature vector obtained by extracting the feature amount of the feature vector z1. In the illustrated example, each of the feature vectors z2 is multiplied by a corresponding weight (collectively represented by w3), and each of the individual feature vectors z2 is input to the three neurons N31, N32, and N33. The feature vector z2 represents a feature between the weights w2 and w3. Finally, the neurons N31 to N33 output the results y1 to y3, respectively.
It is also possible to use a so-called deep learning method using a neural network consisting of three or more layers.

In the machine learning device 100 included in the control device 1, the neural network is used as a value function in Q-learning, the state variable S and the action a are input x, and the learning unit 110 performs a multi-layered structure calculation according to the above-mentioned neural network. Therefore, the value (result y) of the action in the state can be output. The operation mode of the neural network includes a learning mode and a value prediction mode. For example, the weight w is learned using the learning data set in the learning mode, and the value of the action is used in the value prediction mode using the learned weight w. You can make a judgment. In the value prediction mode, detection, classification, inference, and the like can also be performed.

The configuration of the control device 1 described above can be described as a machine learning method (or software) executed by the processor 101. This machine learning method is a machine learning method for learning a movement command of a motor that drives each axis of the robot 2, and a computer CPU obtains movement command data S1, reference position / orientation data S2, and motor temperature data S3. , The step of observing as a state variable S representing the current state of the environment in which the robot 2 operates, and the determination data indicating the suitability determination result of the operation of the robot 2 based on the movement command of the motor that drives each axis of the determined robot 2. A step of acquiring D, and a step of learning by associating the reference position / orientation data S2 and the motor temperature data S3 with the movement command of the motor driving each axis of the robot 2 by using the state variable S and the determination data D. And have.

FIG. 6 is a schematic functional block diagram of the control device 1 and the machine learning device 100 according to the second embodiment, and has a configuration including a learning unit 110 that executes supervised learning as another example of the learning algorithm. It is shown. Supervised learning is new by being given a known dataset of inputs and their corresponding outputs (called supervised data) and identifying features from those teacher data that imply a correlation between inputs and outputs. It is a method of learning a correlation model for estimating the required output for an input.

The machine learning device 100 included in the control device 1 of the present embodiment is a movement indicating a movement command of a motor for driving each axis of the robot 2, which is appropriately performed for the state of the environment, instead of the determination data acquisition unit 108. A label data acquisition unit 109 for acquiring label data L including command data L1 is provided. The label data acquisition unit 109 acquires the movement command of the motor that drives each axis of the robot 2, which is considered to be appropriate in a certain state, as the movement command data L1. This movement command data L1 drives the position / orientation (reference position / orientation data S2) of the movement source / destination of the tip of the robot hand of the robot 2 and each axis of the robot 2 during the operation of the robot 2 performed in the past. The temperature of the motor (motor temperature data S3) and the movement command of the motor that drives each axis of the robot 2 are recorded as log data, and the log data is analyzed to drive the motor that drives each axis of the robot 2. The movement command data in which the robot 2 can be operated without raising the temperature to an unnecessarily high temperature may be acquired as appropriate movement command data (movement command data L1). What constitutes the appropriate movement command data may be considered in the same manner as the determination for the determination data D in the first embodiment.

The state observing unit 106 of the present embodiment does not need to observe the movement command data S1. Further, the label data acquisition unit 109, like the determination data acquisition unit 108, has an indispensable configuration at the stage of learning by the learning unit 110, but the movement source / movement destination of the tip of the robot hand of the robot 2 by the learning unit 110. It is not always an indispensable configuration after learning related to the position / orientation of the robot 2 and the temperature of the motor driving each axis of the robot 2 and the movement command of the motor driving each axis of the robot 2 is completed.

In the machine learning device 100 included in the control device 1 shown in FIG. 6, the learning unit 110 determines the robot from the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor that drives each axis of the robot 2. Correlation model M that estimates the movement command of the motor that drives each axis of 2 and the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 acquired in the past and the motor that drives each axis of the robot 2. The error calculation unit 116 that calculates the error E with the correlation feature identified from the teacher data T obtained from the result of the movement command of the motor that drives each axis of the robot 2 and the temperature of the robot 2, and the error calculation unit 116 that reduces the error E. It is provided with a model update unit 118 that updates the correlation model M so as to do so. In the learning unit 110, the model updating unit 118 repeatedly updates the correlation model M to obtain the position and orientation of the tip of the robot hand of the robot 2 from the moving source and the moving destination and the temperature of the motor driving each axis of the robot 2. Learn to estimate the movement command of the motor that drives each axis of the robot 2.

The initial value of the correlation model M is, for example, a simplified representation (for example, by an Nth-order function) of the correlation between the state variable S and the label data L, and is sent to the learning unit 110 before the start of supervised learning. Given. In the present invention, the teacher data T is appropriate for the position and orientation of the moving source / moving destination of the tip of the robot hand of the robot 2 and the temperature of the motor driving each axis of the robot 2 acquired in the past as described above. It is possible to use the data of the movement command of the motor that drives each axis of the robot 2 commanded to the above, and it is given to the learning unit 110 at any time when the control device 1 is operated. The error calculation unit 116 uses the teacher data T given to the learning unit 110 at any time to determine the position and orientation of the movement source / destination of the tip of the robot hand of the robot 2 and the temperature of the motor that drives each axis of the robot 2 and the robot 2. The correlation feature that implies the correlation with the movement command of the motor that drives each axis of the above is identified, and the error between this correlation feature and the correlation model M corresponding to the state variable S and the label data L in the current state. Find E. The model update unit 118 updates the correlation model M in a direction in which the error E becomes smaller, for example, according to a predetermined update rule.

In the next learning cycle, the error calculation unit 116 estimates the movement command of the motor that drives each axis of the robot 2 using the state variable S according to the updated correlation model M, and the estimation result and the actual state. The error E of the label data L acquired in the above is obtained, and the model update unit 118 updates the correlation model M again. In this way, the correlation between the current state of the unknown environment and the estimation for it gradually becomes clear. When the learning by the learning unit 110 is completed, the decision-making unit 122 uses the learning result of the learning unit 110 to determine the position and orientation of the tip of the robot hand of the robot 2 observed by the state observation unit 106. Based on (reference position / orientation data S2) and the temperature of the motor driving each axis of the robot 2 (motor temperature data S3), an appropriate movement command of the motor driving each axis of the robot 2 is estimated and output. Will be able to. In the second embodiment as well, as in the first embodiment, various state variables S can be observed.

FIG. 7 shows a system 170 according to a third embodiment including the control device 1. The system 170 uses at least one control device 1 mounted as a part of a computer such as a cell computer, a host computer, or a cloud server, a plurality of robots 2 to be controlled, and the control device 1 and the robot 2 with each other. It is provided with a wired / wireless network 172 to be connected.

In the system 170 having the above configuration, the control device 1 including the machine learning device 100 uses the learning result of the learning unit 110 to determine the position and orientation of the moving source / destination of the tip of the robot hand of the robot 2 and each of the robots 2. The movement command of the motor that drives each axis of the robot 2 with respect to the temperature of the motor that drives the shaft can be automatically and accurately obtained for each robot 2. Further, the machine learning device 100 of the control device 1 moves a motor that drives each axis of the robot 2 that is common to all the robots 2 based on the state variables S and the determination data D obtained for each of the plurality of robots 2. It can be configured to learn a command and share the learning result in all the actions of the robot 2. Therefore, according to the system 170, it is possible to improve the speed and reliability of learning the movement command of the motor that drives each axis of the robot 2 by inputting a more diverse data set (including the state variable S and the determination data D). Can be done.

Although the embodiments of the present invention have been described above, the present invention is not limited to the examples of the above-described embodiments, and can be implemented in various embodiments by making appropriate changes.

For example, the learning algorithm and the calculation algorithm executed by the machine learning device 100, the control algorithm executed by the control device 1, and the like are not limited to those described above, and various algorithms can be adopted.

Further, in the above-described embodiment, the control device 1 and the machine learning device 100 are described as devices having different CPUs, but the machine learning device 100 is based on the CPU 11 included in the control device 1 and the system program stored in the ROM 12. It may be realized.

1 Control device 2 Robot 11 CPU
12 ROM
13 RAM
14 Non-volatile memory 15, 18, 19 Interface 17 I / O unit 20 Bus 21 Interface 30 Axis control circuit 34 Control unit 40 Servo amplifier 50 Servo motor 70 Display / MDI unit 71 Operation panel 72 External device 100 Machine learning device 101 Processor 102 ROM
103 RAM
104 Non-volatile memory 106 State observation unit 108 Judgment data acquisition unit 109 Label data acquisition unit 110 Learning unit 112 Reward calculation unit 114 Value function update unit 116 Error calculation unit 118 Model update unit 122 Decision-making unit 170 System 172 Network

Claims (11)

A control device that controls to extend the life of the motor that drives each axis of the robot.
A machine learning device for learning a movement command of the motor with respect to the operating environment of the robot is provided.
The machine learning device
The movement command data indicating the movement command of the motor, the reference position / orientation data indicating the position / orientation of the moving source and the moving destination of the tip of the robot, and the motor temperature data indicating the temperature to the estimated temperature of the motor are displayed in the current state of the environment. The state observation unit that observes as a state variable representing
A judgment data acquisition unit that acquires motor temperature judgment data indicating the suitability judgment result of the motor temperature or the estimated temperature as judgment data, and a judgment data acquisition unit.
Using the state variables and the determination data, a learning unit that learns by associating the position and orientation of the moving source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the movement command of the motor with each other.
A control device comprising. The state observing unit further observes the speed reducer temperature data indicating the temperature or the estimated temperature of the speed reducer installed in the motor as the state variable.
The determination data acquisition unit further acquires the reducer temperature determination data indicating the suitability determination result of the temperature of the reducer or the estimated temperature as the determination data.
The learning unit uses the state variable and the determination data to determine the position / orientation of the moving source / destination of the tip of the robot, the temperature or estimated temperature of the motor, the temperature or estimated temperature of the speed reducer, and the above. Learn by associating with the movement command of the motor,
The control device according to claim 1. The learning unit
The remuneration calculation unit that obtains the remuneration related to the suitability judgment result,
Using the reward, the value function update that updates the function representing the position and orientation of the movement source / destination of the tip of the robot and the value of the movement command of the motor that drives each axis of the robot with respect to the temperature or the estimated temperature of the motor. Department and
With
The reward calculation unit gives a reward high enough to keep the temperature or estimated temperature of the motor low.
The control device according to claim 1 or 2. The learning unit calculates the state variable and the determination data in a multi-layer structure.
The control device according to any one of claims 1 to 3. A control device that controls to extend the life of the motor that drives each axis of the robot.
A machine learning device that learns the movement command of the motor with respect to the operating environment of the robot is provided.
The machine learning device
The movement command data indicating the movement command of the motor, the reference position / orientation data indicating the position / orientation of the moving source and the moving destination of the tip of the robot, and the motor temperature data indicating the temperature to the estimated temperature of the motor are displayed in the current state of the environment. The state observation unit that observes as a state variable representing
A learning unit that learns by associating the position and orientation of the movement source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the movement command of the motor.
A decision-making unit that outputs a movement command of a motor that drives each axis of the robot based on the state variables observed by the state observation unit and the learning result of the learning unit.
A control device comprising. The machine learning device exists in the cloud server.
The control device according to any one of claims 1 to 5. A control device that controls to extend the life of the motor that drives each axis of the robot.
A machine learning device for learning a movement command of the motor with respect to the operating environment of the robot is provided.
The machine learning device
A state observing unit that observes reference position / orientation data indicating the position / orientation of the moving source and moving destination of the tip of the robot and motor temperature data indicating the temperature or estimated temperature of the motor as state variables representing the current state of the environment. ,
A label data acquisition unit that acquires movement command data indicating the movement command of the motor as label data, and
Using the state variables and the label data, a learning unit that learns by associating the position and orientation of the moving source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the moving command of the motor with each other.
A control device comprising. The learning unit
An error calculation unit that calculates an error between a correlation model that estimates the movement command of the motor from the state variable and a correlation feature that is identified from the teacher data prepared in advance.
A model update unit that updates the correlation model so as to reduce the error is provided.
The control device according to claim 7. A control device that controls to extend the life of the motor that drives each axis of the robot.
A machine learning device that learns the movement command of the motor with respect to the operating environment of the robot is provided.
The machine learning device
A state observing unit that observes reference position / orientation data indicating the position / orientation of the moving source and moving destination of the tip of the robot and motor temperature data indicating the temperature or estimated temperature of the motor as state variables representing the current state of the environment. ,
A learning unit that learns by associating the position and orientation of the movement source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the movement command of the motor.
A decision-making unit that outputs a movement command of a motor that drives each axis of the robot based on the state variables observed by the state observation unit and the learning result of the learning unit.
A control device comprising. A machine learning device that learns movement commands of the motor with respect to the operating environment of the robot in order to control so as to extend the life of the motor that drives each axis of the robot.
The movement command data indicating the movement command of the motor, the reference position / orientation data indicating the position / orientation of the moving source and the moving destination of the tip of the robot, and the motor temperature data indicating the temperature to the estimated temperature of the motor are displayed in the current state of the environment. The state observation unit that observes as a state variable representing
A judgment data acquisition unit that acquires motor temperature judgment data indicating the suitability judgment result of the motor temperature or the estimated temperature as judgment data, and a judgment data acquisition unit.
Using the state variables and the determination data, a learning unit that learns by associating the position and orientation of the moving source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the movement command of the motor with each other.
A machine learning device equipped with. A machine learning device that learns a movement command of the motor with respect to the operating environment of the robot in order to control so as to extend the life of the motor that drives each axis of the robot.
The movement command data indicating the movement command of the motor, the reference position / orientation data indicating the position / orientation of the moving source and the moving destination of the tip of the robot, and the motor temperature data indicating the temperature to the estimated temperature of the motor are displayed in the current state of the environment. The state observation unit that observes as a state variable representing
A learning unit that learns by associating the position and orientation of the movement source / destination of the tip of the robot, the temperature or estimated temperature of the motor, and the movement command of the motor.
A decision-making unit that outputs a movement command of a motor that drives each axis of the robot based on the state variables observed by the state observation unit and the learning result of the learning unit.
A machine learning device equipped with.

Images