JP2019162712A - Control device, machine learning device and system - Google Patents

Abstract

To provide a control device and a machine learning device which can optimize polishing quality.SOLUTION: A control device 1 controlling a robot which polishes a workpiece comprises a machine learning device 100 that learns a polishing condition for performing polishing. The machine learning device 100 comprises: a state observing part 106 that observes characteristics of a surface state of the polished workpiece and the polishing condition as state variables representing a current state of an environment; a determination data acquiring part 108 that acquires determination data indicating an evaluated result of the surface state of the polished workpiece; and a learning part 110 that uses the state variables and the determination data to associate the characteristics of the surface state of the polished workpiece and the polishing condition with each other and learns them.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a control device, a machine learning device, and a system, and more particularly to a control device, a machine learning device, and a system that optimize polishing quality.

  Conventionally, when a robot performs a polishing operation on a machine part or the like, the polishing quality confirmation operation is generally performed by human visual inspection. Further, in order to improve the polishing quality, it is necessary to repeatedly perform the test polishing while changing various conditions such as the operation speed of the robot, the pressing force, the number of rotations of the polishing tool, and the torque.

  Patent Document 1 describes a flash grinding robot that alternately repeats a measurement operation for measuring the remaining flash height with a sensor and a cutting operation. Patent Document 2 describes a method in which an inspection robot monitors defects in a surface state of a workpiece using an imaging unit.

JP-A-7-246552 JP-A-5-196444

  It takes a lot of labor and time to obtain a desired polishing quality by trial and error by hand. In this respect, neither Patent Document 1 nor Patent Document 2 discloses specific technical means for automatically optimizing the polishing quality.

  Therefore, a control device, a machine learning device, and a system that optimize the polishing quality are desired.

A control device according to an aspect of the present invention is a control device that controls a robot that polishes a workpiece, and includes a machine learning device that learns polishing conditions when performing the polishing, and the machine learning device includes: A state observation unit for observing characteristics of the surface state of the workpiece after polishing and the polishing conditions as state variables representing the current state of the environment, and an evaluation result of the surface state of the workpiece after polishing A determination data acquisition unit that acquires determination data, and a learning unit that learns the characteristics of the surface state of the workpiece after the polishing and the polishing conditions in association with each other using the state variable and the determination data. .
A machine learning device according to another aspect of the present invention is a machine learning device that learns polishing conditions when a workpiece is polished by a robot, and the characteristics of the surface state of the workpiece after the polishing and the polishing A state observing unit for observing the condition as a state variable representing the current state of the environment, a determination data acquiring unit for acquiring determination data indicating an evaluation result of the surface state of the workpiece after the polishing, and the state variable A learning unit that learns the characteristics of the surface state of the workpiece after the polishing and the polishing conditions in association with each other using the determination data.
A system according to another aspect of the present invention is a system in which a plurality of devices are connected to each other via a network, and the plurality of devices includes a control device having at least a machine learning device. It is a system including

  The present invention can provide a control device and a machine learning device that optimize polishing quality.

2 is a hardware configuration diagram of a control device 1. FIG. It is a functional block diagram of the control apparatus 1 by 1st Embodiment. 2 is a functional block diagram showing one form of a control device 1. FIG. It is a schematic flowchart which shows one form of the machine learning method. It is a figure explaining a neuron. It is a figure explaining a neural network. It is a functional block diagram of the control apparatus 1 by 2nd Embodiment. It is a figure which shows the example of the system of 3 hierarchy structure containing a cloud server, a fog computer, and an edge computer. It is a functional block diagram which shows one form of the system incorporating the control apparatus. It is a functional block diagram which shows the other form of the system incorporating the control apparatus 1. It is a functional block diagram which shows the other form of the system incorporating the control apparatus 1. FIG. 11 is a schematic hardware configuration diagram of the computer shown in FIG. 10. It is a functional block diagram which shows the other form of the system incorporating the control apparatus 1. It is a schematic model diagram of the robot which grind | polishes. It is a schematic model diagram of the robot which grind | polishes. It is a figure which shows an example of the surface state of a workpiece | work. 2 is a functional block diagram showing one form of a control device 1. FIG.

  FIG. 1 is a schematic hardware configuration diagram showing a control device 1 according to an embodiment of the present invention and main parts of an industrial robot controlled by the control device 1. The control device 1 is a control device that controls, for example, an industrial robot that performs polishing (hereinafter simply referred to as a robot). The control device 1 includes a CPU 11, ROM 12, RAM 13, nonvolatile memory 14, interface 18, interface 19, interface 21, interface 22, bus 20, axis control circuit 30, and servo amplifier 40. A servo motor 50, a teaching operation panel 60, a polishing tool 70, and an imaging device 80 are connected to the control device 1.

  The CPU 11 is a processor that controls the control device 1 as a whole. The CPU 11 reads the system program stored in the ROM 12 via the interface 22 and the bus 20 and controls the entire control device 1 according to the system program.

  The ROM 12 stores in advance a system program (including a system program for controlling interaction with the machine learning device 100 described later) for executing various types of control of the robot.

  The RAM 13 temporarily stores temporary calculation data and display data, data input by the operator via a teaching operation panel 60 described later, and the like.

  The non-volatile memory 14 is backed up by a battery (not shown), for example, and retains the storage state even when the control device 1 is powered off. The nonvolatile memory 14 stores data input from the teaching operation panel 60, robot control programs and data input via an interface (not shown), and the like. The program and data stored in the nonvolatile memory 14 may be expanded in the RAM 13 at the time of execution and use.

  The axis control circuit 30 controls axes such as joints of arms provided in the robot. The axis control circuit 30 receives an axis movement command amount output from the CPU 11 and outputs an axis movement command to the servo amplifier 40.

  The servo amplifier 40 drives the servo motor 50 in response to an axis movement command output from the axis control circuit 30.

  The servo motor 50 is driven by the servo amplifier 40 to move the axis of the robot. The servo motor 50 typically includes a position / speed detector. The position / velocity detector outputs a position / velocity feedback signal, and this signal is fed back to the axis control circuit 30 to perform position / velocity feedback control.

  In FIG. 1, only one axis control circuit 30, servo amplifier 40, and servo motor 50 are shown, but in actuality, as many axes as the number of axes provided in the robot to be controlled are prepared. For example, when controlling a robot having six axes, a total of six sets of the axis control circuit 30, servo amplifier 40, and servo motor 50 corresponding to each axis are prepared.

  The teaching operation panel 60 is a manual data input device provided with a display, a handle, a hardware key, and the like. The teaching operation panel 60 displays the information received from the CPU 11 via the interface 18 on the display. The teaching operation panel 60 passes pulses, commands, data, and the like input from the handle, hardware keys, and the like to the CPU 11 via the interface 18.

  The polishing tool 70 is held at the tip of the arm of the robot and polishes a workpiece (workpiece) with a rotating grindstone. The polishing tool 70 performs polishing with a rotation speed, a rotation torque, and a pressing force based on a command received from the CPU 11 via the interface 19.

  The imaging device 80 is a device for photographing the surface state of the workpiece, and is a vision sensor, for example. The imaging device 80 captures the surface state of the workpiece according to a command received from the CPU 11 via the interface 22. The imaging device 80 passes the captured image data to the CPU 11 via the interface 22.

  The interface 21 is an interface for connecting the control device 1 and the machine learning device 100. The machine learning apparatus 100 includes a processor 101, a ROM 102, a RAM 103, and a nonvolatile memory 104.

  The processor 101 controls the entire machine learning device 100. The ROM 102 stores system programs and the like. The RAM 103 performs temporary storage in each process related to machine learning. The nonvolatile memory 104 stores a learning model and the like.

  The machine learning device 100 provides various information that can be acquired by the control device 1 (rotation speed, rotation torque and pressing force of the polishing tool 70, operation speed of the arm of the robot, image data acquired by the imaging device 80, etc.) in the interface 21. Observe through. The machine learning device 100 outputs a command for controlling the servo motor 50 and the polishing tool 70 to the control device 1 via the interface 21. The control device 1 receives a command from the machine learning device 100 and corrects the control command of the robot.

13 and 14 are schematic diagrams illustrating an example of a robot controlled by the control device 1.
The robot shown in FIG. 13 includes an arm that freely moves when the servomotor 50 is driven. A polishing tool 70 with an image pickup device 80 (vision sensor) is provided at the tip of the arm, and the polishing tool 70 polishes the surface of the workpiece to be polished. After polishing, the imaging device 80 takes an image of the surface state of the workpiece as shown in FIG.

  FIG. 2 is a schematic functional block diagram of the control device 1 and the machine learning device 100 according to the first embodiment. The machine learning device 100 includes a state observation unit 106, a determination data acquisition unit 108, and a learning unit 110. The state observation unit 106, the determination data acquisition unit 108, and the learning unit 110 can be realized as one function of the processor 101, for example. Alternatively, for example, it may be realized by the processor 101 executing software stored in the ROM 102.

  The state observation unit 106 observes a state variable S that represents the current state of the environment. The state variable S includes a rotational speed S1 of the polishing tool 70, a rotational torque S2 of the polishing tool 70, a pressing force S3 of the polishing tool 70, an operating speed S4 of the robot arm, and a feature S5 of the surface state of the workpiece.

  The state observation unit 106 acquires the rotational speed S1, the rotational torque S2, and the pressing force S3 of the polishing tool 70 from the control device 1. The control device 1 can acquire these values from a motor of the polishing tool 70 or a sensor added to the polishing tool 70.

  The state observation unit 106 obtains the robot arm operating speed S4 from the control device 1. The control device 1 can acquire this value from the servo motor 50 or a sensor attached to the arm.

  The state observation unit 106 acquires the surface state feature S <b> 5 of the workpiece from the control device 1. The surface condition feature S5 of the workpiece is data indicating a feature extracted from the image of the surface state of the workpiece photographed by the imaging device 80 after polishing. For example, the feature S5 of the surface state of the workpiece can be acquired by extracting the feature amount included in the image of the surface state of the workpiece by the function of the imaging device 80 or by the image processing software provided in the control device 1. The imaging device 80 or the control device 1 can automatically extract feature quantities indicating the density (depth), smoothness, spacing, and the like of the surface of the workpiece by a known method such as deep learning.

  FIG. 15 shows an example of an image of the surface state of the work taken by the imaging device 80 after polishing. In this way, various darkness (depth), smoothness and spacing streaks are left on the surface of the workpiece after polishing. The state observation unit 106 recognizes such a muscle feature from the image, and extracts it as the feature S5 of the surface state of the workpiece.

  The determination data acquisition unit 108 acquires determination data D, which is an index indicating a result when the robot performs polishing under the state variable S. The determination data D includes a stripe density D1, a stripe smoothness D2, and a stripe interval D3 in the image of the surface state of the workpiece taken by the imaging device 80 after polishing.

  For example, the image processing software included in the function of the imaging device 80 or the image processing software included in the control device 1 analyzes the image of the surface state of the workpiece photographed by the imaging device 80 after polishing, whereby the density D1 of the streaks, the smoothness D2, and the interval Each D3 can be digitized and output. Alternatively, the operator visually evaluates the image of the surface state of the workpiece photographed by the imaging device 80 after polishing, and gives a value indicating the evaluation result (for example, appropriate = “1”, not = “0”) to the teaching operation panel 60. D1, D2 and D3 may be given by inputting from.

  As a modification, the determination data D may include the rotational torque D4 of the polishing tool 70. It is known that the rotational torque D4 has a correlation with the smoothness of the workpiece surface. The determination data D may include the temperature D5 of the polishing tool 70. It is known that the temperature D5 has a correlation with an appropriate pressing force.

  Using the state variable S and the determination data D, the learning unit 110 uses the surface condition feature S5 of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70, the robot arm operation). The correlation with the speed S4) is learned. That is, the learning unit 110 generates a model structure indicating the correlation between the components S1, S2, S3, S4, and S5 of the state variable S.

  When the state variable S input to the learning unit 110 is considered in the learning cycle in the learning unit 110, the state variable S is based on data one learning cycle before the determination data D is acquired. While the machine learning apparatus 100 proceeds with learning, in the environment, (1) acquisition of the surface condition feature S5 of the workpiece, (2) rotation speed S1, rotation torque S2, pressing force S3 of the polishing tool 70, robot arm Setting of the operation speed S4, that is, setting of polishing conditions, (3) execution of polishing according to the above (1) and (2), and (4) acquisition of the determination data D are repeatedly performed. The rotational speed S1, rotational torque S2, pressing force S3, and robot arm operating speed S4 of the polishing tool 70 in (2) are polishing conditions obtained based on learning results up to the previous time. The determination data D in (4) is an evaluation result of polishing performed according to the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70, and the operating speed S4 of the robot arm.

  By repeating such a learning cycle, the learning unit 110 determines the surface condition S5 of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70, and the operating speed S4 of the robot arm). ) And features that imply a correlation can be automatically identified. At the start of the learning algorithm, the correlation between the surface condition feature S5 of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4 of the polishing tool 70) is substantially The learning unit 110 gradually identifies features and interprets the correlation as the learning proceeds. The correlation between the surface condition feature S5 of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4 of the polishing tool 70) is interpreted to a certain level of reliability. The learning result that the learning unit 110 repeatedly outputs is the polishing condition (the rotational speed S1, the rotational torque S2, the pressing force S3, the robot S3 of the polishing tool 70) for the current state, that is, the surface state feature S5 of the workpiece. It can be used to select an action (decision making) as to whether the arm operating speed S4) should be set. That is, the learning unit 110 can output an optimal solution of behavior corresponding to the current state.

  The state variable S is composed of data that is hardly affected by disturbance, and the determination data D is uniquely obtained by acquiring the analysis result of the image data of the imaging device 80 from the control device 1. Therefore, according to the machine learning device 100, by using the learning result of the learning unit 110, the optimum polishing conditions (the rotation speed S1, the rotation torque S2, the rotation torque S2, the polishing tool 70) for the feature S5 of the current state, that is, the surface state of the workpiece. The pressing force S3 and the robot arm operating speed S4) can be automatically and accurately obtained without calculation or calculation. In other words, the optimum polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4 of the polishing tool 70) can be obtained only by grasping the feature S5 of the current state, that is, the surface state of the workpiece. Can be determined quickly. Therefore, it is possible to efficiently set the polishing conditions in the robot polishing.

  As a modified example of the machine learning device 100, the learning unit 110 uses the state variable S and the determination data D obtained for each of a plurality of robots that perform the same work, and sets appropriate polishing conditions common to all the robots. Can learn. According to this configuration, the amount of the data set including the state variable S and the determination data D obtained in a certain time can be increased and more diverse data sets can be input, so that the learning speed and reliability can be improved. it can.

  Note that the learning algorithm executed by the learning unit 110 is not particularly limited, and a learning algorithm known as machine learning can be employed. FIG. 3 shows one configuration of the control device 1 shown in FIG. 2 and includes a learning unit 110 that performs reinforcement learning as an example of a learning algorithm. Reinforcement learning is a trial-and-error cycle that observes the current state (ie, input) of the environment where the learning target exists, executes a predetermined action (ie, output) in the current state, and gives some reward to that action. This is a method of learning, as an optimal solution, a policy (in this embodiment, setting of the polishing conditions) that iteratively maximizes the total reward.

  In the machine learning device 100 included in the control device 1 illustrated in FIG. 3, the learning unit 110 includes a reward calculation unit 112 and a value function update unit 114.

  The reward calculation unit 112 rewards R related to the evaluation result of polishing when the polishing condition is set based on the state variable S (corresponding to the determination data D used in the next learning cycle in which the state variable S is acquired). Ask for.

  The value function updating unit 114 updates the function Q representing the value of the polishing condition using the reward R. When the value function update unit 114 repeats the update of the function Q, the learning unit 110 determines the surface condition of the workpiece S5 and the polishing conditions (the rotation speed S1, the rotation torque S2, the pressing force S3 of the polishing tool 70, the robot force). The correlation with the arm operating speed S4) is learned.

  An example of the reinforcement learning algorithm executed by the learning unit 110 will be described. The algorithm according to this example is known as Q-learning (Q-learning), and the behavior s in the state s is defined as an independent variable with the behavior s state s and the behavior a that the behavior subject can select in the state s. This is a method for learning a function Q (s, a) representing the value of an action when a is selected. The optimal solution is to select the action a that has the highest value function Q in the state s. The value function Q is iteratively updated by repeating trial and error by starting Q learning in a state where the correlation between the state s and the action a is unknown, and selecting various actions a in an arbitrary state s. Approach the solution. Here, when the environment (that is, the state s) changes as a result of selecting the action a in the state s, a reward (that is, the weight of the action a) r corresponding to the change is obtained, and a higher reward By inducing learning to select an action a that gives r, the value function Q can be brought close to the optimal solution in a relatively short time.

The updating formula of the value function Q can be generally expressed as the following formula 1. In equation (1), s _t and a _t is a state and behavior at each time t, the state by action a _t is changed to s _{t + 1.} r t _{+ 1} is a reward obtained by the state changes from s _t in s _{t + 1.} The term maxQ means Q when the action a having the maximum value Q at time t + 1 (and considered at time t) is performed. α and γ are a learning coefficient and a discount rate, respectively, and are arbitrarily set such that 0 <α ≦ 1 and 0 <γ ≦ 1.

  When the learning unit 110 executes Q-learning, the state variable S observed by the state observation unit 106 and the determination data D acquired by the determination data acquisition unit 108 correspond to the update state s, and the current state, that is, the surface of the workpiece The behavior of how to determine the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4 of the polishing tool 70) with respect to the state feature S5 is an updateable behavior. A reward R that corresponds to a and is calculated by the reward calculation unit 112 corresponds to an update-type reward r. Therefore, the value function updating unit 114 repeatedly updates the function Q representing the value of setting the polishing condition with respect to the current state by Q learning using the reward R.

  For example, the reward calculation unit 112 performs polishing based on the determined polishing conditions (the rotation speed S1, the rotation torque S2, the pressing force S3, and the robot arm operation speed S4 of the polishing tool 70). ”Can be a positive (plus) value. On the other hand, when the polishing evaluation result is determined as “No”, the reward R can be set to a negative (minus) value. The absolute values of the positive and negative rewards R may be the same or different.

  If the determination data D is given in multiple values, for example, a difference between a value D1 indicating the density of a line, a value D2 indicating smoothness, a value D3 indicating an interval, and a predetermined reference value for each. Can be determined to be “appropriate” when the polishing evaluation result is within a predetermined range. On the other hand, if it is outside the predetermined range, it can be determined as “No”. When the determination data D is given as a binary value, for example, when D1, D2, and D3 are given as appropriate values such as “1” and not = “0”, if the input is “1”, polishing evaluation is performed. If the result is “appropriate” and “0”, the polishing evaluation result can be determined as “no”.

  The evaluation result of polishing can be set not only in two ways of “appropriate” and “not” but also in a plurality of stages. For example, the reward calculation unit 112 decreases the reward R as D1, D2, and D3 are further from the reference value, that is, as the difference between D1, D2, and D3 and the reference value that is predetermined for each increases. Can be configured.

  The reward calculation unit 112 may determine suitability by combining a plurality of values included in the determination data D.

  The value function updating unit 114 can have an action value table in which the state variable S, the determination data D, and the reward R are arranged in association with the action value (for example, a numerical value) represented by the function Q. In this case, the action that the value function updating unit 114 updates the function Q is synonymous with the action that the value function updating unit 114 updates the action value table. At the start of Q-learning, the correlation between the surface condition S5 of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4 of the polishing tool 70) is unknown. Therefore, in the behavior value table, various state variables S, determination data D, and rewards R are prepared in a form associated with randomly determined behavior value values (function Q). The reward calculation unit 112 can immediately calculate the reward R corresponding to the determination data D, and the calculated value R is written in the action value table.

  When Q learning is advanced using the reward R according to the evaluation result of polishing, the learning is guided in a direction to select an action that can obtain a higher reward R, and the environment changes as a result of executing the selected action in the current state In accordance with the state (that is, the state variable S and the determination data D), the action value value (function Q) for the action performed in the current state is rewritten, and the action value table is updated. By repeating this update, the value of the action value (function Q) displayed in the action value table is rewritten so that the more appropriate the action, the larger the value. In this way, the characteristic S5 of the current state of the environment that is unknown, that is, the surface condition of the workpiece, and the action corresponding thereto, that is, the polishing conditions that are set (the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70, the robot) The correlation with the operating speed S4) of the arm gradually becomes clear. In other words, by updating the behavior value table, there is a correlation between the feature S5 of the surface state of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4). Gradually approach the optimal solution.

With reference to FIG. 4, the Q learning flow (that is, one form of the machine learning method) executed by the learning unit 110 will be further described.
Step SA01: The value function updating unit 114 refers to the polishing condition (the rotation speed of the polishing tool 70) as the action to be performed in the current state indicated by the state variable S observed by the state observation unit 106 while referring to the action value table at that time. S1, rotational torque S2, pressing force S3, and robot arm operating speed S4) are randomly selected.
Step SA02: The value function updating unit 114 takes in the state variable S of the current state that is being observed by the state observation unit 106.
Step SA03: The value function updating unit 114 takes in the determination data D in the current state acquired by the determination data acquisition unit 108.
Step SA04: Based on the determination data D, the value function updating unit 114 determines whether or not the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, the robot arm operating speed S4) are appropriate. to decide. If appropriate, the process proceeds to step SA05. If not, the process proceeds to step SA07.
Step SA05: The value function updating unit 114 applies the positive reward R obtained by the reward calculating unit 112 to the function Q update formula.
Step SA06: The value function updating unit 114 updates the action value table using the state variable S, the determination data D, the reward R, and the action value (updated function Q) in the current state.
Step SA07: The value function updating unit 114 applies the negative reward R obtained by the reward calculating unit 112 to the update formula of the function Q.

  The learning unit 110 repeats steps SA01 to SA07 to repeatedly update the behavior value table, and advances the learning. It should be noted that the processing for obtaining the reward R and the value function updating processing from step SA04 to step SA07 are executed for each data included in the determination data D.

  FIG. 16 shows another configuration of the control device 1 shown in FIG. 2, and shows a configuration including a learning unit 110 that performs supervised learning as another example of the learning algorithm. Supervised learning is different from the above-described reinforcement learning in which learning is started in a state where the relationship between input and output is unknown, and a large number of known data sets (called teacher data) of the input and the corresponding output are given in advance. Thus, by identifying features implicating the correlation between the input and the output from the teacher data, a correlation model for estimating the required output for the new input (in the machine learning device 100 of the present application, the robot polishing is performed). This is a technique for learning polishing conditions).

  In the machine learning device 100 shown in FIG. 16, the learning unit 110 has a correlation identified from a correlation model M for deriving a polishing condition at the time of polishing by the robot from the state variable S and the determination data D and a teacher data T prepared in advance. An error calculation unit 116 that calculates an error E from the sex feature and a model update unit 118 that updates the correlation model M so as to reduce the error E are provided. The learning unit 110 learns the polishing conditions for polishing by the robot by causing the model updating unit 118 to repeatedly update the correlation model M.

  The initial value of the correlation model M is, for example, a simplified representation (for example, a linear function) of the correlation between the state variable S and the determination data D and the polishing conditions at the time of polishing by the robot. It is given to the learning unit 110 before starting. The teacher data T is, for example, an empirical value accumulated by recording polishing conditions determined by a skilled worker in past polishing (a feature of the surface condition of the workpiece and a known polishing condition at the time of polishing by a robot). And is given to the learning unit 110 before supervised learning is started. The error calculation unit 116 identifies a correlation feature that implies a correlation between the feature of the surface state of the workpiece and the polishing condition at the time of polishing by the robot from the large amount of teacher data T given to the learning unit 110, An error E between the correlation feature and the correlation model M corresponding to the state variable S and the determination data D in the current state is obtained. The model update unit 118 updates the correlation model M in a direction in which the error E becomes smaller, for example, according to a predetermined update rule.

  In the next learning cycle, the error calculation unit 116 uses the state variable S and the determination data D that have been changed by attempting polishing according to the updated correlation model M, and uses the changed state variable S and the determination data D as the changed state variable S and determination data D. The error E is obtained for the corresponding correlation model M, and the model update unit 118 updates the correlation model M again. In this way, the correlation between the unknown current state of the environment (features of the surface state of the workpiece) and the behavior (determination of the polishing conditions during polishing by the robot) gradually becomes clear. That is, by updating the correlation model M, the relationship between the characteristics of the surface state of the workpiece and the polishing conditions when polishing by the robot is gradually brought closer to the optimal solution.

  In the machine learning device 100, the learning unit 110 performs supervised learning at the initial stage of learning, and learning is performed as an initial polishing condition value at the time of polishing by the robot obtained by supervised learning when the learning progresses to some extent. The unit 110 may be configured to perform reinforcement learning. Since the initial value in reinforcement learning has a certain degree of reliability, the optimal solution can be reached relatively quickly.

  When performing reinforcement learning or supervised learning, for example, a neural network can be used instead of Q learning. FIG. 5A schematically shows a model of a neuron. FIG. 5B schematically shows a model of a three-layer neural network configured by combining the neurons shown in FIG. 5A. The neural network can be configured by, for example, an arithmetic device or a storage device imitating a neuron model.

The neuron shown in FIG. 5A outputs a result y for a plurality of inputs x (here, as an example, inputs x ₁ to x ₃ ). Each input x ₁ ~x _3, the weight w corresponding to the input x (w ₁ ~w ₃₎ is multiplied. As a result, the neuron outputs an output y expressed by the following equation (2). In Equation 2, the input x, the output y, and the weight w are all vectors. Further, θ is a bias, and f _k is an activation function.

  In the three-layer neural network shown in FIG. 5B, a plurality of inputs x (in this example, inputs x1 to x3) are input from the left side, and a result y (in this case, as an example, results y1 to y3) is input from the right side. Is output. In the illustrated example, each of the inputs x1, x2, and x3 is multiplied by a corresponding weight (generically expressed as W1), and each of the inputs x1, x2, and x3 is assigned to three neurons N11, N12, and N13. Have been entered.

  In FIG. 5B, the outputs of the neurons N11 to N13 are collectively represented by z1. z1 can be regarded as a feature vector obtained by extracting the feature amount of the input vector. In the illustrated example, each feature vector z1 is multiplied by a corresponding weight (generically represented by W2), and each feature vector z1 is input to two neurons N21 and N22. The feature vector z1 represents a feature between the weight W1 and the weight W2.

In FIG. 5B, the outputs of the neurons N21 to N22 are collectively represented by z2. z2 can be regarded as a feature vector obtained by extracting the feature amount of the feature vector z1. In the illustrated example, each feature vector z2 is multiplied by a corresponding weight (generically represented by W3), and each feature vector z2 is input to three neurons N31, N32, and N33. The feature vector z2 represents a feature between the weight W2 and the weight W3. Finally, the neurons N31 to N33 output the results y1 to y3, respectively.
It is also possible to use a so-called deep learning method using a neural network having three or more layers.

  In the machine learning device 100, the learning unit 110 performs a multilayer structure calculation according to a neural network using the state variable S and the determination data D as input x, thereby polishing conditions (rotation speed S1, rotation torque S2 of the polishing tool 70). The pressing force S3 and the robot arm operating speed S4) can be output as the result y. Further, in the machine learning device 100, the neural network is used as a value function in reinforcement learning, the state variable S and the action a are input x, and the learning unit 110 performs a multilayer structure operation according to the neural network, so that the state is obtained. It is also possible to output the value (result y) of the action at. The operation mode of the neural network includes a learning mode and a value prediction mode. For example, the weight w is learned using the learning data set in the learning mode, and the value of the action in the value prediction mode using the learned weight w. Judgment can be made. In the value prediction mode, detection, classification, inference, etc. can be performed.

  The configuration of the control device 1 described above can be described as a machine learning method (or program) executed by the processor 101. This machine learning method is a machine learning method that learns polishing conditions (rotation speed S1, rotation torque S2, pressing force S3, operation speed S4 of a robot arm) of a polishing tool 70 by a robot. The step of observing the feature S5 of the surface state of the workpiece as a state variable S representing the current state of the environment in which polishing is performed, and the set polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70). , Using the state variable S and the determination data D to obtain the determination data D indicating the evaluation result of the polishing performed according to the operation speed S4) of the robot arm, the surface condition feature S5 of the workpiece, and the polishing Learning by associating the conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the operating speed S4 of the robot arm of the polishing tool 70). And a step that.

FIG. 6 shows a control device 2 according to a second embodiment of the present invention. The control device 2 includes a machine learning device 120 and a state data acquisition unit 3.
The state data acquisition unit 3 acquires, as state data S0, the feature S5 of the surface state of the workpiece and the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4). And supplied to the state observation unit 106. The state data acquisition unit 3 can acquire the state data S0 from, for example, the control device 2 and various devices and sensors included in the robot.

  The machine learning device 120 includes a decision making unit 122 in addition to the state observation unit 106, the determination data acquisition unit 108, and the learning unit 110. The decision making unit 122 can be realized as one function of the processor 101, for example. Alternatively, for example, it may be realized by the processor 101 executing software stored in the ROM 102.

  The machine learning device 120 is software (learning algorithm) for self-learning polishing conditions (rotation speed S1, polishing torque S2, pressing force S3, and robot arm operating speed S4 of the polishing tool 70) by machine learning. Etc.) and hardware (processor 101, etc.), as well as the polishing conditions (rotation speed S1, rotation torque S2, pressing force S3, robot arm operation speed S4 of the polishing tool 70) obtained based on the learning result are controlled. Software (arithmetic algorithm etc.) and hardware (processor 101 etc.) for outputting as a command to the apparatus 2 are included. The machine learning device 120 may have a configuration in which one common processor executes all software such as a learning algorithm and an arithmetic algorithm.

  Based on the result learned by the learning unit 110, the decision deciding unit 122 polishes the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70, and the robot arm) corresponding to the surface condition feature S5 of the workpiece. A command value C including a command for determining the operation speed S4) is generated. When the decision making unit 122 outputs the command value C to the control device 2, the control device 2 controls the robot according to the command value C. Thus, the state of the environment changes.

  The state observation unit 106 observes the state variable S that has changed as a result of the command determination unit 122 outputting the command value C to the environment in the next learning cycle. The learning unit 110 uses the changed state variable S to update the value function Q (that is, the action value table), for example, thereby polishing conditions for polishing by the robot (the rotation speed S1, the rotation torque S2, and the pressing of the polishing tool 70). The force S3 and the robot arm operating speed S4) are learned. At that time, the state observation unit 106 obtains the state data S0 from which the state data acquisition unit 3 acquires the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4). Instead of acquiring from, it may be observed from the RAM 103 of the machine learning device 120 as described in the first embodiment.

  Then, the decision making unit 122 again gives the command value C for instructing the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force S3, the operating speed S4 of the robot arm) obtained based on the learning result. Output to the control device 2. By repeating this learning cycle, the machine learning device 120 advances the learning, and the reliability of the polishing conditions (the rotation speed S1, the rotation torque S2, the pressing force S3, and the robot arm operation speed S4) determined by the machine learning device 120 itself. Gradually improve sex.

  The machine learning device 120 has the same effect as the machine learning device 100 of the first embodiment. In addition, the machine learning device 120 can change the state of the environment according to the output of the decision making unit 122. In the machine learning device 100, the learning result of the learning unit 110 can be reflected in the environment by obtaining a function corresponding to the decision making unit 122 from an external device.

  In the following third to fifth embodiments, the control devices 1 and 2 according to the first and second embodiments include a plurality of cloud servers, host computers, fog computers, and edge computers (robot controllers, control devices, etc.). An embodiment in which the apparatus is connected to each other via a wired / wireless network will be described. As illustrated in FIG. 7, in the following third to fifth embodiments, a layer including a cloud server 6 and the like, a layer including a fog computer 7 and the like, and an edge in a state where each of a plurality of devices is connected to a network. Assume a system that is logically divided into three layers including a computer 8 (a robot controller, a control device, and the like included in the cell 9). In such a system, the control devices 1 and 2 can be mounted on any of the cloud server 6, the fog computer 7, and the edge computer 8, and are networked with a plurality of devices. The learning data is shared with each other through distributed learning, the generated learning models are collected in the fog computer 7 and the cloud server 6 for a large-scale analysis, and the generated learning models are further reciprocated. You can use it. In the system illustrated in FIG. 7, a plurality of cells 9 are provided in factories in various places, and each cell 9 is arranged in a predetermined unit (a factory unit, a plurality of factory units of the same manufacturer, etc.) Manage. The data collected and analyzed by the fog computer 7 is further collected and analyzed by the cloud server 6 of the upper layer, and the information obtained as a result can be used for control of each edge computer. .

  FIG. 8 shows a system 170 according to a third embodiment in which a plurality of robots are added to the control devices 1 and 2. The system 170 includes a plurality of robots 160 and a robot 160 '. All the robots 160 and 160 'are connected to each other by a wired or wireless network 172.

  The robot 160 and the robot 160 ′ have mechanisms required for the same purpose work and perform the same work. On the other hand, the robot 160 includes the control devices 1 and 2, but the robot 160 ′ does not include the same control device as the control devices 1 and 2.

  The robot 160 having the control devices 1 and 2 uses the learning result of the learning unit 110 to polish the polishing conditions (the rotational speed S1, the rotational torque S2, the pressing force of the polishing tool 70) corresponding to the feature S5 of the surface state of the workpiece. S3, the robot arm operating speed S4) can be automatically and accurately determined without calculation or calculation. In addition, the control device 2 of at least one robot 160 uses the state variable S and the determination data D obtained for each of the other plurality of robots 160 and 160 ′, and is common to all the robots 160 and 160 ′. Polishing conditions (polishing speed S1, rotating torque S2, pressing force S3, robot arm operating speed S4 of the polishing tool 70) in the polishing by the robot that performs the learning are learned, and all the robots 160 and 160 ′ share the learning results. Can be configured to According to the system 170, a more diverse data set (including the state variable S and the determination data D) is used as an input, and polishing conditions for polishing by the robot (the rotational speed S1, the rotational torque S2, the pressing force S3 of the polishing tool 70, the robot) The learning speed and reliability of the arm operating speed S4) can be improved.

  FIG. 9 shows a system 170 according to a fourth embodiment comprising a plurality of robots 160 '. The system 170 includes a plurality of robots 160 ′ having the same machine configuration and the machine learning device 120 (or the machine learning device 100). The plurality of robots 160 ′ and the machine learning device 120 (or the machine learning device 100) are connected to each other via a wired or wireless network 172.

  The machine learning device 120 (or the machine learning device 100) is based on the state variables S and the determination data D obtained for each of the plurality of robots 160 ′, and polishing conditions (polishing) for polishing by the robots common to all the robots 160 ′. The tool 70 learns the rotational speed S1, the rotational torque S2, the pressing force S3, and the robot arm operating speed S4). The machine learning device 120 (or the machine learning device 100) uses the learning result to polish the polishing conditions (rotation speed S1, rotation torque S2, pressing force S3 of the polishing tool 70, the robot's surface state feature S5, the robot, The arm operating speed S4) can be automatically and accurately obtained without calculation or calculation.

  The machine learning device 120 (or the machine learning device 100) may be mounted on a cloud server, a fog computer, an edge computer, or the like. According to this configuration, the necessary number of robots 160 ′ are connected to the machine learning device 120 (or the machine learning device 100) when necessary, regardless of the location and timing of each of the plurality of robots 160 ′. Can do.

  The system 170 or an operator engaged in the system 170 is subjected to polishing conditions (rotation of the polishing tool 70) by the machine learning device 120 (or machine learning device 100) at an appropriate time after the learning by the machine learning device 120 (or 100) is started. Learning degree of speed S1, rotational torque S2, pressing force S3, robot arm operating speed S4 (that is, polishing conditions to be output (rotational speed S1, rotational torque S2, pressing force S3 of polishing tool 70), robot It can be determined whether the reliability of the arm operating speed S4) has reached the required level.

  FIG. 10 shows a system 170 according to a fifth embodiment provided with a control device 1. The system 170 is implemented as at least one machine learning device 100 ′ mounted on the computer 5 such as an edge computer, a fog computer, a host computer, or a cloud server, and a control device (edge computer) that controls the robot 160. And at least one control device 1 and a wired / wireless network 172 that connects the computer 5 and the robot 160 to each other.

  In the system 170 having the above configuration, the computer 5 including the machine learning device 100 ′ is obtained as a result of machine learning by the machine learning device 100 included in the control device 1 from the control device 1 that controls each robot 160. Get a learning model. Then, the machine learning device 100 ′ included in the computer 5 generates and optimizes a learning model that is newly optimized or made efficient by performing knowledge optimization and efficiency processing based on the plurality of learning models. The learning model is distributed to the control device 1 that controls each robot 160.

  An example of optimization or efficiency improvement of the learning model performed by the machine learning device 100 ′ includes generation of a distillation model based on a plurality of learning models acquired from each control device 1. In this case, the machine learning device 100 ′ according to the present embodiment creates input data to be input to the learning model, and uses an output obtained as a result of inputting the input model to each learning model. A new learning model (distillation model) is generated by learning from the above. As described above, the distillation model generated in this way is more suitable for distribution to other computers via an external storage medium or a network.

As another example of optimization or efficiency improvement of the learning model performed by the machine learning device 100 ′, in the process of performing distillation on a plurality of learning models acquired from each control device 1, the output of each learning model with respect to input data It is also possible to analyze the distribution of the data by a general statistical method, extract outliers of the input data and output data sets, and perform distillation using the input data and output data sets excluding the outliers. . Through this process, exceptional estimation results are excluded from the input data and output data pairs obtained from each learning model, and the input data and output data pairs are excluded from the exceptional estimation results. Can be used to generate a distillation model. The distillation model generated in this way can generate a general-purpose distillation model for the robot 160 controlled by the control device 1 from the learning models generated by the plurality of control devices 1.
In addition, other general learning model optimization or efficiency methods (such as analyzing each learning model and optimizing the hyperparameters of the learning model based on the analysis result) can be introduced as appropriate. is there.

  In the system 170 according to the present embodiment, for example, the machine learning device 100 ′ is arranged on a computer 5 as a fog computer installed with respect to a plurality of robots 160 (control devices 1) as edge computers, and each robot 160. The learning model generated by the (control device 1) is aggregated and stored on a fog computer, and is optimized or made efficient after performing optimization or efficiency based on the stored learning models. The learning model can be redistributed to each robot 160 (control device 1) as necessary.

  Further, in the system 170 according to the present embodiment, for example, learning models stored in an aggregate on the computer 5 serving as a fog computer, or learning models optimized or made efficient on the fog computer are further converted into higher-order host computers. Collected on cloud servers and cloud servers, and using these learning models to apply to intelligent work at factories and robot 160 manufacturers (construction and redistribution of more general-purpose learning models on higher-level servers, results of learning model analysis) Based on the maintenance work, analysis of the performance of each robot 160, application to the development of new machines, etc.).

FIG. 11 is a schematic hardware configuration diagram of the computer 5 shown in FIG.
A CPU 511 provided in the computer 5 is a processor that controls the computer 5 as a whole. The CPU 511 reads the system program stored in the ROM 512 via the bus 520 and controls the entire computer 5 according to the system program. The RAM 513 temporarily stores temporary calculation data, various data input by the operator via the input device 531, and the like.

  The non-volatile memory 514 includes, for example, a memory backed up by a battery (not shown), an SSD (Solid State Drive), and the like, and the storage state is maintained even when the computer 5 is turned off. The non-volatile memory 514 stores a setting area in which setting information related to the operation of the computer 5 is stored, data input from the input device 531, a learning model acquired from each robot 160 (control device thereof), an external (not shown) Data read via a storage device or a network is stored. The program and various data stored in the nonvolatile memory 514 may be expanded in the RAM 513 at the time of execution / use. In addition, a system program including a known analysis program for analyzing various data is written in the ROM 512 in advance.

  The computer 5 is connected to the network 172 via the interface 516. At least one robot 160 and other computers are connected to the network 172, and exchange data with the computer 5.

The display device 530 outputs and displays data, etc. obtained as a result of executing each data, program, etc. read on the memory via the interface 517. Further, the input device 531 configured by a keyboard, a pointing device, and the like passes commands, data, and the like based on operations by the worker to the CPU 511 via the interface 518.
The machine learning device 100 has the same hardware configuration as that described in FIG. 1 except that the machine learning device 100 is used for optimization or efficiency improvement of the learning model in cooperation with the CPU 511 of the computer 5.

  FIG. 12 shows a system 170 according to a sixth embodiment provided with a control device 1. The system 170 includes a plurality of control devices 1 implemented as control devices (edge computers) that control the robot 160, a plurality of other robots 160 (control devices 1), and a wired / wireless network that connects them to each other. 172.

  In the system 170 having the above configuration, the control device 1 including the machine learning device 100 receives state data and determination data acquired from the robot 160 to be controlled, and other robots 160 (not including the machine learning device 100). Machine learning based on the acquired state data and determination data is performed to generate a learning model. The learning model generated in this way is used for determining the polishing conditions in the polishing operation of the robot 160 controlled by itself, and in response to a request from another robot 160 ′ that does not include the machine learning device 100. It is also used to determine the polishing conditions in the polishing operation by another robot 160 (control device thereof). Further, when the control device 1 including the machine learning device 100 before the generation of the learning model is newly introduced, the learning model is acquired from another control device 1 including the learning model via the network 172. It can also be used.

  In the system according to the present embodiment, it is possible to share and use data and learning models used for learning among a plurality of robots 160 (control device 1) as so-called edge computers, so that the efficiency of machine learning is improved. Or reducing the cost of machine learning (for example, introducing the machine learning device 100 only in one control device (control device 1) for controlling the robot 160 and sharing it with other robots 160). Can do.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and can be implemented in various modes by making appropriate changes.

  For example, the learning algorithm executed by the machine learning device 100 or the machine learning device 120, the arithmetic algorithm executed by the machine learning device 120, the control algorithm executed by the control device 1 or the control device 2 are not limited to those described above, Various algorithms can be adopted.

  In the above embodiment, the control device 1 (or the control device 2) and the machine learning device 100 (or the machine learning device 120) are described as devices having different CPUs, but the machine learning device 100 ( Alternatively, the machine learning device 120) may be configured to be realized by a CPU 11 provided in the control device 1 (or the control device 2) and a system program stored in the ROM 12.

DESCRIPTION OF SYMBOLS 1, 2 Control apparatus 3 Status data acquisition part 5 Computer 6 Cloud server 7 Fog computer 8 Edge computer 9 Cell 11 CPU
12 ROM
13 RAM
14 Nonvolatile memory 18, 19, 21, 22 Interface 20 Bus 30 Axis control circuit 40 Servo amplifier 50 Servo motor 60 Teaching operation panel 70 Polishing tool 80 Imaging device 100 Machine learning device 101 Processor 102 ROM
103 RAM
104 non-volatile memory 106 state observation unit 108 determination data acquisition unit 110 learning unit 112 reward calculation unit 114 value function update unit 116 error calculation unit 118 model update unit 120 machine learning device 122 decision determination unit 160, 160 ′ robot 170, 170 ′ System 172 Network 511 CPU
512 ROM
513 RAM
514 Non-volatile memory 516, 516, 517 Interface 520 Bus 530 Display device 531 Input device

Claims (13)

A control device for controlling a robot for polishing a workpiece,
A machine learning device for learning polishing conditions when performing the polishing;
The machine learning device is characterized by observing the surface condition of the workpiece after the polishing and the polishing condition as a state variable representing a current state of the environment,
A determination data acquisition unit for acquiring determination data indicating an evaluation result of the surface state of the workpiece after the polishing;
A control device, comprising: a learning unit that learns the characteristics of the surface state of the workpiece after the polishing and the polishing conditions in association with each other using the state variable and the determination data. Of the state variables, the polishing condition includes at least one of a rotation speed of a polishing tool, a rotation torque of the polishing tool, a pressing force of the polishing tool, and an operation speed of a robot,
2. The control device according to claim 1, wherein the determination data includes at least one of a stripe density D <b> 1 on the surface of the workpiece after the polishing, a smoothness D <b> 2 of the stripe, and an interval D <b> 3 of the stripes. The learning unit
A reward calculation unit for obtaining a reward related to the evaluation result;
The control apparatus according to claim 1, further comprising: a value function updating unit that updates a function representing a value of the polishing condition with respect to a feature of the surface state of the workpiece after the polishing by using the reward. The learning unit
An error calculation unit that calculates an error between a correlation model for deriving a polishing condition when performing the polishing from the state variable and the determination data and a correlation feature identified from teacher data prepared in advance;
A model updating unit that updates the correlation model so as to reduce the error,
The control device according to claim 1 or 2. The control device according to claim 1, wherein the learning unit calculates the state variable and the determination data in a multilayer structure. The control device according to any one of claims 1 to 5, further comprising a decision-making unit that outputs a command value based on the polishing condition based on a learning result by the learning unit. The control device according to claim 1, wherein the learning unit learns the polishing condition using the state variable and the determination data obtained from a plurality of the robots. The control device according to claim 1, wherein the machine learning device is realized by a cloud computing, fog computing, or edge computing environment. A machine learning device that learns polishing conditions when a workpiece is polished by a robot,
A state observation unit that observes the characteristics of the surface state of the workpiece after the polishing and the polishing conditions as state variables representing the current state of the environment;
A determination data acquisition unit for acquiring determination data indicating an evaluation result of the surface state of the workpiece after the polishing;
A machine learning device, comprising: a learning unit that learns the characteristics of the surface state of the workpiece after polishing and the polishing conditions in association with each other using the state variable and the determination data. A system in which a plurality of devices are connected to each other via a network,
The system includes a first robot including at least the control device according to claim 1. The plurality of devices includes a computer equipped with a machine learning device,
The computer acquires at least one learning model generated by learning in the learning unit of the control device,
The machine learning device provided in the computer performs optimization or efficiency based on the acquired learning model.
The system according to claim 10. The plurality of devices include a second robot different from the first robot,
The learning result by the learning unit provided in the control device provided in the first robot is shared with the second robot.
The system according to claim 10. The plurality of devices include a second robot different from the first robot,
The data observed in the second robot can be used for learning by a learning unit provided in a control device provided in the first robot via the network.
The system according to claim 10.

Images