KR102139229B1 - Collision Detection Method and System of Robot Manipulator Using Artificial Neural Network - Google Patents

Abstract

The present invention relates to a system for detecting a collision of a robot manipulator using an artificial neural network, which may comprise: a joint driving unit which is provided in a plurality of joints of the robot manipulator and drives each of the joints; an encoder unit which is provided on one side of the joint driving unit and measures angles of the plurality of joints; and a neural network operation unit which learns a neural network with a large amount of data, and infers to sense that the plurality of joints collide with the outside through preprocessing operation.

Description

Collision Detection Method and System of Robot Manipulator Using Artificial Neural Network

The present invention relates to a method and system for detecting a collision of a robot manipulator using an artificial neural network, and relates to a technique for detecting a collision by a robot manipulator through learning using an artificial neural network.

It has a function similar to that of the human upper extremity, and holds the object by means of a mechanical hand or the like corresponding to its tip to move it spatially, or spray gun, welding torch, etc. attached to the tip. It means that work such as painting and welding can be performed using the tool. Here, phage refers to those that are grasped, adsorbed, and maintained.

Similar to the human upper extremity means that a hand is attached to an arm, but it is like a human being without a hand attached to a tool, or holding a tool in a hand to perform work. It can be treated as something similar to the upper extremity.

Also, the mechanical hand may include an adsorber and a hook. Moreover, manipulators operated by humans are referred to as manual manipulators, and although they are out of safety rules, those with manipulators that operate based on information from memory (fixed sequence controllers, variable sequence controllers) are classified as industrial robots. Doing.

On the other hand, a six-character induction robot manipulator mainly used for automation can perform precise motion control. When the robot collides with the operator when operating according to the programmed motion, the robot may be injured because the robot outputs a large torque to continuously move to the desired position. Therefore, in order to ensure the safety of the worker working in the same space as the robot, a collision detection function is required to detect the collision of the robot itself and stop the driving operation.

In the prior art, the robot attaches a torque sensor for each joint to detect a collision, and detects an external collision through a value measured by the torque sensor. Through this, it is possible to sensitively detect the collision. However, if a torque sensor is attached to all joints of a 6-axis multi-joint robot, economic problems may occur.

In addition, a number of methods have been proposed to estimate external collisions using the dynamics and kinematics information of a robot even without a joint torque sensor. Since the robot dynamics information or a formula for a nonlinear function is used, a problem that a system load is increased may occur.

Due to these conventional problems, there is a need for a method and a system for robot to detect collision more efficiently than conventional techniques.

Republic of Korea Patent Publication No. 10-2019-0114083

The present invention is to solve the problems of the prior art as described above, the purpose is to detect the collision sensitively by learning the data about the collision detection through the artificial neural network equipped with a multi-axis robot.

A system for detecting collision of a robot manipulator using an artificial neural network according to the present invention for achieving the above object, comprising: a joint driving unit provided at a plurality of joints of the robot manipulator to drive each of the plurality of joints; An encoder unit provided on one side of the joint driving unit to measure an angle of the plurality of joints; It characterized in that it comprises; a neural network operation unit for learning the neural network with a large amount of data and inferring to detect that the plurality of joints collide with the outside through a pre-processing operation.

Here, the pre-processing operation is characterized in that generalization of the neural network is performed through cycle normalization.

In addition, the cycle normalization defines a plurality of designated waypoints, passes through the sequence from the first waypoint to the last waypoint among the designated waypoints, and then moves the path back to the first waypoint. Characterized in that to define.

In addition, a memory for storing information related to the cycle is further included, and the entire signal of the defined cycle is defined as a reference cycle and stored in the memory.

In addition, it characterized in that it further comprises a control unit for applying and applying a signal related to the control input to control the driving joint.

In addition, the signal is characterized in that it includes the joint angle, joint angular velocity and control torque obtained at every control cycle.

In addition, the signal is used as the input of the neural network, and characterized in that the pattern of the signal is analyzed by a sliding technique.

In addition, the neural network is characterized by learning a collision signal through supervised learning based on learning data consisting of a pair of signals and labels.

In addition, the neural network is characterized in that it is a binary signal that outputs a True signal when a collision occurs and a False signal when a collision does not occur.

In addition, learning the collision signal is characterized in that the collision is measured through the contact of a pressure sensor or a force sensitive resistor (FSR) provided on one side of the plurality of joints.

In addition, the last layer among the layers of the neural network is a probability value between 0 and 1, and it is characterized in that collision occurs when the probability value is 0.5 or more according to the inference of the neural network.

On the other hand, a method of detecting a collision of a robot manipulator using an artificial neural network according to the present invention for achieving the above object comprises: defining a plurality of way points to perform a pre-processing operation through a cycle normalization by a neural network calculation unit; Obtaining a collision signal from the collision measurement unit; A neural network learning based on a collision signal; The learned neural network may perform control by transmitting a signal that detects a collision in real time.

Method and system for detecting a collision of a robot manipulator using an artificial neural network according to the present invention has the following effects.

First, it is possible to sensitively detect collisions without changing performance. In the present invention, real-time collision detection can be performed using only the joint angle and the joint angular velocity measured by the encoder attached to the joint, which is essential for controlling the current and robot applied to the robot.

Second, it is economical. In the present invention, since a real-time collision detection can be performed without a separate performance change, such as having a plurality of torque sensors in the joint, the cost can be reduced compared to the prior art.

1 is a perspective view of a robot manipulator equipped with a system for detecting a collision of a robot manipulator according to an embodiment of the present invention.
2 is a block diagram of a system for detecting a collision of a robot manipulator according to an embodiment of the present invention.
3 is a flowchart of a method for detecting a collision of a robot manipulator according to an embodiment of the present invention.
4 is a diagram of a flow for detecting whether a collision has occurred in a method for detecting a collision of a robot manipulator according to an embodiment of the present invention.
5 is a diagram schematically showing a process in which a neural network learns learning data in a method of detecting a collision of a robot manipulator according to an embodiment of the present invention.
FIG. 6 is a diagram showing a result of experiment and comparison of a system for detecting a collision of a robot manipulator and a conventional momentum observer detecting a collision according to an embodiment of the present invention.
7 is a view showing a result of experiment and comparison with a system for detecting a collision of a robot manipulator according to an embodiment of the present invention by reducing a boundary value for collision detection of a momentum observer in the experiment illustrated in FIG. 6.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. However, some components irrelevant to the subject matter of the present invention will be omitted or compressed, but the omitted components are not necessarily required in the present invention, and are used in combination by those skilled in the art to which the present invention pertains. Can.

<Robot Manipulator Collision Detection System>

1 is a perspective view of a robot manipulator equipped with a system for detecting a collision of a robot manipulator according to an embodiment of the present invention, and FIG. 2 is a block diagram of a system for detecting a collision of a robot manipulator according to an embodiment of the present invention .

1 and 2, the robot manipulator 200 equipped with a system for detecting a collision of the robot manipulator 200 according to an embodiment of the present invention includes a plurality of joints or axes, and a plurality of The joints can operate by rotating based on the provided rotation axis.

In addition, the robot manipulator 200 collision detection system 100 may include a joint driving unit 110, an encoder unit 120, a neural network computing unit 130, a control unit 140, and a collision measurement unit 150.

The joint driving unit 110 is a component that drives the joint so that the robot manipulator 200 can operate. The joint driving unit 110 may be provided with a motor, for example, and may be provided in plural in each of a plurality of joints forming the robot manipulator 200. The joint driving unit 110 may drive the joint according to a signal transmitted from the control unit 140. Here, the signals included in controlling the joint are signals related to the joint angle, signals related to the angular angular velocity and control error obtained through the operation of the controller 140 from the joint angle, and current signals applied to the joint motor. The signal included in controlling such a joint may also be referred to as a robot signal.

The encoder unit 120 is provided on one side of the joint driving unit 110 to measure the angle of the joint.

The neural network operation unit 130 is provided with a neural network, and the neural network performs learning on a collision signal based on the learning data, and the learned neural network detects a collision by inferring an input signal in real time. The neural network computing unit 130 may be configured with, for example, General-Purpose computing on Graphics Processing Units (GPGPU). In addition, the neural network calculator 130 may undergo a pre-processing operation to facilitate detection of collision when the trained neural network infers the collision. In this case, the normalization performance of the neural network may be improved through cycle normalization in the pre-processing operation. If the neural network does not achieve generalization, if the motion of the robot manipulator 200 is changed or the weight of the object held by the robot manipulator 200 is changed, the detection performance for collision may be changed and reliability may be reduced. However, if the generalization performance is improved through cycle normalization as in the present invention, the motion of the robot manipulator 200 (for example, the speed and path of the robot manipulator 200), the weight of the object held by the robot manipulator 200 Even if changed, the detection performance of the collision of the neural network can be maintained at a high level.

The control unit 140 may apply a control input to the robot manipulator 200 and perform an operation. Here, the control input may include current, torque, and the like.

<The whole process for collision detection>

Hereinafter, a method of detecting the collision by the robot manipulator 200 will be described with reference to the drawings.

3 is a flowchart of a method for detecting a collision of a robot manipulator 200 according to an embodiment of the present invention.

As illustrated in FIG. 3, in the present invention, the method for detecting the collision of the robot manipulator 200 initially defines a number of way points, so that the neural network operator 150 performs pre-processing through cycle normalization. <S30>

Thereafter, a collision signal is obtained from the collision measurement unit 150. <S31>

Next, the neural network learns based on the collision signal. <S32>

Finally, the trained neural network performs control by transmitting a signal that detects collision in real time. <S33>

Through this entire process, the robot manipulator 200 according to the present invention learns a collision, and then detects a collision in real time to perform control of the joint.

<cycle normalization>

Hereinafter, description of cycle normalization, which is a pre-processing operation, will be continued through the drawings.

4 is a diagram of a flow for detecting whether a collision has occurred in a method of detecting a collision of the robot manipulator 200 according to an embodiment of the present invention.

As shown in FIG. 4, in the present invention, a pre-processing process called cycle normalization is performed before transmitting a robot signal for controlling a joint by detecting an impact or a collision. Cycle normalization improves the generalization performance by preprocessing the neural network, so that the collision detection performance of the neural network can be maintained at a high level even if the robot's motion (robot speed and path) and the weight of the object held by the robot change.

Specifically, when a robot motion program is defined as passing through a number of waypoints designated by a user in sequence, it passes through the first stoppoint to the last stoppoint in sequence, and then back to the first stoppoint. Path movement is defined as one cycle.

For example, assuming that there are three waypoints, the operation of the robot manipulator 200 may be defined as a cycle that starts from waypoint 1 and passes through waypoints 2 and 3 to reach waypoint 1 again. have. Therefore, the robot manipulator 200 that performs repetitive tasks for automation means repeating cycles.

When the waypoint of the robot manipulator 200 is first programmed and executed, while performing the first cycle motion, the controller 140 stores the robot signals in all sections of the first cycle in a memory provided in the controller 140, In this case, a cycle in which the signal of the entire cycle is stored may be defined as a reference cycle. In addition, the cycle after the reference cycle can be defined as an execution cycle.

When the initial robot program is created, the cycle size is fixed to a certain size, and this size can be determined by the control cycle and the length of the motion. For example, when it takes 2 seconds for any robot program to pass all of the given waypoints, and the control cycle of the robot manipulator 200 is 4000 Hz (controls 4000 times per second), a total of 8,000 times for 2 seconds Since the robot signal is obtained and stored by performing the control operation, the cycle size of the corresponding motion is 8,000. Accordingly, 8,000 robot signals can be configured in a reference cycle and stored in a memory. Since the robot manipulator 200 performs repetitive tasks as defined in the program, the size of the reference cycle and the execution cycle will be the same unless the robot program is modified. However, due to the error in the control cycle operation of the real-time operating system, errors of +5 and -5 may occur in the size of the reference cycle and the execution cycle during actual driving.

, 관절 각속도를

, 그리고 로봇에 인가되는 제어토크를

이라고 가정한다.The robot signal will be described in more detail. For example, when applying to the six-degree-of-freedom robot manipulator 200, the joint angle obtained at every control cycle through the control unit 140 is determined.

, Angular joint velocity

, And the control torque applied to the robot

Is assumed.

로 정의할 수 있다. 로봇 신호는 자유도에 따라 가변적인 것뿐만 아니라 제어부(140)의 연산을 통해 획득되는 별도 신호 또한 로봇 신호로 추가할 수 있다. 예를 들어, 목표 관절 각도

와 관절각 오차

등도 로봇 신호에 포함되거나 제외될 수 있다.At this time, the robot signal includes all signals related to the joint angle and the joint angular velocity.

It can be defined as The robot signal is not only variable according to the degree of freedom, but also a separate signal obtained through the operation of the controller 140 can be added as a robot signal. For example, the target joint angle

And joint angle error

The back may also be included or excluded from the robot signal.

Here, when the time variable is t, the robot signal at the current time is expressed as x _t . The robot signal is used as the input of the neural network, and a sliding window technique can be applied so that the neural network can analyze the pattern of the robot signal. Assuming that the size of the sliding window is W, the robot signal to which the sliding window is applied can be expressed as the following equation.

This means that W data of the past is included as the current time reference. Since the control of the robot manipulator 200 is made of discrete time, the time variable t can be expressed as an index indicating the position in the cycle.

As in the example above, it is assumed that the sliding window of size 200 is applied in the execution cycle in which the motion of the robot manipulator 200 is performed through 8,000 controls for 2 seconds. At this time, if the current time is the time when the 2,000th control cycle starts based on the cycle start, the robot signal in the execution cycle can be expressed as follows.

This is to connect the past W robot signals including the present based on the current time, and for this, it is necessary to temporarily store more past data than W in the memory to obtain the sliding window signal every time.

When the sliding window is applied to the reference cycle, the robot signal of the entire cycle is stored, so by extracting the 1801th to 2000th signals from the stored robot signals, the sliding window robot signal for the reference cycle can be obtained. Can.

At this time, the cycle normalization is expressed by the difference between the reference cycle at the same timing and the robot signal of the execution cycle, which can be expressed by the following equation.

은 사이클 시작 인덱스를 0이라 하였을 때 t 번째 시각에서의 최종 사이클 정규화 로봇 신호이며, 이는 신경망의 입력으로 사용된다. At this time

Is the final cycle normalization robot signal at the t-th time when the cycle start index is 0, which is used as the input of the neural network.

In this way, the cycle normalization performs signal normalization in real time through a signal difference from the motion in the reference cycle in the execution cycle, and through this, the waypoint of the motion, the speed of the robot manipulator 200, the weight of the robot manipulator 200, etc. Even if this is changed, the normalized signal pattern can be kept constant, thereby improving the generalization performance of the neural network.

<Learning Neural Networks>

In addition, the process of the neural network learning based on the learning data will be described in detail through the drawings.

5 is a diagram schematically showing a process in which a neural network learns learning data in a method of detecting a collision of a robot manipulator 200 according to an embodiment of the present invention.

As shown in FIG. 5, in the present invention, the neural network may learn a collision signal through supervised learning based on learning data consisting of a pair of signals and labels. Supervised learning is a method of learning by repeatedly notifying the correct answer to a given signal to a neural network through data consisting of a pair of signals and labels.

In addition, in the present invention, the purpose of detecting a collision is to determine whether a collision has occurred in terms of an algorithm. This is a binary signal that can be represented as True or False. Therefore, the neural network can be trained to output a True signal when a collision occurs and a False signal when a collision does not occur.

In the present invention, the difference is that an additional sensor is not used in comparison with the prior art, but in the process of acquiring learning data, a collision measurement unit 150 is provided on one side of the joint to acquire a signal for collision through contact. Can. The collision measurement unit 150 may be equipped with, for example, a pressure sensor or a force sensitive resistor (FSR), and may measure collision as it is turned on and off through contact.

The collision measurement unit 150 is connected to the control unit 140, and when the control unit 140 performs a control operation of the robot manipulator 200, the collision measurement unit 150 signal can be obtained by synchronizing with the robot signal, and the robot After attaching to any part of the joint of the manipulator 200, it can be operated repeatedly according to a program.

While the robot manipulator 200 is operating, when a shock is applied to the collision measuring unit 150 attached to the joint using a hand or a tool, a collision signal applied to the joint through the collision measuring unit 150 is generated. Can be obtained.

In addition, a robot signal and a collision signal may be obtained by repeating a motion program of a joint, an attachment position of the collision measurement unit 150, and a timing in which a collision occurs in motion at random.

라고 하였을 때, 신경망을 학습시키기 위한 신호-라벨 데이터 쌍은

로 정의할 수 있다.At this time, the signal of the collision measurement unit 150 among the acquired signals may be surfaced with y and defined as a collision signal. The collision signal at the current time

Is a signal-label data pair for training a neural network.

It can be defined as

After sufficiently obtaining the learning data, the neural network is trained through a supervised learning method. The neural network or artificial neural network used at this time is in the form of a convolutional neural network. This is to train a neural network through a back propagation method commonly used in the prior art, and the loss function can use binary cross-entropy, which is mainly used to learn classification. .

<Real-time neural network inference>

In addition, collision detection can be performed in real time while the robot manipulator 200 is operating through real-time neural network inference. Assuming that the first user specifies several stops by programming a program that drives the robot manipulator 200, it will be assumed that the robot manipulator program repeatedly visits the stop.

In this case, the first cycle is a reference cycle, and while performing the motion of the reference cycle, the robot signal of the entire cycle section can be stored in the memory inside the controller 140 as described above. In this situation, a collision should not occur.

In the reference cycle, collision detection is performed using a conventional method, and when a collision is detected in the reference cycle in the conventional method, the next cycle is again stored as a reference cycle, and the robot signal is newly stored. If all robot signals of the cycle are normally stored without collision in the reference cycle, the execution cycle is performed next.

From the execution cycle, the control unit 140 stores the robot signal in the memory for a predetermined sliding window size. The robot signal to which the cycle normalization is applied becomes an input of a previously learned neural network, and the neural network performs inference calculation using this.

보다 높을 경우 충돌이 발생한 것으로 설정할 수 있다. 이 때 일반적인

은 0.5이지만, 실시예에 따라 0.6 또는 0.7이 적용될 수도 있다.As shown in Fig. 5, the result of inference calculation is a probability value between 0 and 1 because the last layer of the neural network is Sigmoid.

If it is higher, it can be set as a collision. Common at this time

Is 0.5, but 0.6 or 0.7 may be applied depending on the embodiment.

The existence of a collision is determined through inference calculation, and when a collision occurs, the controller 140 performs a safety stop function. Safety stop is a general robot reaction when a collision occurs, and various robot reactions can be set according to the method of implementation.

<Experimental results of the robot manipulator collision detection system>

Hereinafter, experimental results comparing the robot manipulator 200 collision detection system 100 according to the present invention and a collision detection system through a conventional momentum observer will be described.

FIG. 6 is a diagram showing the results of experiments comparing a system for detecting a collision of a robot manipulator 200 and a conventional momentum observer detecting a collision according to an embodiment of the present invention, and FIG. 7 is a diagram illustrated in FIG. 6 It is a diagram showing a result of experiment and comparison with a system for detecting a collision of a robot manipulator 200 according to an embodiment of the present invention by reducing a boundary value for collision detection of a momentum observer in an experiment.

As shown in Figs. 6 and 7, experiments were performed to compare the collision normalization algorithm using the cycle normalization and neural networks proposed in the present invention with a conventional method. Among the performance evaluations of the aforementioned robot collision detection, false false, false positive and collision detection accuracy items were compared with the conventional methods. The conventional method of performing collision detection without using a sensor is a method of estimating an external force through a momentum-based observer and then detecting a collision when the estimated external force exceeds a preset collision detection boundary value. . This is called Momentum Observer-based collision detection (MOD, Momentum-based Observer-based collision Detector). In general, it is a method commonly used by commercial robots and researchers performing robot research.

The lower the collision detection boundary value, the more sensitive collision detection can be, whereas the lower the boundary detection value, false positives may occur. For a fair experiment, the collision detection boundary value of the conventional method was set to a minimum value in a range in which false positives do not occur. At this time, the collision detection accuracy and sensitivity of the conventional method and the method proposed in the present invention were experimentally compared.

Figure 6 shows the results of the experiments as described above. In this experiment, the collision detection proposed in the present invention was applied to the six-degree-of-freedom robot manipulator 200, and a conventional momentum observer-based collision detection was applied to the same system. It can be seen that the proposed collision detection is faster and more sensitive than the conventional MOD even when the collision detection boundary value of the MOD is set to a minimum value that does not generate false positives.

At this time, if the boundary value of the MOD is set to be lower to allow the more conventional method to detect the collision sensitively, a false positive occurs as in FIG. 7, and in this case, the method proposed in the present invention does not generate a false positive. Can be confirmed. Through this, it can be confirmed that the neural network-based method proposed in the present invention has high sensitivity to collision detection and robustness against false negatives when compared with the conventional MOD.

The above-described preferred embodiments of the present invention are disclosed for the purpose of illustration, and those skilled in the art having ordinary knowledge of the present invention, various modifications, changes, and additions will be possible within the spirit and scope of the present invention. And additions should be considered to fall within the scope of the claims of the present invention.

100: robot manipulator collision detection system
110: joint driving unit
120: encoder unit
130: neural network operation unit
140: control unit
150: collision measurement unit
200: robot manipulator

Claims (12)

In the system for detecting the collision of the robot manipulator using an artificial neural network,
A joint driving unit provided at a plurality of joints of the robot manipulator to drive the plurality of joints, respectively;
An encoder unit provided on one side of the joint driving unit to measure an angle of the plurality of joints;
It includes; a neural network operation unit for learning a neural network provided based on a large amount of learning data about a collision signal and inferring to detect that the plurality of joints collide with the outside through a pre-processing operation.
The pre-processing operation performs generalization of the neural network through cycle normalization,
The cycle normalization defines a number of waypoints, and one cycle is defined by going through the path from the first waypoint to the last waypoint and then moving to the first waypoint again. And
The neural network operation unit learns a collision signal through supervised learning based on learning data consisting of a pair of signals and labels, outputs a true signal when a collision occurs, and is a false signal when a collision does not occur. Output a binary signal,
The last layer among the layers provided in the neural network is a probability value between 0 and 1,
The neural network operation unit performs a reasoning to detect a collision of the robot manipulator, characterized in that if the probability value is 0.5 or more, a collision occurs.
delete delete According to claim 1,
Further comprising a memory for storing information related to the cycle,
A system for detecting collision of a robot manipulator, characterized in that the entire signal of the defined cycle is defined as a reference cycle and stored in the memory.
According to claim 1,
A system for detecting collision of a robot manipulator, further comprising a control unit for applying and calculating a signal related to a control input to control the joint driving unit.
The method of claim 5,
The signal is a system for detecting a collision of a robot manipulator, characterized in that it comprises a joint angle, joint angular velocity and control torque obtained at every control cycle.
The method of claim 6,
The signal is used as an input of the neural network,
A system for detecting a collision of a robot manipulator, characterized by analyzing the pattern of the signal using a sliding technique.
delete delete According to claim 1,
Learning the collision signal is a system for detecting a collision of a robot manipulator, characterized in that the collision is measured through contact of a pressure sensor or a force sensitive resistor (FSR) provided on one side of the joint.
delete A method for detecting a collision of a robot manipulator using an artificial neural network,
Defining a plurality of way points to perform a preprocessing operation through cycle normalization by a neural network operation unit equipped with a neural network;
Obtaining a collision signal from the collision measurement unit;
Learning the neural network based on a collision signal;
Including the step of performing control by transmitting the signal that the learned neural network detects a collision in real time;
The pre-processing operation performs generalization of the neural network through cycle normalization,
The cycle normalization defines a number of waypoints, and one cycle is defined by going through the path from the first waypoint to the last waypoint and then moving to the first waypoint again. And
The neural network operation unit learns a collision signal through supervised learning based on learning data consisting of a pair of signals and labels, outputs a true signal when a collision occurs, and is a false signal when a collision does not occur. Output a binary signal,
The last layer among the layers provided in the neural network is a probability value between 0 and 1,
The neural network operation unit detects the collision of the robot manipulator, characterized in that for detecting the collision of the robot manipulator, characterized in that as the probability value is 0.5 or more to determine that a collision occurs.

Images