ES2759082T3 - Portable device to control a robot and its method - Google Patents

Abstract

A portable apparatus (12) for controlling a robot (10), including the portable apparatus (12): an orientation sensor (13), which is adapted to measure the orientation of said portable apparatus (12); a human-machine interface device, HMI (15), which is adapted to detect two-dimensional manual movement in relation to said HMI device (15); and a processing unit (17), which is adapted to receive a first signal representing said measured orientation of said portable apparatus (12) and a second signal representing said two-dimensional manual movement detected in relation to said HMI device (15) and controlling a part of said robot to move in a direction corresponding to a combination of said measured orientation of said portable apparatus and a direction of said two-dimensional manual movement detected in relation to said HMI device (15), the portable apparatus (12) being ) characterized in that: said processing unit (17) is further adapted to receive a third signal representing a speed of movement of said part of said robot (10) from a controller of said robot and to judge whether a scale factor between a speed of said two-dimensional manual movement detected in relation to said HMI device (15) and the movement speed of said part of said robot (10) is within an allowable range.

Description

DESCRIPTION

Portable device to control a robot and its method

Technical field

The invention relates to the field of a robot control apparatus and method, and more particularly to a portable robot control apparatus and method thereof.

Technical background

Typically, a robot is equipped with a teaching panel. The device is relatively large (with a touchscreen, operator buttons, etc.) and is connected to a wired robot controller.

European patent EP 2055 446 published on May 6, 2009 describes a portable robot control apparatus for controlling the movement of a robot or the end of a robot tool. The portable robot control apparatus comprises an inertial device with at least one acceleration sensor and / or at least one rotation sensor, where said inertial device measures its relative motion and the apparatus sends a signal to the robot controller representing the relative motion, so that the robot controller is enabled to control the robot such that said relative motion is repeated by the robot or by the end of the robot tool in real time.

Document "Design of 6-DOF Manipulator Intuitive Teaching System by Using Smart Phone Orientation - User Friendly and Intuitive Teaching Operation for 6-DOF Manipulator", Sanghun Pyo, Syed Hassan, Yasir Jan and Jungwon Yoon, 4th International Conference on Intelligent Systems, modeling and simulation, 2013, describes a smartphone that can carry out the user's intention for the industrial robot to move, and the information from the orientation sensor becomes the translation and orientation of the robot assuming that the orientation of the smartphone can be a conventional joystick equipped with a universal joint in the base part. The method can move the robot's end effector per direction as the XY plane.

EVANS III A WILLIAM ET AL describes: "Control solutions for robots using Android and iOS devices", UNMANNED SYSTEMS TECHNOLOGY XIV, SPIE, 100020TH ST. BELLINGHAM WA 98225-6705 USA, vol. 8387, no. May 1, 11, 2012 (2012-05-11), pages 1-10, XP060003615, DOI: 10.1117 / 12.923023

Rosemarie E Yagoda ET AL describes: "Using Mobile Devices for Robotic Controllers: Examples and Some Initial Concepts for Experimented", June 2011 (2011-06), XP055419186, retrieved from the Internet: URL: http: //www.dtic.mi1 / docs / citations / ADA549454

SEBASTIAN MUSZYNSKI ET AL describes "Adjustable autonomy for mobile teleoperation of personal service robots", RO-MAN, 2012 IEEE, IEEE, September 9, 2012 (2012-09-09), pages 933-940, XP032466959, DOI: 10.1109 / ROMAN.2012.6343870 ISBN: 978-1-4673-4604-7

US 2010/241273 A1 describes a device for retrieving data from radio frequency identification (RFID) tags.

KALMAN BOLLA ET AL: "A fast image processing based robot identification method for Surveyor SRV-1 robots", ADVANCED INTELLIGENT MECHATRONICS (AIM), 2011 IEEE / ASME INTERNATIONAL CONFERENCE ON, IEEE, July 3, 2011 (2011-07-03 ), pages 1003-1009, XP032053609, DOI: 10.1109 / AIM. 2011.6027147 ISBN: 978-1 4577-0838-1

US 2008/027591 A1 describes a system for controlling more than one remote vehicle.

WO 2013/094821 A1 and US 2014/371954 A1 describe a method and system for remote control and a remote controlled user interface. According to these conventional solutions, the orientation of the teaching remote / smartphone is assigned to the orientation of the center point of the robot tool, however, the linear movement of the robot by the teaching remote / smartphone is less intuitive. Furthermore, the user normally changes the teaching settings by setting various parameters on the smartphone / teaching remote, which diverts the attention of the robot operator to the one teaching and makes teaching less comfortable.

Brief summary of the invention

The problems in the state of the art are at least partially overcome by the present object.

According to an aspect of the invention, a portable apparatus for controlling a robot includes: an orientation sensor, which is adapted to measure the orientation of said portable apparatus; an HMI device, which is adapted to detect two-dimensional manual movement relative to said HMI device; and a processing unit, which is adapted to receive a first signal representing said measured orientation of said portable apparatus and a second signal representing said two-dimensional manual movement detected in relation to said device. HMI and controlling a part of said robot to move in one direction taking into account said measured orientation of said portable apparatus and said two-dimensional manual movement detected in relation to said HMI device.

According to another aspect of the invention, a method of manually controlling a robot with a portable apparatus includes: measuring the orientation of said portable apparatus; detecting two-dimensional manual movement in relation to an HMI device of said portable apparatus; controlling a part of said robot to move in one direction taking into account said measured orientation of said portable apparatus and said two-dimensional manual movement detected in relation to said HMI device of said robot; receiving a speed of movement of said part of said robot (10) from a controller of said robot (10) and judging whether a scale factor between the speed of said two-dimensional manual movement detected in relation to said HMI device (15) and the The movement of said part of said robot is in an admissible range.

By having the handheld and its method as explained herein, two-dimensional manual movement on the touchpad integrates with the orientation of the handheld and the robot maps an integration of these, making it possible to define a route in three-dimensional space to move / teach the movements of the robot in three dimensions. This becomes more intuitive than performing the linear motion of the robot by portable orientation.

Brief description of the drawings

The object of the invention will be explained in more detail in the following text with reference to preferred exemplary embodiments illustrated in the drawings, in which:

Figure 1 shows an arrangement of a robot manipulated by an operator using a portable apparatus 12;

Figure 2 illustrates a block diagram of the portable apparatus according to an embodiment of the present invention; and

Fig. 3 shows a simplified flow diagram for carrying out a method for manually controlling a robot according to an embodiment of the invention.

The reference symbols used in the drawings and their meanings are listed in summary form in the list of reference symbols. In principle, identical parts with the same reference symbols are provided in the figures.

Preferred embodiments of the invention

Figure 1 shows an arrangement of a robot 10 manipulated by an operator 11 using a portable apparatus 12. A robot 10 comprises a manipulator 16 and a control system 18 to control the movements of the manipulator 16. The control system 18 is located in an external computer 20, in this case. The control system 18 can also be located on computer means in the portable apparatus 12 and / or in the manipulator 16. The manipulator 16 is adapted to be programmed to execute a plurality of tasks. During manual movement and programming of robot 10, robot operator 11 communicates with control system 18 through handset 12. Operator 11 enters commands in control system 18, for example, to start and stop. a program, or to move manipulator 16 to a desired position. The control system 18 further comprises path planning means for calculating how the manipulator 16 must move in order to execute the programmed tasks.

Fig. 2 illustrates a block diagram of the portable apparatus according to an embodiment of the present invention. The portable device can be a smartphone, tablet, PDA, etc. The portable apparatus 12 is used to manually manipulate and program the robot 10 interacting with the control system 18. The portable apparatus 12 comprises an orientation sensor 13, an HMI device 15 (human-machine interface) and a processing unit 17. Orientation Sensor 21 is adapted to measure the orientation of portable apparatus 12, for example, it can be a three-way magnetometer or a combination of a three-way accelerometer and a three-way gyroscope. The HMI device 15 is adapted to detect two-dimensional manual movement relative to the HMI device 15; for example, the device h M i may comprise an input unit such as a touch panel that tracks the movement of one or more fingers of the operator 11 in two degrees of freedom. The processing unit 17, for example a processor or a programmable logic unit, is adapted to receive a first FS signal representing the measured orientation of the portable apparatus 12 and a second SS signal representing the two-dimensional manual movement detected in relation to said device HM115 and controlling a part of the robot 10 to move in one direction taking into account the measured orientation of the portable apparatus 12 and the two-dimensional manual movement detected in relation to the HMI device 15. A part of the robot 10, for example a tool mounted on the wrist of the robot 10 (conventionally known as TCP or tool center point) or a joint of the robot 10, can be controlled by moving one or two fingers on the HMI device 15 of the portable apparatus 12, for example a touchpad . The movement on the touch panel with the movement of the fingers is limited to two dimensions, which is detected by the HMI device 15. This HMI device 15 also sends the detected two-dimensional movement to the processing unit 17 of the portable apparatus 12; in addition, three additional dimensions are introduced by changing the orientation (i.e. the gesture) of the portable apparatus 12 which is measured by the orientation sensor 13 of the portable apparatus 12 and the measurement is also sent to the processing unit 17 of the portable apparatus 12. The processing unit 17 thus considers the two-dimensional movement of the fingers on the touchpad and the measurement of the three-dimensional orientation of portable apparatus 12 and transforms them into a three-dimensional path in the real world. For example, if the portable apparatus 12 will be held vertically and the operator 11 will move his finger upwards on the touch panel 15, the robot 10 will move the TCP or the joint upwards, and vice versa. If the operator 12 wants to make a forward movement, he must orient the portable apparatus 12 horizontally and move his finger forward on the touch screen 15, and vice versa. By having the portable apparatus to control the robot, the two-dimensional manual movement on the touch panel integrates with the orientation of the portable apparatus 12 and the robot maps an integration of these, making it possible to define a path in a three-dimensional space to move / teach robot movements in three dimensions. This becomes more intuitive than performing the linear motion of the robot by portable orientation.

Preferably, the processing unit 17 of the portable apparatus 12 is further adapted to control the part of the robot 10 (for example, the TCP of the robot joint) to move in a direction corresponding to a combination of the measured orientation of the portable apparatus 12 and a direction of the detected two-dimensional manual movement relative to the HMI device 15 and at a speed corresponding to the speed of the detected two-dimensional manual movement relative to the HMI device 15.

For example, the orientation sensor 13 of the portable apparatus 12 is further adapted to measure the orientation of a first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) at one, two or three degrees of freedom, which is defined in in relation to the portable apparatus 12 and which follows the movement of the portable apparatus 12. The robot 10 is operable in a second three-dimensional coordinate system (X ² , Y ² , Z ² ) attached to the robot 10. The HMI device 15 is further adapted for detecting two-dimensional manual movement relative to the HMI device 15 in the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ), for example, a touchpad arranged to detect operator finger input at two degrees of freedom; processing unit 17 is further adapted to determine a relative orientation between the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) and the second three-dimensional coordinate system (X ² , Y ² , Z ² ) based on measurements orientation of the orientation sensor 13 and the fixed orientation of the second three-dimensional coordinate system, calculate a transformation between the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) and the second three-dimensional coordinate system (X ² , Y ² , Z ² ) based on the relative orientation between these coordinate systems and transforming the detected two-dimensional manual movement relative to the HMI device 15 into the corresponding movements of the manipulator part in the second three-dimensional coordinate system (X ² , Y ² , Z ² ) based on the calculated transformation. The workspace of the manipulator 16 of the robot 10 is defined to move the manipulator 16 between different positions in the workspace in a controlled manner when the robot 10 is manually moved, for example to move the TCP of the tool held by the robot or the joint of the manipulator 16 (the robot part). These positions in the workspace of the robot 10 are defined by the use of a coordinate system, for example a Cartesian coordinate system, having an origin and the directions of the axes defined in relation to the robot 10 or the apparatus. portable 12 or the HMI 15 device of the portable apparatus 12. The manipulators are generally adapted to be operated with up to six degrees of freedom (DOF), which in this case means three translational degrees of freedom represented by the X, Y, Z axis and three rotational degrees of freedom represented by rotations about the X, Y, Z axis. In this case, the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) is defined in relation to the portable apparatus 12 so that follows the movements of the portable apparatus 12 (such as a smartphone or tablet), the second three-dimensional coordinate system (X ² , Y ² , Z ² ) is defined fixed to the manipulator 16 of the robot 10. The movements of dif Parts of the manipulator are then defined in the second coordinate system (X ² , Y ² , Z ² ), and manual movement relative to the HMI device is defined in the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ).

During a manual movement task, operator 11 can move manipulator 16 in different directions, and therefore, the operator generally holds the portable apparatus in various gestures and moves his finger on the touch panel of the portable apparatus. Operator 11 thus causes a reorientation of the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) relative to the second three-dimensional coordinate system (X ² , Y ² , Z ² ), because the first coordinate system ( X ¹ , Y ¹ , Z ¹ ) is defined in relation to the portable apparatus 12.

To determine this reorientation of the portable apparatus 12, the processing unit 17 is adapted to determine the relative orientation between the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) and the second three-dimensional coordinate system (X ² , Y ² , Z ² ), that is, how the first three-dimensional coordinate system (X ¹ , Y ¹ , Z ¹ ) has been rotated relative to the second three-dimensional coordinate system (X ² , Y ² , Z ² ). The processing unit 17 of the portable apparatus 12 is further adapted to repeatedly update the first three-dimensional coordinate system so that each axis of the first three-dimensional coordinate system corresponds to a corresponding X ² , Y ² , Z ² axis in the second coordinates (X ² , Y ² , Z ² ). This is done by calculating the transformation from the first coordinate system (X ¹ , Y ¹ , Z ¹ ) to the second coordinate system (X ² , Y ² , Z ² ) and applying this transformation to each movement of the portable device measured in the first coordinate system (X ¹ , Y ¹ , Z ¹ ). The transformation includes information about the rotation, but not the translation between the coordinate systems. Regarding the speed of the robot movement, for example, the operator's finger 11 moves with respect to the touch panel at a speed of 0.1 m / s, and the robot is controlled to move at 0.1 m / s multiplied by a predetermined scale ratio.

As shown in Figure 2, the portable apparatus 12 further includes an identification marker reader 19 which is adapted to receive a signal representing information about the part of the robot 10 from an external identification marker. The identification marker can be an RFID (radio frequency identification) tag, an NFC (near field communication) tag, or a QR code (quick response code) tag. Processing unit 17 is further adapted to select the robot part from among several robot parts 10 based on the information from the identification marker, for example to select the robot part to move / teach. Preferably, the identification marker that records the information on the part of the robot is connected to said part of the robot. For example, as shown in Figure 1, the identification marker for the first joint is attached to the first joint. This allows the operator to set the shift / teach target more intuitively. In particular, operator 11 touches the NFC tag on the robot's first link with handheld device 12. Thus, the operator can manually control the first joint of robot manipulator 16 using handset 12. As described above , the first joint follows the orientation change of the portable apparatus 12. When the operator 11 touches the second joint with the portable apparatus 12, the part of the robot that could move changes from the first joint to the second joint, then the second joint follows changing the orientation of the portable device 12.

As an alternative, it is possible to select different joints and displacement modes based on the information recoded by the identification marker: joint group, linear movement or reorientation. Although the handheld can be rotated in three dimensions (tilt, yaw, roll), only rotation along one of the directions is used to map to a robot axis, and rotation in other directions is ignored. By using identification techniques, the teaching settings can be changed without using any screen. The operator does not need to monitor the robot all the time. You don't need to select anything from the menu, but simply move the handheld device to a certain area relative to the robot, which is faster and doesn't require much operator attention.

Preferably, the identification marker reader 19 is adapted to receive a signal representing information about various robots from external identification marker labels, and the processing unit 17 is further adapted to select one of the robots as master and the others as slaves.

Preferably, the processing unit 17 is further adapted to receive a third signal representing the speed of movement of the robot part 10 from the control system 18 of the robot 10 and to judge whether a scale factor between the two-dimensional manual movement speed detected in relation to said HMI device 15 and that of the movement of the robot part 10 is within an allowable range. The processing unit 17 is further adapted to receive a fourth signal from the control system 18 of the robot 10 and judge whether the position is colliding with an external object, if there is a malfunctioning internal component, the robot is closer to be or is already out of reach. The HMI device is further adapted to send sound, vibrate, or change its background color to indicate various conditions of said robot 10 as indicated above.

Preferably, the HMI device 15 is further adapted to display robot information based on a robot signal received from robot 10 control system 18, such as actual position, actual speed, actual acceleration, actual torque, I / O, data internal status of the robot (for example, motor current), etc. By having the display function, the operator can judge whether there is a malfunction of an internal component. This implies that the processing unit 17 is further adapted to receive more than one signal representing information from the different robot.

Figure 3 shows a simplified flow diagram for carrying out a method for manually controlling a robot according to an embodiment of the invention. The method repeatedly performs the steps described in the next paragraph.

A measurement of the orientation of the portable apparatus, block 100. In practice, an initial position is defined with a known orientation relative to the second three-dimensional coordinate system. After self-guiding, the orientation sensor measures the reorientation relative to the starting position. The measurement of the orientation of the portable apparatus can be performed by measuring the orientation of a first three-dimensional coordinate system which is defined in relation to said portable apparatus and which follows the movement of said portable apparatus. Two-dimensional manual movement is detected relative to an HMI device of the portable apparatus, block 110. For example, two-dimensional manual movement relative to the HMI device of the portable apparatus is detected in the first three-dimensional coordinate system. A part of said robot is controlled to move in one direction taking into account the measured orientation of said portable apparatus and said two-dimensional manual movement detected in relation to said HMI device of said robot, block 120. The part of said robot is controlled to move at a speed corresponding to a speed of said detected two-dimensional manual movement relative to said HMI device of said portable apparatus, block 130.

Although the present invention has been described on the basis of some preferred embodiments, those skilled in the art should appreciate that those embodiments should in no way limit the scope of the present invention. Without departing from the scope of the present invention, any variations and modifications of the embodiments should be understood by those skilled in the art and, therefore, be within the scope of the present invention as defined in the appended claims.

Claims (14)

1. A portable apparatus (12) for controlling a robot (10), including the portable apparatus (12): an orientation sensor (13), which is adapted to measure the orientation of said portable apparatus (12); a human-machine interface device, HMI (15), which is adapted to detect two-dimensional manual movement in relation to said HMI device (15); and a processing unit (17), which is adapted to receive a first signal representing said measured orientation of said portable apparatus (12) and a second signal representing said two-dimensional manual movement detected in relation to said HMI device (15) and controlling a part of said robot to move in a direction corresponding to a combination of said measured orientation of said portable apparatus and a direction of said two-dimensional manual movement detected in relation to said HMI device (15), the portable apparatus (12) being ) characterized by: said processing unit (17) is further adapted to receive a third signal representing a speed of movement of said part of said robot (10) from a controller of said robot and to judge whether a scale factor between a speed of said manual movement two-dimensional detected in relation to said HMI device (15) and that of the movement of said part of said robot (10) is within an admissible range. 2. The portable apparatus (12) according to claim 1, wherein: said processing unit (17) is further adapted to control that said part of said robot (10) moves at a speed corresponding to the speed of said two-dimensional manual movement detected in relation to said HMI device (15). 3. The portable apparatus (12) according to claim 2, wherein: said orientation sensor (13) is further adapted to measure the orientation of a first three-dimensional coordinate system which is defined in relation to said portable apparatus (12) and which follows the movement of said portable apparatus (12); said robot (10) is operable in a second fixed three-dimensional coordinate system; said HMI device (15) is further adapted to detect said two-dimensional manual movement relative to said HMI device (15) in said first three-dimensional coordinate system; said processing unit (17) is further adapted to determine a relative orientation between the first and second three-dimensional coordinate systems, calculating a transformation between the first and second three-dimensional coordinate systems based on said relative orientation between these coordinate systems and transforming said two-dimensional manual movement detected in relation to said HMI device (15) in the corresponding movements of said part of said robot (10) in the second three-dimensional coordinate system based on said calculated transformation. 4. The portable apparatus (12) according to claim 1 or 2 or 3, wherein: said orientation sensor (13) is a three-way magnetometer or a combination of a three-way accelerometer and a three-way gyroscope. 5. The portable apparatus (12) according to claim 1 or 2 or 3, wherein: said HMI device (15) is a touch panel. 6. The portable apparatus according to claim 1 or 2 or 3, wherein: said part of said robot (10) is a tool attached to said tool; or said part of said robot (10) is a joint. 7. The portable apparatus (12) according to claim 1 or 2 or 3, further including: at least one identification marker reader (19), which is adapted to receive a signal representing information about said part of said robot (10) from an external identification marker; where: said processing unit (17) is further adapted to select said part of said robot (10) among several parts of said robot based on said information on said part of said robot, where: said identification marker is attached to said part of said robot; and / or said processing unit (17) is further adapted to establish a mode of movement based on said information about said part of said robot. 8. The portable apparatus (12) according to claim 1 or 2 or 3, further including: an identification marker reader (19), which is adapted to receive a signal representing information about various robots from an external identification marker label; where: said processing unit (17) is further adapted to select one of said robots (10) as master and the others as slaves. 9. The portable apparatus according to claim 7 or 8, wherein: Said identification marker is an RFID tag, an NFC tag or a QR code tag. 10. The portable apparatus (12) according to claim 1 or 2 or 3, wherein; said processing unit (17) is further adapted to receive a fourth signal representing the position of said part of said robot from the controller of said robot and to judge if said position is in collision with an external object. 11. The portable apparatus (12) according to claim 1 or 2 or 3, wherein: said HMI device (15) is further adapted to display robot information based on the robot signal received from the controller of said robot (10), where preferably: said robot information represents the malfunction of a part of said robot (10). 12. The portable apparatus (12) according to claim 1 or 2 or 3, wherein: said HMI device (15) is further adapted to send sound, vibrate or change its background color to indicate various conditions of said robot (10). 13. A method of manually controlling a robot (10) with a portable apparatus (12) according to any of the preceding claims, the method including: measuring the orientation of said portable apparatus (12) with an orientation sensor (13) included in said apparatus (12); detecting, with a human-machine interface, HMI (15), a two-dimensional manual movement in relation to the HMI device (15) of said portable apparatus (12); controlling a part of said robot (10) to move in one direction taking into account a combination of said measured orientation of said portable apparatus (12) and a direction of said detected two-dimensional manual movement in relation to said HMI device (15) of said robot the method being characterized in that it also includes: receiving a speed of movement of said part of said robot (10) from a controller of said robot (10) and judging whether a scale factor between the speed of said two-dimensional manual movement detected in relation to said HMI device (15) and the The movement of said part of said robot is in an admissible range. The method of manually controlling a robot according to claim 13, further including: controlling said portion of said robot to move at a speed corresponding to the speed of said detected two-dimensional manual movement in relation to said HMI device (15) of said portable apparatus (12).