JP6499272B2 - Teaching apparatus and control information generation method - Google Patents

Description

  The present invention relates to a teaching device for teaching an operation to a robot and a method for generating control information for controlling the robot.

  In recent years, automation of work using industrial robots is progressing in order to save labor at production sites. Some industrial robots, for example, robot arms, include a so-called parallel link mechanism that supports one end effector by a plurality of arm units arranged in parallel, or a plurality of arm units such as an articulated robot in one direction. Some have a serial link mechanism that is connected to the end effector to support the end effector.

  Conventionally, there is a teaching device that simulates a work performed by a robot arm by a person, acquires the motion of the simulated person by motion capture, and teaches the robot arm (for example, Patent Document 1). In the teaching device disclosed in Patent Literature 1, when an operator operates the measuring device (in the literature, “motion capture”) while wearing the hand, based on the three-dimensional coordinate data transmitted from the measuring device, The position of the measuring device is sampled at a predetermined sampling interval. The teaching device calculates the movement position, movement speed, and the like based on the sampled data, and controls the arm unit to move the calculated movement position at the movement speed.

JP 2011-200997 A

  By the way, in the production site, not only one robot arm but also a plurality of robot arms are operated simultaneously on the same object. On the other hand, in the teaching device described above, the arm of the operator who wears the measuring device in his / her hand is inserted into the space where the motion capture is performed, so when trying to teach the operation of multiple robot arms, A plurality of workers' arms will be inserted. For this reason, there is a possibility that the arms of a plurality of workers interfere with each other. Further, when the robot arm moves in a complicated direction, there is a possibility that it cannot be simulated by human hands.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a teaching device and a control information generation method capable of teaching cooperative work by a plurality of robots.

In order to solve the above problems, a teaching device according to claim 1 of the present application is provided in the first and second moving units, the first working unit provided in the first moving unit, and the second moving unit. A teaching device that generates control information for controlling the operation of the robot including the second working unit, the first jig having a first position marker unit indicating the position of the first moving unit, and the second moving unit And a second jig having a second position marker portion having a characteristic different from that of the first position marker portion, and a first jig and a second jig that move with the movement of each of the first and second jigs. Work information relating to work performed by each of the first and second work parts at the work position in a state where the detection part for detecting the position marker part and the first and second jigs are moved to the predetermined work position. And an input unit for inputting the first and second position markers. A detection data detected each over portion, and a processing unit for processing the work information input unit includes a processing unit, a determining process of determining whether it has received an input of the working information by the input unit, According to the determination result of the determination process , based on the detection data, the position information generation process for generating the position information of the three-dimensional coordinates of each of the first and second position marker units, and the first and the second based on the position information A movement information generation process for generating movement information related to the movement direction and movement speed of each of the second position marker parts, and each of the first and second movement parts is moved in cooperation according to the position information and movement information, And a control information generation process for generating a series of work control information for causing each of the first and second work units to work according to the work information.

  Further, in the teaching device according to claim 2, in the teaching device according to claim 1, the plurality of first position marker units and the plurality of second position marker units different in number from the plurality of first position marker units. And the processing unit sets the center of gravity of the plurality of first position marker units as the first center of gravity position, and sets the center of gravity of the plurality of second position marker units as the second center of gravity position. Second position-of-gravity position setting processing is executed, and position information of the three-dimensional coordinates of each of the first position of the center of gravity and the position of the second position of the center of gravity is generated as position information generation processing.

  In the teaching device according to claim 3, in the teaching device according to claim 1, the detection unit images the first and second position marker units, and outputs the captured imaging data to the processing unit as detection data. And an illuminating unit that irradiates each of the first and second position marker units with the first irradiation light and the second irradiation light having different wavelengths. The first position marker unit includes: The second position marker unit is configured to be capable of reflecting the irradiation light, and the second position marker unit is configured to be capable of reflecting the second irradiation light.

  Further, in the teaching device according to claim 4, in the teaching device according to any one of claims 1 to 3, the processing unit is configured such that the first and second position marker units are based on the position information. A distance determination process for determining a distance between the two and a notification process for notifying that when the distance is equal to or less than a predetermined distance are executed.

Further, in the teaching device according to claim 5, in the teaching device according to any one of claims 1 to 4, the first jig includes a first movable part and a first movable part that drives the first movable part. A second movable part; a second drive part that drives the second movable part; and a second drive part that includes a drive part and a first movable part marker part that indicates a position of the first movable part. 2, the second movable part marker part indicating the position of the movable part, and the processing part receives the input of the work information by the judgment process, the inputted work information is the first drive part And the drive information determination process for determining whether or not the information is information indicating that at least one of the second drive units is to be driven, and the drive information determination process, the work information is obtained from the first drive unit and the second drive unit. in response to determining that the information indicating that drive at least one position As the information generation process, the position information of each of the first and second movable part marker units that moves as each of the first and second movable units operates based on the driving of the first and second drive units. Is generated.

  Further, in the teaching device according to claim 6, in the teaching device according to any one of claims 1 to 5, the processing unit performs the first operation after moving the first moving unit in the control information generating process. As control information to be executed at at least one of the two timings before performing the work by the working unit and after moving the second moving unit and before performing the work by the second working unit, Control information for correcting the positions of the first and second working units is added.

  Further, in the teaching device according to claim 7, in the teaching device according to any one of claims 1 to 6, the processing unit includes a plurality of feature points from the generated position information in the position information generation process. And a correction process for approximating position information between feature points is performed.

  Further, in the teaching device according to claim 8, in the teaching device according to any one of claims 1 to 6, the processing unit performs first and second based on the detection data as position information generation processing. Each position of the position marker unit is sampled, and the position of the sampling point is generated as position information.

  Further, in the teaching device according to claim 9, in the teaching device according to claim 8, each of the first and second position marker units generated by the position information generation process as the movement information generation process. The moving direction is detected based on the positional relationship between the adjacent sampling points, and the moving speed is detected based on the distance between the adjacent sampling points and the sampling period.

  Further, in the teaching device according to claim 10, in the teaching device according to claim 9, when the curvature of the curve connecting the sampling points exceeds a predetermined curvature, the processing unit moves as the movement information generation process. The position information is corrected when the speed exceeds a predetermined speed and at least one of the cases where the acceleration at the moving speed exceeds the predetermined acceleration.

  The teaching device according to claim 11 is the teaching device according to any one of claims 1 to 10, further comprising a reference marker portion provided at a position serving as a reference for the operation of the robot. The reference marker unit is detected, and the processing unit determines, in the position information generation process, the relative position of the first position marker unit to the reference marker unit and the relative position of the second position marker unit to the reference marker unit. Each is generated as position information.

  Further, in the teaching device according to claim 12, in the teaching device according to any one of claims 1 to 11, the robot includes a serial link mechanism as a driving mechanism for each of the first and second moving units. It is characterized by that.

In order to solve the above problem, the control information generation method according to claim 13 of the present application includes a first and second moving unit, a first working unit provided in the first moving unit, and a second A control information generation method for controlling the operation of a robot including a second working unit provided in a moving unit, the first jig having a first position marker unit indicating a position of the first moving unit, 2 indicates the position of the moving part, the second jig having the second position marker part having a characteristic different from that of the first position marker part, and the first jig that moves with the movement of each of the first and second jigs And a detection unit for detecting the second position marker unit, and the first and second work units in the work position performed at the work position in a state where the first and second jigs are moved to the predetermined work position. an input unit for inputting a work information relating, to teaching apparatus comprising, input A judgment step of judging whether it has received an input of the working information by parts, according to a determination result of the determination step, the detection unit based on detection data detected each of the first and second position marker portion, the A position information generation step for generating position information of the three-dimensional coordinates of each of the first and second position marker portions, and movements related to the moving direction and moving speed of each of the first and second position marker portions based on the position information; A movement information generation step for generating information, and each of the first and second moving units is moved in cooperation according to the position information and the movement information, and each of the first and second working units is moved according to the work information. And a control information generation step process for generating control information for a series of operations to be performed.

  In the teaching device according to claim 1 of the present application, for example, control information for controlling a robot including two sets of combinations of a moving unit and a working unit is generated. In the work of motion capture, the first jig provided with the first position marker portion and the second jig provided with the second position marker portion are used as the detected objects, so that the person wearing the measurement device There is no need to use hands. Thereby, for example, when simulating the operation of a plurality of moving units and working units, the arms of the plurality of workers are not inserted into the space where the motion capture is performed, and the arms of the workers interfere. Problems can be prevented. Also, even robot movements that cannot be expressed by human movement, for example, movements that move the joints of human arms in the opposite direction, can be taught by moving the jig.

  Further, the second position marker unit is configured with characteristics different from those of the first position marker unit. As a detection method of the first and second position marker portions, for example, from imaging data obtained by irradiating each of the first and second position marker portions with light of different wavelengths and imaging the reflected light of each position marker portion, An optical motion capture that calculates position information or the like can be used. In this case, for example, the second position marker portion is made of a material that reflects light having a wavelength different from that of the first position marker portion. Note that the detection method of the position marker portion is not limited to this, and other methods, for example, image processing for detecting the shape of the position marker portion with respect to the imaging data may be executed, and the position may be detected from the execution result. Alternatively, using a magnetic sensor as a position marker unit, receiving position data with identification information from a plurality of moving magnetic sensors, and calculating position information of each magnetic sensor from the received position data Motion capture can be used.

In addition, the teaching device moves the first and second jigs to a predetermined work position. The work performed by the work unit at the work position, for example, sandwiches the work, grabs the work, and emits laser. An input unit for inputting work information related to work such as imaging, picking up a workpiece, and the like. For this reason, the user operates the input unit at an appropriate timing in a state where the first and second jigs are moved to the work position, and inputs the work content desired to be performed by the robot as work information. It becomes possible to set the fine work of each of the first and second working units for the teaching device.

Further, the processing unit generates position information according to a determination result of whether or not input of work information has been received by the input unit. Thereby, it can be controlled whether position information should be generated according to the input of work information. The processing unit moves the first and second moving units in cooperation according to the position information and the movement information generated from the detection data, and causes the first and second working units to perform work according to the work information. Generate control information for a series of tasks. That is, the processing unit can generate control information by linking movement information and work information and the like in the control information generation process. Accordingly, in the teaching device, for example, correction processing for adjusting the work positions of the first and second working units is performed in accordance with the movement of the two moving units so as to be performed at an appropriate timing and work content. Can be incorporated into a series of work processes.

  In the teaching device according to claim 2 of the present application, for example, a plurality of first position marker portions are attached to the first jig and grouped, and the first center of gravity positions of the plurality of first position marker portions are Motion capture as a detection target. Similarly, a plurality of second position marker portions (a number different from the first position marker portion) are attached to the second jig and grouped, and the second center of gravity positions of the plurality of second position marker portions are detected. Motion capture as a target. In this case, the number of marker portions and the difference in the position of the center of gravity are different characteristics. As a result, position information matching is performed in group units of the grouped position marker portions, so that the position information can be prevented from being confused and the operation of each position marker portion can be detected with high accuracy. For example, position information matching is facilitated by changing the attachment position and arrangement of the position marker portions of each group and setting unique shapes and center-of-gravity positions. Also, within each group, even if some position information of multiple position marker parts cannot be detected, the position of the position marker part that cannot be detected based on the position information of other position marker parts is interpolated to obtain the center of gravity position By doing so, it is possible to prevent the loss of position information.

  In the teaching device according to claim 3 of the present application, the first and second position marker portions are configured with different reflection characteristics. A detection part images the reflected light which the 1st position marker part reflected the 1st irradiation light, and the reflected light which the 2nd position marker part reflected the 2nd irradiation light with an image sensor. The processing unit can detect the operation of each position marker unit with high accuracy by detecting the position of each of the first and second position marker units while identifying them from the imaging data of the image sensor by luminance or the like. Become.

  The first and second jigs simulate an actual robot arm or the like, but it is conceivable to reduce the outer shape of the actual object in consideration of user operability. In this case, if the robot is moved based on the control information created by operating the two jigs, the first and second moving units may collide and interfere with each other. Therefore, in the teaching device according to claim 4 of the present application, for example, a distance corresponding to the external dimensions of the actual robot from the centers of the first and second position marker portions is set as a distance that may cause a collision. When the processing unit determines that the distance between the first and second position marker units is equal to or less than the distance that may cause a collision, the processing unit notifies the fact. As a result, the user can recognize that the first and second jigs have actually approached a distance that may cause a collision during the work, and can take appropriate measures such as starting again.

In the teaching device according to claim 5 of the present application, each of the first and second jigs includes the first and second movable parts, the first and second movable part marker parts, the first and second drives. With each of the parts. When the processing unit receives the input of the work information , the processing unit determines whether or not the input work information is information indicating that at least one of the first and second driving units is to be driven, and indicates that the driving is performed. In response to the determination that there is a motion, the first and second movable part marker parts of the first and second movable parts that are movable according to the driving of the first and second drive parts are motion-captured. Thereby, it can be judged whether it is the timing which carries out motion capture of the 1st and 2nd marker for movable parts based on work information. The user grasps or separates the workpiece by moving each of the first and second movable parts by the first and second drive parts at a predetermined work position, as compared with the case of simulating with a human finger or the like. Can be reproduced more faithfully.

  In the teaching device according to claim 6 of the present application, for example, when a person moves the first jig by hand, the accuracy of movement of the first position marker portion depends on the accuracy of manipulating the jig. To do. For this reason, when higher work accuracy is required in the teaching device, high accuracy is required by performing position correction processing before performing work at the work position and correcting the final work position. It is possible to cope with work to be performed.

  Further, in the teaching device according to claim 7 of the present application, the processing unit performs extraction of feature points and correction processing for approximating position information between the feature points. For the feature point extraction, for example, a point whose movement direction is changed by a predetermined angle or more is extracted as a feature point. Alternatively, the feature points are extracted by, for example, extracting points at fixed intervals from the plurality of pieces of position information as feature points.

  The processing unit sets the extracted feature points as, for example, a starting point or an ending point of the robot operation, and corrects position information between the feature points so that the robot can move from the starting point toward the ending point. For example, as a correction process, when a point whose position is significantly shifted due to external noise is between feature points, the position information related to the point is discarded as unnecessary data. Alternatively, as correction processing, position information between feature points is approximated by a straight line or a curve connecting the feature points. As a result, the robot can be operated more smoothly based on the generated control information, and wasteful operations can be omitted to improve work efficiency. In addition, it is possible to correct a positional information shift caused by shaking of a person's hand when the person operates the jig.

  In the teaching device according to claim 8 of the present application, the processing unit generates position information from the detection data at a predetermined sampling period. For this reason, in the said teaching apparatus, it becomes possible to adjust the precision which a process part detects each position of a 1st and 2nd position marker part by changing the time of a sampling period.

  In the teaching device according to claim 9 of the present application, the processing unit detects the moving direction and the moving speed based on the position information of the adjacent sampling points. For this reason, in the teaching apparatus, it is possible to adjust the accuracy with which the processing unit detects the moving direction and the moving speed of each of the first and second position marker units by changing the time of the sampling cycle.

  In addition, for example, when the first and second jigs are operated by a person, the moving direction, moving speed, and acceleration of the first and second jigs exceed the moving ability of the robot to which the control information is applied. In this case, even if the control information is generated, it is difficult for the robot to perform a desired operation. Therefore, in the teaching device according to claim 10 of the present application, for example, when the curvature of the curve connecting the sampling points exceeds a predetermined curvature (for example, a curvature that the robot can move), the curvature becomes movable. The position information (coordinates etc.) is corrected as follows. Thereby, the control information generated by the motion capture can be easily used as data for actually controlling the robot.

  In addition, when performing motion capture, there is a risk that the positions of the tracking area for tracking the operations of the first and second jigs and the work area where the robot actually works are shifted. Therefore, in the teaching device according to claim 11 of the present application, position information is generated with reference to the reference marker portion. As a result, when the generated control information is used, the robot can be accurately controlled by matching the position of the reference marker portion with the reference in the actual work area. The reference in the work area here is, for example, an XY robot, a position used as a reference when determining the positions in the X direction and the Y direction.

  Also, for example, when it is desired to check the movement of the robot by importing the control information into a simulation tool (such as 3D CAD), the control information can be captured and used by matching the position of the reference marker portion with the reference position of the tool. It becomes easy. Further, for example, by generating the control information of the two robots separately using the same reference marker part and simulating the position of the reference marker part of the two control information according to the reference position of the tool, 2 It is possible to confirm whether the operations of the two robots do not interfere with each other.

  In the teaching device according to claim 12 of the present application, it is effective to apply the present invention to a robot having a serial link mechanism widely used at production sites, for example, a multi-joint robot arm.

  Furthermore, the invention according to the present application is not limited to the teaching device, and can also be implemented as an invention of a method for generating control information by the teaching device.

It is the figure which showed roughly the structure of the principal part of the teaching apparatus of this embodiment. It is a perspective view of the frame part to which the some camera was attached. It is a schematic diagram which shows the structure of the industrial robot which is the object controlled using control information. It is a schematic diagram which shows the state which implements a motion capture using the teaching apparatus of this embodiment. It is a flowchart which shows the production | generation process of the control information by CPU. It is a figure which shows the sampling point which sampled the position of each marker part with which two jigs are provided. It is a schematic diagram which shows the jig | tool of another example. It is a schematic diagram which shows the jig | tool of another example. It is a schematic diagram which shows the jig | tool of another example. It is a schematic diagram which shows the jig | tool of another example.

  Hereinafter, an embodiment of the teaching device of the present invention will be described with reference to the drawings. FIG. 1 schematically shows a configuration of a main part of the teaching device 10 of the present embodiment. The teaching device 10 includes a plurality of cameras 13 (only one is shown in FIG. 1), two jigs 15 (only one is shown in FIG. 1), and a control information generating device 17 by optical motion capture. Prepare. The teaching device 10 images each movement of the jig 15 with a plurality of cameras 13, and uses the control information generating device 17 to obtain control information D5 for controlling the robot arms 101 and 103 shown in FIG. Generate.

<About camera 13>
As shown in FIG. 2, each of the plurality of cameras 13 is attached to a frame portion 23 in which a plurality of (12 in this embodiment) pipes 21 are assembled in a rectangular parallelepiped shape. Each of the plurality of pipes 21 is formed with the same length, and any three of the twelve pipes 21 are connected to each other by the connecting member 25 at the corner portion of the rectangular parallelepiped frame portion 23. It is connected. Each connecting member 25 inserts and holds the end portions 21A of the three pipes 21, and fixes the three pipes 21 so as to be orthogonal to each other. Hereinafter, as shown in FIG. 2, the direction orthogonal to the placement surface of the table 19 on which the frame portion 23 is arranged is the vertical direction, the direction orthogonal to the vertical direction and going forward and backward in FIG. The direction orthogonal to the front-rear direction is referred to as the left-right direction and will be described.

  In the teaching device 10 of the present embodiment, for example, a total of six cameras 13 are attached to the frame portion 23. Hereinafter, when it is necessary to distinguish between the plurality of cameras 13, an explanation will be given by adding an alphabet after the symbol of the camera 13 as shown in FIG. 2. When there is no need to distinguish between them, the six cameras will be collectively referred to as “camera 13”. Of the six cameras 13, four cameras 13 </ b> A, 13 </ b> B, 13 </ b> C, and 13 </ b> D are attached to each of the four pipes 21 on the upper side of the frame portion 23 by a fixing member 27. Each of the four cameras 13 </ b> A to 13 </ b> D is attached at a position close to each of the upper four connecting members 25. The fixing member 27 fixes each of the cameras 13 </ b> A to 13 </ b> D so that the imaging direction faces the central portion of the frame portion 23.

  Of the six cameras 13, the remaining two cameras 13E and 13F are connected to each of a pair of pipes 21 that face each other diagonally among the four pipes 21 provided along the vertical direction. The fixing member 27 is attached. The cameras 13 </ b> E and 13 </ b> F are attached to a lower end portion of the pipe 21 on the side of the base 19, and are fixed by a fixing member 27 so that the imaging direction faces the central portion of the frame portion 23. These six cameras 13 move the tracking region R1, that is, the jig 15 and the marker unit 43, in the cube-shaped region surrounded by the frame unit 23 in order to photograph the marker unit 43 of the jig 15 to be described later. Is set as an area for tracking. Note that the six cameras 13 are set so that the imaging ranges overlap each other in order to track the marker unit 43, for example, and the tracking region R1 can be imaged three-dimensionally. Moreover, the shape of the frame part 23 shown in FIG. 2, the number of cameras 13, the attachment position of the camera 13, etc. are examples, and can be changed suitably.

  As shown in FIGS. 1 and 2, each of the cameras 13 includes an image sensor 31 and illumination devices 33 and 34. The image sensor 31 is, for example, a CCD image sensor or a CMOS image sensor. The illumination devices 33 and 34 are, for example, LED illuminations, and irradiate light having different wavelengths. This is two types of light corresponding to the marker portions 43A and 43B provided in each of two jigs 15A and 15B (see FIG. 4) described later. The camera 13 receives reflected light reflected from the marker units 43 </ b> A and 43 </ b> B by the imaging device 31 while being irradiated from the illumination devices 33 and 34. The camera 13 outputs the captured data as imaging data D1 to the teaching device 10 via the video cable 35. The camera 13 has an optical filter corresponding to the wavelength of light emitted from the illumination devices 33 and 34 attached to the light entrance of the image sensor 31 so that the reflected light of the marker units 43A and 43B can be easily detected. Also good.

<About the jig 15>
Next, the jig 15 to be detected will be described. A jig 15 shown in FIG. 1 is a detection target that simulates the robot arms 101 and 103 of the industrial robot 100 shown in FIG. 3, and includes a main body portion 41, a marker portion 43, an end effector 45, and a gripping portion 47. And. FIG. 3 schematically shows the configuration of the industrial robot 100. The robot arm 101 is an articulated robot having a serial link mechanism that connects two arm portions 105 (an example of a moving unit) in one direction and supports a hand unit 109 (an example of a working unit) that is an end effector at a tip portion. It is. Similarly, the robot arm 103 connects the two arm portions 107 in one direction and supports the hand portion 111 at the tip portion. For example, the industrial robot 100 drives the robot arms 101 and 103 to attach the workpieces W1 and W2 sandwiched between the hand units 109 and 111 to the substrate B. The workpieces W1 and W2 are, for example, electronic parts, screws, and the like.

  The body part 41 of the jig 15 shown in FIG. 1 corresponds to the arm parts 105 and 107 of the robot arms 101 and 103. Here, as will be described later, the teaching device 10 of the present embodiment generates control information D5 for operating the two robot arms 101 and 103 in a coordinated manner. Therefore, two types of jigs 15A and 15B (see FIG. 4) are used, and motion capture is performed assuming that the jigs 15A and 15B are different robot arms 101 and 103, respectively. In the following description, when it is necessary to distinguish between the two types of jigs 15 or the parts (marker parts 43 and the like) provided in the jigs 15, as shown in FIG. It will be described later with an alphabet. When there is no need to distinguish between them, the two jigs will be collectively referred to as “jigs 15”.

  The marker portion 43 is fixed to the outer peripheral portion of the main body portion 41. The marker unit 43 forms a sphere and reflects light emitted from the illumination devices 33 and 34 of each camera 13. For example, the marker portion 43A provided in the jig 15A shown in FIG. 4 is made of a material having a reflection characteristic that reflects light of a specific wavelength irradiated by the illumination device 33. Further, the marker portion 43B provided on the other jig 15B is made of a material having a reflection characteristic that reflects light of a specific wavelength irradiated by the illumination device 34.

  The end effector 45 has a shape simulating the hand portions 109 and 111 sandwiching the workpieces W1 and W2 of the robot arms 101 and 103 (see FIG. 3), and a pair of end portions bent in a direction approaching each other. It is comprised by the rod-shaped member. A pair of end effector 45 is provided in the position which pinches the marker part 43, and the front-end | tip part is comprised so that opening and closing is possible. A movable portion marker portion 46 for tracking the movement of the end effector 45 is provided at the tip portion of each end effector 45. For example, like the marker portions 43A and 43B, the movable portion marker portions 46A and 46B are configured with mutually different reflection characteristics, and reflect the light emitted from each of the illumination devices 33 and 34 of the camera 13. Thereby, the control information generation device 17 can also acquire the position information D2 of the end effector 45 as in the case of the jig 15. The camera 13 may include a dedicated illumination device that irradiates the end effector 45 with light separately from the illumination devices 33 and 34 used for the jig 15.

  In addition, the main body 41 incorporates an actuator 49 for opening and closing the end effector 45. The main body 41 is attached with a tip end portion of a rod-shaped gripping portion 47 at a portion opposite to the marker portion 43 and the end effector 45. For example, the gripping portion 47 is a state in which the user hands the proximal end portion of the gripping portion 47 protruding out of the frame portion 23 in a state where the jig 15 is placed in the tracking region R1 (see FIG. 2) of the frame portion 23. It has a length that can be held by Thereby, the user can operate the jig 15 without putting a part of the body into the tracking region R1.

  A drive switch 51 for driving or stopping the actuator 49 is provided at the base end portion of the grip portion 47 opposite to the main body portion 41. The drive switch 51 is connected to the actuator 49 by a connection line 53 disposed in the grip portion 47 and the main body portion 41. For example, when executing the motion capture, the user holds the proximal end portion of the grip portion 47 and the jig 15 provided at the distal end portion from the start position within the tracking region R1 of the frame portion 23. , For example, the robot arms 101 and 103 are moved to the work position where the hands W1 and W2 are clamped by the hand units 109 and 111, respectively. After moving, the user turns on the drive switch 51 to close the distal end portion of the end effector 45. Alternatively, the user turns off the drive switch 51 to open the tip end portion of the end effector 45. The teaching device 10 tracks the operation of the end effector 45 and generates control information D5 described later. The control information D5 is control information D5 for controlling the hand units 109 and 111 of the industrial robot 100 shown in FIG.

<About the control information generation device 17>
Next, the configuration of the control information generation device 17 will be described. The control information generation device 17 is, for example, a personal computer mainly composed of a CPU (Central Processing Unit) 61, and includes a conversion unit 63, a storage unit 65, an input unit 67, a display unit 69, and the like. The control information generation device 17 inputs the imaging data D1 output from the camera 13 to the conversion unit 63 via the video cable 35 (see FIG. 2). The conversion unit 63 arranges the imaging data D1 captured by the plurality of cameras 13 in time series, adds identification information of the camera 13, time information, and the like, and outputs them to the CPU 61. The CPU 61 stores the imaging data D1 input from the conversion unit 63 in the storage unit 65. The storage unit 65 includes a memory, a hard disk, and the like, and stores a control program D7, design information D6 and the like in addition to the imaging data D1. The control program D7 is a program executed on the CPU 61. The design information D6 is information related to the industrial robot 100 shown in FIG. 3, and is information such as the outer dimensions of the arm portions 105 and 107, the maximum moving speed of the arm portions 105 and 107, and the like.

  The CPU 61 implements various processing modules of the position information generation unit 71, the movement information generation unit 73, and the control information generation unit 75 by reading and executing the control program D7 stored in the storage unit 65. In the present embodiment, the position information generation unit 71 and the like are configured as software realized by the CPU 61 executing the control program D7, but may be configured as dedicated hardware.

  The input unit 67 is an input device such as a keyboard and a mouse that receives input from the user. As described above, the teaching device 10 of the present embodiment moves the jig 15 to the working position, and then opens and closes the end effector 45 by operating the drive switch 51, and the hand portions 109 and 111 (see FIG. 3). ) Can be taught to the teaching device 10 to generate the work information D3. As another method, the user can instruct the teaching device 10 to generate other work information D3 by operating the input unit 67 after moving the jig 15 to the work position. It has become. For example, the user operates the input unit 67 to suck the workpieces W1 and W2, to irradiate a part of the workpieces W1 and W2 with a laser, and to apply an adhesive to the workpieces W1 and W2. Etc. can be entered. Thereby, the teaching device 10 can generate a series of work control information D5 in which the position information D2 of the jig 15 and the work information D3 are linked. Further, the display unit 69 displays various information such as the progress in the process of generating the control information D5 and the result information after generation.

  Next, an example of generation processing of the control information D5 in the control information generation device 17 will be described. First, work contents for executing motion capture will be described. In the following description, as an example, the two robot arms 101 and 103 (see FIG. 3) perform motion capture of the work of mounting the workpieces W1 and W2 on the substrate B while operating in cooperation with each other. FIG. 4 schematically shows a state in which motion capture is performed.

  As shown in FIG. 4, for example, supply devices 81 and 82 for supplying the workpieces W <b> 1 and W <b> 2 (see FIG. 3) are arranged in the tracking region R <b> 1 of the frame portion 23. The supply devices 81 and 82 are, for example, a tape feeder type supply device for feeding taped electronic components (workpieces) one by one to a supply position, or a plurality of trays in which electronic components are arranged at predetermined intervals. A tray-type supply device. A supply position marker portion 84 is provided at the supply position of the workpiece W2 of the supply device 81. A supply position marker portion 85 is provided at the supply position of the workpiece W1 of the supply device 82. The supply device 81 and the supply device 82 are arranged so that the supply positions (supply position marker portions 84 and 85) face each other in the front-rear direction.

  A substrate 86 is disposed between the supply devices 81 and 82 in the front-rear direction. The substrate 86 is formed in a rectangular shape, and is disposed horizontally such that the plane is along the front-rear direction and the left-right direction. On the four corners of the substrate 86, mounting position marker portions 88 are provided. In the following description, the mounting position marker portion 88 on which the jig 15A performs the mounting operation is referred to as a mounting position marker portion 88A in order to distinguish it from the other mounting position marker portions 88. Further, the mounting position marker portion 88 on which the jig 15B performs the mounting operation is referred to as a mounting position marker portion 88B in order to distinguish it from other mounting position marker portions 88. The supply devices 81 and 82 and the substrate 86 described above may be actual devices or substrates, or may be members that simulate shapes. Further, three reference marker portions 91 are provided adjacent to each other at the center of the substrate 86. The reference marker unit 91 is a position serving as a reference for the operation of the robot arms 101 and 103 (see FIG. 3).

  For example, the jig 15A teaches the work of picking up the workpiece W1 from the supply position of the supply device 82 (see FIG. 4) by the hand unit 109 of the robot arm 101 shown in FIG. In this case, as indicated by a solid line arrow 93 in FIG. 4, the user operates the jig 15A by holding the grip portion 47A, and moves the jig 15A from the start position shown in FIG. 4 to the supply position marker portion 85. Let At the position of the supply position marker unit 85, the user turns on the drive switch 51A to close the end effector 45A. Before operating the drive switch 51A, the user operates the input unit 67 of the control information generation device 17 and inputs work information D3 indicating that the end effector 45A is operated. Next, the user moves the jig 15A from the supply position marker portion 85 to the position of the mounting position marker portion 88A. Then, at the position of the mounting position marker portion 88A, the user turns off the drive switch 51A to open the end effector 45A. Also in this case, the user operates the input unit 67 before operating the drive switch 51A, and inputs work information D3 indicating that the end effector 45A is operated.

  Further, the jig 15B teaches the work of picking up the workpiece W2 from the supply position of the supply device 81 (see FIG. 4) by the hand unit 111 of the robot arm 103 and mounting it on the substrate B. This operation is performed simultaneously with the operation of the robot arm 101 described above. In this case, the user moves the jig 15B from the start position shown in FIG. 4 to the supply position marker section 84 as shown by the broken line arrow 95 in FIG. 4 to operate the jig 15B while holding the grip portion 47B. Let After the user inputs the work information D3 by the input unit 67 at the position of the supply position marker unit 84, the user turns on the drive switch 51B to close the end effector 45B. Next, the user moves the jig 15B from the supply position marker portion 84 to the position of the mounting position marker portion 88B. Then, after the user inputs the work information D3 with the input unit 67 at the position of the mounting position marker unit 88B, the user turns off the drive switch 51B to open the end effector 45B. Note that the user who operates the jig 15B may be a different user from the user who operates the jig 15A.

  Next, a process in which the CPU 61 generates the control information D5 during the above-described motion capture will be described with reference to the flowchart shown in FIG. Note that the processing contents and processing order of the flowchart shown in FIG. 5 are examples, and can be changed as appropriate. The control information generation device 17 generates the control information D5 by tracking the operation of the jigs 15A and 15B. More specifically, after the CPU 61 executes the control program D7 and starts processing, the user operates the input unit 67 at step 11 shown in FIG. 5 (hereinafter simply referred to as “S”). It is determined whether or not information D3 has been input. This is because when the user operates the input unit 67 and inputs the work information D3, a process corresponding to the content of the work information D3 is executed. When the work information D3 is input (S11: YES), the CPU 61 determines whether or not the input work information D3 is information indicating that the end effector 45 (drive switch 51) is operated (S12). Further, when the work information D3 is not input (S11: NO), the CPU 61 starts the processes after S15.

  In S12, when it is not the work information D3 indicating that the end effector 45 is operated (S12: NO), the CPU 61 performs control information corresponding to the type of the input work information D3, for example, work for sucking the workpieces W1 and W2. The corresponding subroutine program is added to the work information D3 and stored in the storage unit 65 (S13). In the operations indicated by arrows 93 and 95 shown in FIG. 4, the operation information D3 other than the end effector 45 has not been input. However, for example, when the jig 15 reaches a predetermined operation position, the user moves the input unit 67. By operating, it is possible to input operation information such as an adsorption operation, a laser irradiation operation, and an adhesive application operation. As a result, various operations can be added to a series of operation steps of the generated control information D5.

  If the input work information D3 is information indicating that the end effector 45 is to be operated in S12 (S12: YES), the CPU 61 generates position information D2 and the like shown in S15 and subsequent steps, and the end effector 45A, Control information D5 corresponding to the position, inclination, movement direction, etc. of each of the movable portion marker portions 46A and 46B of 45B, that is, control information D5 for causing the hand portions 109 and 111 (see FIG. 3) to perform a desired operation is generated. To do. In the following description, a process of generating the control information D5 based mainly on the imaging data D1 obtained by imaging the marker portion 43 of the jig 15 will be described. Since the process of generating the control information D5 based on the imaging data D1 obtained by imaging the movable part marker unit 46 of the actuator 49 is the same as that of the marker unit 43, it is omitted as appropriate.

  Next, the CPU 61 executes a process of taking the imaging data D1 from the conversion unit 63 and storing it in the storage unit 65 (S15). Note that the CPU 61 of this embodiment processes the imaging data D1 of the camera 13 in real time so as to notify the user of errors detected during motion capture. However, not limited to such real-time processing, for example, the CPU 61 may store all the imaging data D1 once in the storage unit 65 and process all the imaging data D1 collectively later.

  Next, the position information generating unit 71 is based on the identification information and time information of the camera 13 added to the imaging data D1 stored in the storage unit 65, and the marker units 43A and 43B attached to the jigs 15A and 15B. The position of the three-dimensional coordinates for each photographing time is calculated (S17). The position information generation unit 71 stores the calculated position information D2 in the storage unit 65. For example, the position information generation unit 71 performs labeling on the binarized imaging data D1, performs processing using an algorithm such as epipolar matching, and coordinates positions of the marker units 43A and 43B in the three-dimensional space. Is calculated. Further, the position information generation unit 71 calculates the position of the coordinates relative to the reference marker unit 91. For example, the position information generation unit 71 calculates the coordinate positions of the marker units 43 </ b> A and 43 </ b> B on the basis of the barycentric positions of the three reference marker units 91.

  As described above, each of the marker portions 43A and 43B has a structure having different reflection characteristics according to the wavelength of light emitted from the illumination devices 33 and 34. For this reason, the position information generation unit 71 identifies the reflected light from the marker units 43A and 43B, for example, based on the difference in luminance, and calculates the coordinate position for each of the marker units 43A and 43B with respect to the imaging data D1. . The processing method by which the position information generation unit 71 calculates the coordinate position (position information D2) is not particularly limited. For example, the position information D2 may be calculated by the principle of triangulation. If the position information generating unit 71 is the work information D3 indicating that the end effector 45 (drive switch 51) is operated in S12 (S12: YES), the position information generating unit 71 is for the movable unit as in the case of the marker unit 43. The position information D2 is generated for the marker unit 46.

  Further, the position information generation unit 71 performs a process of displaying the position information D2 on the display unit 69 (S19). For example, as illustrated in FIG. 6, the position information generation unit 71 acquires sampling points SP1 and SP2 obtained by sampling the respective positions of the marker units 43A and 43B at predetermined time intervals as the position information D2. The position information generation unit 71 performs a process of displaying all the acquired sampling points SP1 and SP2 (position information D2) on the display unit 69 in real time. Thereby, the user can check the acquired position information D2 as appropriate by checking the display on the display unit 69 and determine whether or not it is appropriate. Note that the position information generation unit 71 may display only the feature points such as the start point, the end point, and the point where the moving direction changes by a predetermined angle or more without displaying all the sampling points SP1 and SP2. Further, a black circle sampling point SP1 shown in FIG. 6 corresponds to the marker portion 43A of the jig 15A. A black square sampling point SP2 corresponds to the marker portion 43B of the jig 15B.

  Next, the position information generation unit 71 determines whether or not the distance between the marker units 43A and 43B is normal based on the sampling points SP1 and SP2 of the position information D2 (S21). The two jigs 15A and 15B simulate the robot arms 101 and 103 shown in FIG. 3, but it is conceivable to reduce the outer shape of the actual object in consideration of operability by the user's hand. In this case, when the robot arms 101 and 103 are moved based on the control information D5 created by operating the jigs 15A and 15B, the arm portions 105 and 107 may collide and interfere with each other.

  Therefore, based on the design information D6 stored in the storage unit 65, the position information generation unit 71, for example, sets a distance corresponding to the dimension of the outer shape of the arm unit 105 from the center of the marker unit 43A to a distance that may cause a collision. Set as. Then, the position information generation unit 71 calculates the distance between the sampling points SP1 and SP2, and when it is determined that the calculated distance is equal to or less than the distance that may cause a collision (S21: YES), an error is displayed on the display unit 69. Is executed (S23). Accordingly, the user can recognize that the jigs 15A and 15B have approached to a distance where the arm portions 105 and 107 and the like may collide, and can take appropriate measures such as restarting motion capture. . The position information generation unit 71 may calculate and determine the distance between the sampling points SP1 and SP2 at the same time, for example. Alternatively, the position information generation unit 71 may calculate and determine the distance from one sampling point SP1 with respect to the sampling point SP2 within a predetermined time with reference to the time of the sampling point SP1.

  After executing S23, the position information generation unit 71 temporarily stops the process until there is a response from the user to the error display, for example. Alternatively, when the predetermined time has elapsed, the position information generation unit 71 delivers necessary data and shifts the subject of processing to the movement information generation unit 73 (S25). If the position information generation unit 71 determines in S21 that the calculated distance is greater than the distance that may cause a collision (S21: NO), the position information generation unit 71 shifts the subject of processing to the movement information generation unit 73 (S25). .

  Next, the movement information generation unit 73 generates movement information D4 related to the movement of the marker units 43A and 43B based on the position information D2 stored in the storage unit 65 (S25). The movement information generation unit 73 calculates physical quantities such as movement distance, movement direction, speed, acceleration, and angle from the position information D2 as movement information D4 of the marker units 43A and 43B.

<Feature point extraction>
For example, the movement information generation unit 73 extracts feature points from a plurality of sampling points SP1 shown in FIG. Specifically, the movement information generation unit 73 extracts, for example, a sampling point SP1A corresponding to the movement start position of the jig 15A as a feature point as a feature point serving as a starting point. In addition, the movement information generation unit 73 greatly changes the movement direction at a point close to the supply position marker unit 85 at a point where the movement direction is changed by a predetermined angle or more from among the plurality of sampling points SP1. Sampling point SP1B is extracted as a feature point. This feature point determination may be made automatically by the movement information generation unit 73 based on the movement speed and movement direction, or may be instructed by the user operating the input unit 67.

  Then, the movement information generation unit 73 calculates the inclination from the coordinate position of the extracted feature points (sampling points SP1A, SP1B), and detects the movement direction of the marker unit 43A (the jig 15A). Or the movement information generation part 73 divides the distance of the extracted feature point by the time between feature points, for example, and detects the moving speed of the marker part 43A.

  Further, the movement information generation unit 73 may perform a correction process that approximates the position information D2 of the sampling point SP1 between the feature points. For example, the movement information generation unit 73 sets the extracted feature points (sampling points SP1A, SP1B) as the movement start point and end point, and moves the robot arm from the start point (sampling point SP1A) toward the end point (sampling point SP1B). The position information D2 of the sampling point SP1 between the feature points is corrected so that the arm part 105 of 101 (see FIG. 3) can move. For example, when the sampling point SP1 whose position is greatly shifted due to external noise is between the feature points, the movement information generation unit 73 discards the position information D2 related to the sampling point SP1 as unnecessary data. Alternatively, the movement information generation unit 73 approximates the position information D2 between feature points with a straight line connecting the feature points.

  The movement information generation unit 73 may approximate the feature points with a curve. For example, among the plurality of sampling points SP2 shown in FIG. 6, the movement information generation unit 73 extracts sampling points SP2A and SP2B whose movement directions are changed by a predetermined angle or more as feature points. The movement between the two feature points (sampling points SP2A, SP2B) is curving while gradually changing the movement direction. In this case, for example, the movement information generation unit 73 approximates the position information D2 of the sampling point SP2 between the feature points (sampling points SP2A and SP2B) with a curve using the sampling point SP2A as a starting point and the sampling point SP2B as an end point. .

<Calculation by sampling>
In addition, the movement information generation unit 73 is not limited to the method using the feature points described above, and may detect the movement direction or the like using another method. For example, the movement information generation unit 73 calculates the inclination from the coordinate position of the adjacent sampling point SP1 among the plurality of sampling points SP1 sampled by the position information generation unit 71, and the movement direction of the marker unit 43A (the jig 15A) May be detected. Alternatively, the movement information generation unit 73 may detect the movement speed of the marker unit 43A by multiplying, for example, the distance between adjacent sampling points SP1 and the sampling period of the position information generation unit 71.

  Next, when at least one of the curvature of the curve connecting the sampling points SP1 and SP2 and the detected moving speed exceeds the moving ability of the industrial robot 100 shown in FIG. The information D2 is corrected, and the generation of the movement information D4 is executed again (S27). For example, as shown in FIG. 6, at the sampling point SP2C, which is one of the plurality of sampling points SP2, the marker unit 43B is greatly curved and moved. When such a curved movement exceeds the movement capability of the actual robot arm 103, it is difficult to control the robot arm 103 even if the control information D5 is generated.

  Therefore, the movement information generation unit 73 uses, for example, the curvature connecting the sampling points SP2 (for example, between the sampling points SP2A and SP2B) based on the design information D6 stored in the storage unit 65 as the movement capability of the robot arm 103. If the curvature is larger than the curvature set accordingly, the position information D2 is corrected so as to have a movable curvature. Then, the movement information generation unit 73 generates movement information D4 again based on the corrected position information D2 (sampling point SP2).

  Further, for example, as shown in FIG. 6, the distance between the sampling points SP1C and SP1D which are two of the plurality of sampling points SP1 is greatly separated. As the distance between the sampling points SP1C and SP1D increases, the moving speed of the calculation result also increases. If this moving speed exceeds the moving ability of the actual robot arm 103 or if it is desired to exceed the safe moving speed, the robot arm 103 is controlled even if the control information D5 is generated as in the above-described curvature. It becomes difficult.

  Therefore, as in the case of curvature, the movement information generation unit 73 determines that the movement speed calculated from the distance between the sampling points SP1C and SP1D based on the design information D6 stored in the storage unit 65 is the robot arm, for example. If it is larger than the maximum moving speed 103, the position information D2 is corrected so as to be a movable speed. Then, the movement information generation unit 73 generates the movement information D4 again based on the corrected position information D2 (sampling points SP1C, SP1D). When the movement information generation unit 73 corrects the sampling points SP1 and SP2 (position information D2) according to the curvature and the movement speed, the distance between the corrected sampling points SP1 and SP2 may cause the above-described collision. Correct so that it is more than a certain distance.

  When the correction processing (S27) by the movement information generation unit 73 is completed, the CPU 61 inquires of the conversion unit 63 whether there is imaging data D1 that has not been captured (S29). When there is imaging data D1 that has not been captured (S29: NO), the CPU 61 executes the processing from S11 again.

  When there is no imaging data D1 captured from the conversion unit 63 (S29: YES), the CPU 61 instructs the control information generation unit 75 to generate the control information D5. The control information generation unit 75 moves the robot arms 101 and 103 based on the position information D2, the movement information D4, and the work information D3 stored in the storage unit 65, and operates the hand units 109 and 111 at the work position. Control information D5 of a series of work to be generated is generated (S31). The control information generation unit 75 allows the user to operate the input unit 67 after the unprocessed work information D3, for example, the jigs 15A and 15B reach the positions of the mounting position marker units 88A and 88B and finish the imaging. If there is work information D3 input in this way, it is preferable to process the work information D3 together.

  And the control information generation part 75 produces | generates the control information D5 which operates the arm parts 105 and 107 (refer FIG. 3) with the position and moving speed according to the positional information D2 and the movement information D4 of the marker part 43, for example. Further, the control information generation unit 75 detects the position and orientation of the end effector 45 (hand units 109 and 111) based on the position information D2 generated from the imaging data D1 of the movable part marker unit 46. The control information generation unit 75 adds control for rotating and opening / closing the hand units 109 and 111 to the control information D5 for moving the arm units 105 and 107. As a result, the control information generation unit 75 moves the arm units 105 and 107 to a predetermined work position (such as the supply position marker unit 84 in FIG. 4), and then holds the workpieces W1 and W2 by the hand units 109 and 111. It becomes possible to generate control information D5 of a series of work to be performed.

  Further, in S31, the control information generation unit 75 executes a process of adding the correction work to the control information D5. Specifically, for example, the control information generation unit 75 moves the arm units 105 and 107 to the supply positions of the supply devices 81 and 82 (positions of the supply position marker units 84 and 85), Before performing the clamping work of the workpieces W1 and W2 by 111, information for correcting an error between the supply position of the supply device 82 and the positions of the hand units 109 and 111 is added to the control information D5. For example, when a camera for imaging the substrate B or the like is mounted on the hand unit 109 of the robot arm 101, the supply position of the supply device 82 is imaged using the camera, and the hand unit 109 is based on the imaging data. A process of correcting the relative position error between the supply position and the supply position is added to the control information D5. In this way, the teaching device 10 can generate the control information D5 by executing motion capture.

In the above embodiment, the camera 13 is an example of a detection unit. The jig 15A is an example of a first jig. The jig 15B is an example of a second jig. The marker unit 43A is an example of a first position marker unit. The marker unit 43B is an example of a second position marker unit. The end effector 45A is an example of a first movable part. The end effector 45B is an example of a second movable part. The actuator 49 is an example of first and second drive units. The CPU 61 is an example of a processing unit. The robot arms 101 and 103 are examples of robots. The arm unit 105 is an example of a first moving unit. The arm unit 107 is an example of a second moving unit. The hand unit 109 is an example of a first working unit. The hand unit 109 is an example of a second working unit. The imaging data D1 is an example of detection data. Sampling points SP1A, SP1B, SP2A, SP2B are examples of feature points. The process of S11 is an example of a determination process. The process of S12 is an example of a drive information determination process. The process of S17 is an example of a position information generation process. The process of S21 is an example of a distance determination process. The process of S23 is an example of a notification process. The process of S25 is an example of a movement information generation process. The process of S31 is an example of a control information generation process.

As mentioned above, according to this embodiment mentioned above, there exist the following effects.
<Effect 1> The teaching device 10 generates control information D5 for controlling the pair of robot arms 101 and 103 including the arm units 105 and 107 and the hand units 109 and 111 shown in FIG. Here, in the conventional motion capture, a human hand or the like wearing a measuring device is used as the detection target, and thus there is a possibility that a human arm or the like enters the tracking region and contacts it. On the other hand, in the teaching apparatus 10, the jig 15A provided with the marker portion 43A and the jig 15B provided with the marker portion 43B are used as the detection target. As a result, when simulating the operation of the two robot arms 101 and 103, a plurality of user's arms are not inserted into the tracking region R1 where motion capture is performed, and the user's arms interfere with each other. It becomes possible to prevent.

  The teaching device 10 further includes an input unit 67 for inputting work information D3 such as work for sandwiching the works W1 and W2 performed by the hand units 109 and 111 at the work position. When the user moves the jigs 15A and 15B provided with the marker portions 43A and 43B to perform motion capture, the user operates the input portion 67 at an appropriate timing to indicate the work contents to be made to the industrial robot 100 as work information. By inputting as D3, it becomes possible to set detailed work contents of the hand units 109 and 111 for the teaching device 10. Accordingly, the teaching device 10 can generate a series of work control information D5 in which the position information D2 of the jigs 15A and 15B and the work information D3 are linked.

<Effect 2> The marker portion 43A provided on the jig 15A is made of a material having a reflection characteristic that reflects light of a specific wavelength irradiated by the illumination device 33. Further, the marker portion 43B provided on the other jig 15B is made of a material having a reflection characteristic that reflects light of a specific wavelength irradiated by the illumination device 34. The plurality of cameras 13 captures the reflected light reflected by the marker portions 43 </ b> A and 43 </ b> B by the image sensor 31. In such a configuration, the CPU 61 can detect the position of each of the marker portions 43A and 43B from the image data D1 of the image sensor 31 while identifying the brightness by a luminance or the like.
<Effect 3> The position information generation unit 71 calculates the distance between the sampling points SP1 and SP2, and when it is determined that the calculated distance is equal to or less than the distance that may cause a collision (S21: YES), the display unit 69 An error display is executed (S23). Accordingly, the user can recognize that the jigs 15A and 15B have approached to a distance where the arm portions 105 and 107 and the like may collide, and can take appropriate measures such as restarting motion capture. .
<Effect 4> In the jigs 15A and 15B, when the drive switch 51 is operated, the actuator 49 is driven. In accordance with the drive of the actuator 49, the end effectors 45A and 45B to which the movable portion marker portions 46A and 46B are attached are driven. The control information generation device 17 tracks the operations of the movable portion marker portions 46A and 46B that are movable in accordance with the drive of the actuator 49 during motion capture. As a result, the user can move the end effectors 45A and 45B by the actuator 49 at a predetermined work position, thereby performing more operations such as gripping the workpieces W1 and W2 than when simulating with human fingers or the like. It becomes possible to reproduce faithfully.

<Effect 5> The control information generation unit 75 moves the arm units 105 and 107 to the supply positions of the supply devices 81 and 82 (positions of the supply position marker portions 84 and 85) in S31, Before performing the clamping work of the workpieces W1 and W2 by 111, information for correcting an error between the supply position of the supply device 82 and the positions of the hand units 109 and 111 is added to the control information D5. When the user moves the jig 15 by hand, the accuracy of the movement of the marker unit 43 depends on the accuracy with which the user operates the jig 15. On the other hand, the control information generation unit 75 adds control information D5 to the control information D5 before performing work at the work position, so that control information that can be used for work that requires high accuracy. D5 can be generated.

<Effect 6> In S25, the movement information generation unit 73 extracts feature points from the plurality of sampling points SP1 shown in FIG. 6, and from the coordinate positions of the extracted feature points (for example, sampling points SP1A and SP1B), The inclination between the feature points is calculated and the movement direction of the marker portion 43A (the jig 15A) is detected. Further, the movement information generation unit 73 performs a correction process for approximating the position information D2 of the sampling point SP1 between the feature points. As a result, the robot arms 101 and 103 can be operated more smoothly based on the generated control information D5, and wasteful operations can be omitted to improve work efficiency.

<Effect 7> Further, as a processing method different from the feature point, the movement information generation unit 73 has an inclination from the coordinate position of the adjacent sampling point SP1 among the plurality of sampling points SP1 sampled by the position information generation unit 71. It is possible to calculate and detect the moving direction of the marker portion 43A (the jig 15A). Alternatively, the movement information generation unit 73 may detect the movement speed of the marker unit 43A by multiplying, for example, the distance between adjacent sampling points SP1 and the sampling period of the position information generation unit 71. In such a configuration, it is possible to adjust the accuracy of detecting the position, moving direction, and moving speed of the marker unit 43 by changing the time of the sampling period.

<Effect 8> When the curvature of the curve connecting the sampling points SP1 and SP2 exceeds the movement capability of the industrial robot 100 in S27, the movement information generation unit 73 corrects the position information D2 and moves information D4. Run generation again. As a result, the generated control information D5 is optimized for the movement capability of the industrial robot 100 that actually controls, and the control information D5 can be easily used as data for controlling the industrial robot 100.

<Effect 9> In S17, the position information generation unit 71 calculates the relative coordinate positions of the marker units 43A and 43B with reference to the barycentric positions of the three reference marker units 91. Thus, when the generated control information D5 is used, the industrial robot 100 is configured by matching the position of the center of gravity of the reference marker unit 91 with the reference in the actual work area, for example, the center position of the substrate B shown in FIG. Can be accurately controlled.

In addition, this invention is not limited to the said embodiment, It is possible to implement in the various aspect which gave various change and improvement based on the knowledge of those skilled in the art.
For example, in the above embodiment, the plurality of jigs 15A and 15B are used. However, the present invention is not limited to this, and one or three or more jigs 15 may be used. Further, a plurality of pieces of control information D5 acquired by operating one jig 15 a plurality of times may be synthesized later.

  Moreover, in the said embodiment, the structure of the jig | tool 15, the position of the marker part 46 for movable parts, etc. are examples, and can be changed suitably. FIG. 7 shows another example of the jig 120. In the following description, the same components as those in the above embodiment are given the same reference numerals, and the description thereof is omitted as appropriate. The pair of end effectors 123 and 124 of the jig 120 has a curved shape that opens outward, and a marker portion 43 is provided between them. Each of the end effectors 123 and 124 is provided with movable portion markers 126 and 127 having different shapes. The movable part markers 126 and 127 have a long rectangular shape on one side, and are arranged in the extending direction of the end effectors 123 and 124. The movable part marker 126 is longer than the movable part marker 127. In such a configuration, it is possible to easily detect the vertical direction and inclination of the jig 120 by providing a difference in the shape as well as the reflection characteristics of the movable part markers 126 and 127.

  In addition, the main body portion 129 of the jig 120 is provided with three main body portion marker portions 131. The three main body marker portions 131 are provided, for example, at positions corresponding to the vertices of one right triangle so that the distance from each other is different. In such a configuration, for example, each position of the three main body marker portions 131 can be detected, and the inclination of the jig 120 can be detected by an algorithm using the principle of triangulation or the like.

  Further, as shown in FIG. 8, a jig 140 that does not include the end effector 45 may be used. The jig 140 shown in FIG. 8 has a T shape, and marker portions 142, 143, and 144 are provided in portions extending in three orthogonal directions. The marker portions 142 to 144 have a long rectangular shape on one side, and have different lengths. Even when the jig 140 having such a configuration is used, it is possible to teach the control information generating device 17 the positions and moving directions of the arm portions 105 and 107.

  In the above embodiment, the marker portion 43A of the jig 15A has a reflection characteristic that reflects light of a specific wavelength irradiated by the illumination device 33, and the illumination device 34 irradiates the marker portion 43B of the other jig 15B. However, the identification method is not limited to this. However, the identification method is not limited to this, but is configured by reflecting the reflection characteristics of the two marker portions 43A and 43B. For example, as shown in FIG. 9, three marker portions 151 are provided on one jig 150 and four marker portions 161 are provided on the other jig 160 as shown in FIG. You may identify. In the following description, the same components as those of the jig 120 in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted as appropriate.

  As shown in FIG. 9, each of the three marker portions 151 is provided at the distal end portion of the end effector 123, the distal end portion of the end effector 124, and the central portion of the main body portion 129, as indicated by a one-dot chain line in FIG. 9. Are provided at the positions of the vertices of a substantially equilateral triangle. Also, as shown in FIG. 10, the four marker portions 161 are provided at the tip portion of the end effector 123, the tip portion of the end effector 124, and the end portions of the main body portion 129 (the left and right end portions in FIG. 10). As shown by the alternate long and short dash line in FIG. 10, it is provided at a position that forms a trapezoid. For this reason, the gravity center position 153 of the marker portion 151 and the gravity center position 163 of the marker portion 161 are different from each other.

  For example, as an initial setting before executing the motion capture, the control information generation device 17 images the jig 150 inserted in the tracking region R1, groups the three marker portions 151, and The barycentric position 153 is set as a detection target (an example of a first barycentric position setting process). Similarly, the control information generation device 17 images the jig 160, groups the four marker portions 161, and sets the centroid position 163 of the marker portion 161 as a detection target (an example of second centroid position setting processing). .

  Accordingly, by matching the position information D2 (coordinate position) extracted from the imaging data D1 in a group unit of the marker parts 151 and 161 that are grouped, the marker parts 151 and 161 (center of gravity positions 153 and 153) 163) is prevented from being confused in the position and the moving direction, and can be detected with high accuracy. Further, since the gravity center positions 153 and 163 of the marker portions 151 and 161 are different from each other, matching between the extracted coordinate positions and the marker portions 151 and 161 is facilitated. Also, for example, even if one of the three marker portions 151 cannot be detected, the position of the marker portion 151 that cannot be detected based on the position information of the other marker portion 151 is interpolated to obtain the center of gravity position 153. It is possible to prevent the position from being lost.

  Further, the identification method of the two marker portions 43A and 43B is not limited to the reflection characteristics and grouping described above, and the two marker portions 43A and 43B are identified by different colors, sizes, shapes and the like. May be. Alternatively, the marker portions 43A and 43B may be identified by giving different characteristics depending on the combination thereof. Or marker part 43A, 43B may be comprised by LED of the different luminescent color, etc., and marker part 43A, 43B itself may be light-emitted.

  In the above embodiment, the CPU 61 corrects the position information D2 based on the curvature and the moving speed in S27. However, the CPU 61 corrects the position information D2 based on the maximum acceleration of the industrial robot 100. Also good.

  Moreover, in the said embodiment, although the input part 67 was operated and the work information D3 was input in real time, it is not restricted to this. For example, a specific marker part may be registered in advance, and the control information generation device 17 may store the timing at which the specific marker part is detected in the control information D5 as timing information for adding the work information D3. . After generating the control information D5, the user may search for information on the timing at which the work information D3 stored in the control information D5 should be inserted, and add the necessary work information D3. In the conventional configuration, it is necessary to search the point of the main program where the necessary subroutine is added while looking at the contents of the main program. On the other hand, according to the method, since a point to be added is set in advance, it is easy to add a subroutine. In this case, the specific marker portion is an example of the input portion in the present application.

In the above-described embodiment, the position information generation unit 71 may execute a process of correcting the generated position information D2 (coordinate position) in order to correct a shake caused by a user's manual work.
Further, the control information D5 extracts only feature points (start point, passing point, arrival point) without using all the position information D2 generated by the position information generation unit 71, and connects the feature points. The control information D5 that enables the movement may be generated.

Moreover, although the example applied to the robot arms 101 and 103 as a robot in this application was demonstrated, it is not limited to this. For example, the robot in the present application may be a robot including a working unit that performs operations such as electronic component suction, laser beam irradiation, and screw tightening. Further, the robot is not limited to a robot having a serial link mechanism, and may be a robot that operates orthogonal to the XY axis direction or a robot that has a parallel link mechanism.
In the above embodiment, the motion capture using the optical method has been described. However, the motion capture in the present application may be a method with another method, for example, a magnetic method for detecting the operation of the magnetic sensor. For example, a magnetic sensor that transmits position data may be attached to the jig 15, and a receiving device that receives position data may be attached instead of the camera 13. In this case, the magnetic sensor corresponds to a position marker portion indicating the position of the moving portion in the present application. The receiving device corresponds to the detection unit.

  DESCRIPTION OF SYMBOLS 10 Teaching device, 13 Camera (detection part), 15A Jig (1st jig), 15B Jig (2nd jig), 43A Marker part (1st position marker part) 43B Marker part (2nd position marker part) ), 45A end effector (first movable part), 45B end effector (second movable part), 46A movable part marker part (first movable part marker part), 46A movable part marker part (for first movable part) Marker part), 49 Actuator (first and second drive part), 61 CPU (processing part), 67 Input part, 91 Reference marker part, 101, 103 Robot arm (robot), 105 Arm part (first moving part) 107 arm part (second moving part), 109 hand part (first working part), 111 hand part (second working part), D1 imaging data (detection data), D2 position information, D3 work information, D5 control information, SP1, SP2 sampling points, SP1A, SP1B, SP2A, SP2B sampling points (feature points).

Claims (13)

Generate control information for controlling the operation of the robot including the first and second moving units, the first working unit provided in the first moving unit, and the second working unit provided in the second moving unit. A teaching device,
A first jig having a first position marker portion indicating a position of the first moving portion;
A second jig indicating a position of the second moving part and having a second position marker part having a characteristic different from that of the first position marker part;
A detection unit for detecting the first and second position marker units moving with the movement of each of the first and second jigs;
While moving the first and second jig at a predetermined working position, an input unit each of the first and second working section to input an operation information related to operations performed in the working position,
A detection unit that detects each of the first and second position marker units and a processing unit that processes the work information of the input unit;
The processor is
A determination process for determining whether or not the input of the work information is accepted by the input unit;
A position information generating process for generating position information of the three-dimensional coordinates of each of the first and second position marker units based on the detection data in accordance with a determination result of the determination process;
Based on the position information, a movement information generation process for generating movement information related to the movement direction and movement speed of each of the first and second position marker units;
A series of causing each of the first and second moving units to move in cooperation according to the position information and the movement information, and causing each of the first and second working units to perform work according to the work information. And a control information generating process for generating the control information of the work. A plurality of the first position marker portions;
A plurality of the second position marker portions different in number from the plurality of first position marker portions,
The processor is
A first centroid position setting process for setting centroids of the plurality of first position marker portions as first centroid positions;
Performing a second center-of-gravity position setting process for setting the center of gravity of the plurality of second position marker portions as a second center-of-gravity position;
The teaching apparatus according to claim 1, wherein as the position information generation processing, position information of three-dimensional coordinates of each of the first centroid position and the second centroid position is generated. The detection unit includes an image sensor that images the first and second position marker units and outputs the captured image data to the processing unit as the detection data.
An illumination unit that irradiates each of the first and second position marker units with first irradiation light and second irradiation light having different wavelengths from each other,
The first position marker unit is configured to reflect the first irradiation light,
The teaching device according to claim 1, wherein the second position marker unit is configured to reflect the second irradiation light. The processor is
A distance determination process for determining a distance between the first and second position marker units based on the position information;
The teaching device according to any one of claims 1 to 3, wherein when the distance becomes equal to or less than a predetermined distance, a notification process for notifying the fact is performed. The first jig includes a first movable part, a first drive part that drives the first movable part, and a first movable part marker part that indicates a position of the first movable part,
The second jig includes a second movable part, a second drive part that drives the second movable part, and a second movable part marker part that indicates a position of the second movable part,
The processor is
Whether the input work information is information indicating that at least one of the first drive unit and the second drive unit is driven in response to the determination that the input of the work information has been received by the determination process. Drive information determination processing for determining whether or not,
In response to determining that the work information is information indicating that at least one of the first drive unit and the second drive unit is driven by the drive information determination process, as the position information generation process, The position information of each of the first and second movable part marker parts that moves as each of the first and second movable parts operates based on the driving of the first and second drive parts. The teaching device according to any one of claims 1 to 4, wherein the teaching device is generated.   In the control information generation process, the processing unit is configured to perform the operation by the first working unit after moving the first moving unit, and after moving the second moving unit, the second working unit. Control information for correcting the positions of the first and second working parts at the work position is added as the control information to be executed at at least one of the two timings before performing the work by The teaching device according to any one of claims 1 to 5.   The said processing part extracts the some feature point from the produced | generated said positional information in the said positional information generation process, and performs the correction process which approximates the said positional information between the said characteristic points. The teaching device according to any one of claims 1 to 6.   The processing unit samples the position of each of the first and second position marker units based on the detection data as the position information generation process, and generates a position of a sampling point as the position information. The teaching device according to any one of claims 1 to 6.   The processing unit, as the movement information generation process, for each sampling point of the first and second position marker unit generated by the position information generation process, based on the positional relationship between the adjacent sampling points 9. The teaching apparatus according to claim 8, wherein the moving direction is detected, and the moving speed is detected based on a distance between adjacent sampling points and a sampling period.   The processing unit, as the movement information generation process, when the curvature of a curve connecting the sampling points exceeds a predetermined curvature, when the movement speed exceeds a predetermined speed, and when the acceleration at the movement speed is The teaching apparatus according to claim 9, wherein the position information is corrected in at least one case where a predetermined acceleration is exceeded. Comprising a reference marker portion provided at a position serving as a reference for the operation of the robot;
The detection unit detects the reference marker unit,
In the position information generation process, the processing unit performs each of a relative position of the first position marker unit with respect to the reference marker unit and a relative position of the second position marker unit with respect to the reference marker unit. The teaching apparatus according to claim 1, wherein the teaching information is generated as the position information.   The teaching device according to claim 1, wherein the robot includes a serial link mechanism as a driving mechanism of each of the first and second moving units. Generation of control information for controlling the operation of a robot including first and second moving units, a first working unit provided in the first moving unit, and a second working unit provided in the second moving unit. A first jig having a first position marker portion indicating a position of the first moving portion; and a second jig indicating a position of the second moving portion and having a characteristic different from that of the first position marker portion. A second jig having a two-position marker portion, a detection portion for detecting the first and second position marker portions that move with the movement of each of the first and second jigs, and the first and first With respect to a teaching device comprising: an input unit for inputting work information related to work performed by each of the first and second work parts in a state where the jig is moved to a predetermined work position. ,
A determination step of determining whether or not the input of the work information has been received by the input unit;
Based on the detection data in which the detection unit detects each of the first and second position marker units according to the determination result of the determination step, the three-dimensional coordinates of each of the first and second position marker units are determined. A position information generation step for generating position information;
A movement information generation step for generating movement information related to the movement direction and movement speed of each of the first and second position marker units based on the position information;
A series of causing each of the first and second moving units to move in cooperation according to the position information and the movement information, and causing each of the first and second working units to perform work according to the work information. And a control information generation step process for generating the control information of the work.

Images