JP2015136742A - Control device, robot system, robot and calibration method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device capable of performing calibration of a marker for work even when a set position of the marker for work is deviated, and further to provide a robot system, a robot, a calibration method and the like.SOLUTION: A control device 100 includes: a captured image reception section 110 receiving a captured image including a marker for calibration 610 provided on a jig for calibration 600 and a marker for work 710 provided on a jig for work 700; and a processing section 130 causing a robot 300 to perform work by using the jig for work 700 on the basis of the captured image.

Description

  The present invention relates to a control device, a robot system, a robot, a calibration method, and the like.

  In recent years, industrial robots have been increasingly introduced to mechanize and automate work performed by humans at production sites. However, when positioning the robot, it is necessary to perform precise calibration in advance.

  When performing this calibration, there is a method for accurately specifying (correcting) the relative position and orientation relationship between the work reference such as a point, line, and surface as a reference for the work performed by the robot and the robot. As an invention relating to such a calibration method, for example, there is an invention described in Patent Document 1. In the invention described in Patent Document 1, an alignment mark (marker) is provided at a position having a certain positional relationship with respect to the axis coordinate origin of the robot, and the relative position and orientation relationship between the alignment mark and the work reference is determined. Set in advance. Then, the tool gripped by the robot is moved to the position of the alignment mark, the robot's axis coordinates at that time are read, and the deviation between the read value and the coordinates of the alignment mark is taken as an offset amount. To identify (correct) the relative position and orientation relationship.

JP-A-1-234181

  Such work standards are usually different for each work. Therefore, as in the invention disclosed in Patent Document 1 described above, when an accurate relative position and orientation relationship between a work reference and a robot is specified using a marker, the work position changes to a suitable position each time the work changes. It is necessary to affix (set) a marker (alignment mark).

  In recent years, opportunities to produce a small number of various types of parts have increased, and the work performed by the robot often changes frequently. Therefore, there are many opportunities for the operator (teacher, user) to apply the marker to a predetermined position.

  However, it is very difficult for the operator (teacher, user) to attach the marker accurately at a predetermined position without shifting. For example, accurate calibration cannot be performed simply by shifting the marker application position by 5 mm or tilting the marker application position by 10 degrees. In this case, it becomes impossible to accurately grasp the relative position and orientation relationship between the robot and the work reference, and the robot cannot perform accurate work.

  According to one aspect of the present invention, a captured image receiving unit that receives a captured image obtained by capturing a calibration marker provided in a calibration jig, a work marker provided in a work jig, and the imaging And a processing unit that performs work using the work jig based on an image.

  In one embodiment of the present invention, a robot is calibrated based on a captured image in which a calibration marker and a work marker are reflected, and the robot is caused to perform work using a work jig.

  Thereby, even when the setting position (sticking position) of the work marker is deviated, the work marker can be calibrated.

  In one aspect of the present invention, the processing unit performs image recognition processing on the captured image, and at least one of a reference point, a reference line, and a reference plane of the work using the work jig. The relative position and orientation relationship between the work standard and the work marker may be specified.

  In one aspect of the present invention, a calibration jig provided with a calibration marker is used when specifying the relative position and orientation relationship between the work reference and the work marker.

  This makes it possible to accurately specify the relative position and orientation relationship between the work reference and the work marker. Furthermore, it is possible to attach the work marker to the work jig without worrying about the deviation of the sticking position and the sticking posture.

  In one aspect of the present invention, the captured image receiving unit may be configured such that the calibration marker and the work marker are in a state where the calibration marker and the work reference are in a known second relative position and orientation relationship. The captured image obtained by imaging a marker is received, and the processing unit performs marker recognition processing of the calibration marker and the working marker based on the captured image to identify the relative position and orientation relationship May be.

  This makes it possible to specify the relative position and orientation relationship between the work reference and the work marker without specifying the second relative position and orientation relationship by image recognition processing or the like.

  In one aspect of the present invention, the processing unit specifies a third relative position and orientation relationship between the calibration marker and the work marker based on a result of the marker recognition process, and the second The relative position / posture relationship may be specified based on the relative position / posture relationship and the third relative position / posture relationship.

  Thereby, for example, it is possible to specify the relative position / posture relationship between the work reference and the work marker by taking the difference between the second relative position / posture relationship and the third relative position / posture relationship.

  In one aspect of the present invention, the processing unit identifies the work reference in a robot coordinate system based on a marker recognition process result of the work marker and the relative position and orientation relationship, and based on the work reference. The robot may be controlled.

  Thereby, even when the setting position of the work marker is shifted, the robot can be accurately operated based on the work reference.

  In the aspect of the invention, the work marker may be a marker attached to the work jig by an operator.

  As a result, it is possible to efficiently prepare a working jig to which a working marker that correctly corresponds to the work of the robot is attached.

  In one aspect of the present invention, the captured image receiving unit captures the calibration marker and the work marker in a state where the calibration jig and the work jig are in contact with each other. Images may be accepted.

  As a result, the operator can easily align the calibration jig and the work jig.

  In one embodiment of the present invention, at least one of the calibration marker and the work marker may have jig property information.

  Accordingly, it is possible to acquire jig property information by performing marker recognition processing.

  In the aspect of the invention, the processing unit recognizes the jig property of the calibration marker by marker recognition processing of the calibration marker, and based on the jig property, the second relative The position and orientation relationship may be specified.

  As a result, it is possible to use a plurality of calibration jigs having different second relative positions and orientations for each work.

  According to another aspect of the present invention, a calibration jig provided with a calibration marker, a work jig provided with a work marker, the calibration jig, and the work jig are imaged. The present invention relates to a robot system including an imaging unit and a robot that performs work using the work jig by a captured image captured by the imaging unit.

  According to another aspect of the present invention, the work jig is used by a captured image obtained by capturing an image of a calibration marker provided on the calibration jig and a work marker provided on the work jig. Related to the robot that performs the work.

  According to another aspect of the present invention, a captured image in which a calibration jig provided with a calibration marker and a work jig provided with a work marker are received, and the captured image is received. Performing image recognition processing to identify a relative position and orientation relationship between a work reference that is at least one of a reference point, a reference line, and a reference plane of the work using the work jig and the work marker; And a calibration method including:

  Another aspect of the present invention relates to a program that causes a computer to function as each of the above-described units, or a computer-readable information storage device that stores the program.

  According to some aspects of the present invention, there are provided a control device, a robot system, a robot, a calibration method, and the like that can calibrate a work marker even when the setting position of the work marker is shifted. be able to.

Explanatory drawing of the operation | work by the robot using a working jig. Explanatory drawing of the shift | offset | difference of the sticking position and sticking posture of a work marker. Explanatory drawing of the jig for calibration. The system configuration example of this embodiment. Explanatory drawing of the relative position / orientation relationship specifying process. 6 is a flowchart for explaining the flow of calibration processing according to the present embodiment. FIG. 7A to FIG. 7D are explanatory diagrams of a second relative position and orientation relationship. The flowchart explaining the flow of the process of the robot control of this embodiment. FIG. 9A to FIG. 9D are explanatory diagrams of the operation of sucking the screw. Explanatory drawing of the operation | work which acquires an E ring. FIG. 11A and FIG. 11B are configuration examples of a robot. 1 is a configuration example of a robot control system that controls a robot via a network.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. First, as an operation to which the method of the present embodiment is applied, for example, the operation shown in FIG. 1 is considered. The robot RB shown in FIG. 1 performs an operation of placing the work WK gripped by the gripping part (hand) HD side by side at positions P1 to P3. Therefore, first, it is necessary to recognize the positions P1 to P3. In this case, the work reference point SP that is the work reference is recognized, and the positions P1 to P3 are recognized based on the work reference point SP. The work standard is a point, a line, a surface, or the like that serves as a standard when performing work, and is the work standard point SP in the example of FIG.

  At that time, the operator (teacher, user) specifies the position of the work reference in the robot coordinate system using the work jig WJG to which the work marker WMK is attached (set). Specifically, the work marker WMK is recognized based on a captured image captured by the imaging unit (camera) CM, and a point having a given positional relationship with the position of the recognized work marker WMK is determined as a work reference point. It is specified as SP. The work jigs are individually prepared for each work or each work standard. For example, as shown in FIG. 1, a part of the work jig WJG is matched with the work standard (SP) (contact or adjacent). Is possible.

  However, as described above, since the operator (teacher, marker) is affixed (set) to the work jig by the operator (teacher, marker), the application position and the application posture are often shifted. For example, as shown in FIG. 2, the work marker is to be attached to the ideal application position IP of the work jig WJG, but the application position is shifted and the actual application position becomes the position RP. Often. In this case, the work reference may not be accurately identified based on the work marker. Specifically, it is not possible to specify an accurate work standard simply by affixing the attachment position by 5 mm or the application posture by 10 degrees. In Patent Document 1 described above, the alignment mark corresponds to a work marker, and there is a possibility that a deviation occurs when the alignment mark is set.

  Therefore, in the present embodiment, as shown in FIG. 3, in addition to the working jig WJG, a calibration jig CJG to which the calibration marker CMK is attached (set) is used, and the work marker WMK is set before the work. Perform calibration. Specifically, an accurate relative position and orientation relationship between the work marker WMK and the work reference (work reference point SP) is specified using the calibration jig CJG. If the correct relative position and orientation relationship between the work marker WMK and the work reference is known, even if the application position of the work marker WMK is deviated from the original ideal application position, the position of the work reference in the robot coordinate system can be accurately determined. It becomes possible to specify.

  Specifically, FIG. 4 illustrates a configuration example of the control device 100 according to the present embodiment and the robot 300 including the control device 100. The control device 100 according to the present embodiment includes a captured image receiving unit 110 that receives a captured image, and a processing unit 130 that causes the robot 300 to perform work using the work jig 700 based on the captured image. The control device 100 of the present embodiment is not limited to the configuration of FIG. 4, and various modifications such as omitting some of these components or adding other components are possible. is there. Further, the control device 100 of this embodiment is included in the robot 300 as shown in FIG. 11A described later, and may be configured integrally with the robot 300. Furthermore, as shown in FIG. 12 described later, the function of the control device 100 may be realized by a server 500 and a terminal device 330 included in each robot 300.

  First, the captured image reception unit 110 acquires a captured image in which a calibration marker 610 provided in the calibration jig 600 and a work marker 710 provided in the work jig 700 are reflected. The captured image receiving unit 110 is realized by an interface unit that receives a captured image from the imaging unit 200, or a communication unit that receives a captured image from the imaging unit 200 via a network including at least one of wired and wireless. it can.

  Then, the processing unit 130 performs image recognition processing on the captured image, a work reference that is at least one of a reference point, a reference line, and a reference plane of the work using the work jig 700, and a work The relative position and orientation relationship with the marker 710 is specified. The function of the processing unit 130 can be realized by hardware such as various processors (CPU or the like), ASIC (gate array or the like), a program, or the like.

  Specifically, the captured image reception unit 110 captures the calibration marker 610 and the work marker 710 in a state where the calibration marker 610 and the work reference are in the known second relative position and orientation relationship. Obtained captured images are acquired.

  Then, the processing unit 130 performs marker recognition processing of the calibration marker 610 and the work marker 710 based on the captured image, and specifies the work position and the relative position and orientation relationship of the work marker 710.

  More specifically, the processing unit 130 specifies the third relative position and orientation relationship between the calibration marker 610 and the work marker 710 based on the result of the marker recognition process, and the second relative position and orientation. Based on the relationship and the third relative position and orientation relationship, the work reference and the relative position and orientation relationship of the work marker 710 are specified.

  Here, a specific example will be described with reference to FIG. First, in this example, the calibration marker CMK and the work marker WMK are imaged in a state where the calibration marker CMK and the work reference point SP (work reference) are in the second relative position and orientation relationship.

  Here, the calibration marker CMK is accurately affixed to the calibration jig CJG by the manufacturer. That is, the calibration marker CMK is affixed to the calibration jig CJG so that the relative position and orientation relationship between the calibration marker CMK and the work reference point SP becomes a predetermined second position and orientation relationship. Yes. For example, the second relative position / posture relationship in FIG. 5 is a relative position / posture relationship represented by a vector DR2 from the center point CMC of the calibration marker CMK to the work reference point SP. It is not necessary to use the calibration jig CJG when performing actual work. The calibration jig CJG is used to accurately specify the relative position and orientation relationship between the work marker WMK and the work reference before work. However, it is also possible to use a tool used for work as a calibration jig CJG as in the examples of FIGS. 9A to 9D described later and the example of FIG.

  Next, marker recognition processing is performed based on the captured image to recognize the work marker WMK and the calibration marker CMK, and specify the third relative position and orientation relationship. For example, the third relative position and orientation relationship in FIG. 5 is a relative position and orientation relationship represented by a vector DR3 from the center point CMC of the calibration marker CMK to the center point WMC of the work marker WMK.

  The relative position and orientation relationship between the work reference point SP and the work marker WMK is expressed by taking the difference between the vector DR2 that is the second relative position and orientation relationship and the vector DR3 that is the third relative position and orientation relationship. The vector DR is specified. For example, the vector DR is a vector from the center point WMC of the work marker WMK to the work reference point SP.

  In the example of FIG. 5, the case where the work reference point SP is used as the work reference has been described. However, as described above, a work reference line or a work reference plane may be used as the work reference. For example, the work reference line is a first straight line connecting the point SP and the end point P1, a second straight line connecting the point SP and the end point P2, and the like. The work reference plane includes a first straight line and includes a first surface perpendicular to the paper surface, a second straight line, the second surface perpendicular to the paper surface, and the first straight line. And a third surface including the second straight line.

  This makes it possible to calibrate the work marker even when the work marker setting position (sticking position) is deviated, and as described above, the relative position and orientation relationship between the work reference and the work marker. Can be specified accurately.

  Further, since the second relative position / posture relationship is a known relative position / posture relationship, the relative position / posture of the work reference and the work marker can be determined without specifying the second relative position / posture relationship by image recognition processing or the like. It becomes possible to specify the relationship.

  Furthermore, by specifying the third relative position / posture relationship, for example, by taking the difference between the second relative position / posture relationship and the third relative position / posture relationship, the relative position / posture relationship between the work reference and the work marker can be determined. It becomes possible to specify.

  Then, the processing unit 130 specifies a work reference in the robot coordinate system based on the result of the marker recognition process of the work marker 710 and the specified relative position and orientation relationship, and controls the robot 300 based on the work reference. .

  Thereby, even when the setting position of the work marker is shifted, the robot 300 can be accurately operated based on the work reference. For example, in the example of FIG. 1 described above, even when the work marker WMK is attached with a deviation, the work reference point SP in the robot coordinate system is accurately recognized by the method described above, and the recognized work reference point SP is used as the recognized work reference point SP. Based on this, it is possible to operate the robot RB and place the workpiece WK side by side at the positions P1 to P3.

  Since the processing as described above is performed, it is possible to attach the work marker to the work jig without worrying about the deviation of the sticking position and the sticking posture.

  Therefore, the work marker 710 may be a marker attached to the work jig 700 by the worker.

  This makes it possible to improve work preparation efficiency, particularly when there are many types of work to be performed by the robot. The preparation of the work in this case is to prepare a work jig to which a work marker corresponding to the work correctly is attached. The work marker and the calibration marker are not limited to those shown in the figure, and any marker may be used.

  Further, the present embodiment is clearly different from Patent Document 1 described above in that a calibration jig and a calibration marker are used. In Patent Document 1 described above, the work reference is specified based on the alignment mark (work marker). However, there is a possibility that the setting position of the alignment mark is shifted. In this case, the work reference is accurately specified. Can not do it. On the other hand, this embodiment specifies an accurate relative position and orientation relationship between the work marker and the work reference using the calibration marker. For this reason, even when the work markers are pasted with a slight shift, it is possible to accurately specify the work reference position (position and posture) in the robot coordinate system and control the robot.

2. Details of Processing Next, the flow of calibration processing in the present embodiment will be described with reference to the flowchart of FIG.

  First, a worker (user, teacher) attaches a work marker to a work jig (S101). Next, the operator places the calibration jig and the work jig in accordance with the work standard (S102), and the imaging unit 200 images the calibration marker and the work marker (S103).

  At this time, for example, as shown in FIG. 5 described above, the captured image receiving unit 110 has the calibration marker 610 (CMK) and the calibration jig 600 (CJG) and the work jig 700 (WJG) in contact with each other. A captured image obtained by capturing the work marker 710 (WMK) is acquired (S102, S103).

  In this way, by placing the two jigs in contact with each other, the operator can easily align the calibration jig and the work jig.

  Next, the processing unit 130 performs marker recognition processing for the calibration marker and the work marker based on the captured image (S104). Then, the processing unit 130 specifies the second relative position and orientation relationship based on the result of the marker recognition process (S105).

  Here, a specific example of the process of specifying the second relative position / posture relationship will be described. First, at least one of the calibration marker 610 and the work marker 710 has jig property information. The jig property information includes, for example, information on a jig or marker ID or jig shape, work standard information corresponding to the jig, work content information, and the like.

  Here, for example, an example in which the calibration marker has a jig ID (marker ID) as jig property information will be described with reference to FIGS. 7A to 7D.

  In this example, different calibration markers are attached to each calibration jig having a different shape. Specifically, a calibration marker CMK1 is attached to the calibration jig CJG1 in FIG. 7A, and a calibration marker CMK2 is attached to the calibration jig CJG2 in FIG. 7B. A calibration marker CMK3 is affixed to the calibration jig CJG3 in FIG. These calibration jigs (CJG1 to CJG3) are jigs having different shapes. These calibration markers (CMK1 to CMK3) are also different from each other. Furthermore, the jig properties possessed by these markers for calibration (CMK1 to CMK3) are also different from each other.

  In this case, if the upper left corner of each calibration jig is placed in alignment with the work reference point, the second relative position and orientation relationship between the calibration marker and the work reference point will be different from each other. Specifically, in the case of FIG. 7A, the second relative position and orientation relationship between the calibration marker MK1 and the work reference point SP1 is PR1, and in the case of FIG. 7B, the calibration is performed. The second relative position and orientation relationship between the calibration marker MK2 and the work reference point SP2 is PR2, and in the case of FIG. 7C, the second relative position between the calibration marker MK3 and the work reference point SP3. The posture relationship is PR3. In this example, each such second relative position and orientation relationship is stored in association with each jig ID as shown in the table of FIG.

  Then, the processing unit 130 recognizes the jig property of the calibration marker 610 through the marker recognition process, and specifies the second relative position and orientation relationship based on the jig property (S105).

  Specifically, in this example, the jig ID of the calibration marker is recognized by the marker recognition process, and the second relative position and orientation relationship stored in association with the recognized jig ID is specified. For example, if the jig ID is specified as CMK1 by the marker recognition process, the second relative position / posture relationship is specified as PR1. The same applies to other cases. In this example, the case where the work reference is a work reference point has been described. However, the same processing can be performed when the work reference is a work reference line or a work reference plane. Further, when the calibration marker is not a jig ID but directly has information on the second relative position / posture relationship, the second relative position / posture relationship is not specified via the storage unit. The second relative position / posture relationship may be directly recognized by the marker recognition process.

  Accordingly, it is possible to acquire jig property information by performing marker recognition processing.

  Then, it is possible to specify the second relative position and orientation relationship based on the result of the marker recognition process. Accordingly, it is possible to use a plurality of calibration jigs having different second relative positions and orientations for each work.

  Next, the processing unit 130 specifies the third relative position and orientation relationship between the calibration marker and the work marker based on the result of the marker recognition process of the calibration marker (S106). For example, the third relative position / posture relationship is DR3 in the example of FIG. 5 described above.

  Then, the processing unit 130 identifies the relative position and orientation relationship between the work reference and the work marker based on the second relative position and orientation relationship and the third relative position and orientation relationship (S107), and the identified relative position and orientation relationship. And the jig ID are associated and stored in the storage unit (S108). The above is the flow of the calibration process in the present embodiment.

  Next, the flow of processing when actually operating the robot will be described using the flowchart of FIG.

  First, the worker installs a work jig in the work area (S201). At this time, as in the case of calibration, the work jig is installed in accordance with the work standard, but it is not necessary to install the calibration jig.

  Next, the imaging unit 200 captures a work jig (work marker) and a work reference, and obtains a captured image (S202). Then, the jig ID is recognized by the marker recognition process of the work marker, and the relative position and orientation relationship stored in association with the recognized jig ID is specified (S203). This relative position / posture relationship is stored in the storage unit in association with the jig ID in step S108 of FIG.

  Then, based on the specified relative position and orientation relationship, the position (position and orientation) of the work reference in the robot coordinate system is specified (S204), and the robot is operated based on the specified work reference (S205). The above is the flow of processing when operating the robot.

  In the above description, the example using the L-shaped work jig WJG and the rectangular calibration jig CJG as shown in FIG. 5 has been described, but the present embodiment is not limited to this. For example, as shown in FIGS. 9A to 9D, a screwdriver may be used as the calibration jig CJG, and a screw feeder may be used as the work jig WJG. Hereinafter, examples of FIGS. 9A to 9D will be described.

  First, FIG. 9A to FIG. 9D illustrate how the robot performs the work of assembling the screw SC to the work WK (the cutter unit in this example). In this example, attention is particularly paid to the work of obtaining the screw SC in the screw feeder WJG. When the screw SC is to be picked up by the screw feeder WJG, it is necessary to align the tip CP of the driver CJG gripped by the robot hand HD with the tip WP of the screw SC. In other words, in this example, the tip WP of the screw SC becomes the work reference point, and the robot is caused to perform the work of aligning the tip CP of the driver CJG with the work reference point WP.

  When this work is actually performed, a marker recognition process is performed on the work marker WMK attached to the screw feeder WJG, and the robot is operated based on the result of the marker recognition process. However, since the work marker WMK is affixed by the operator, the affixing position and the affixing posture may be deviated as described above.

  Therefore, the work marker WMK is calibrated based on the driver CJG. Note that a calibration marker CMK is accurately attached to the driver CJG by the manufacturer.

  At this time, first, as shown in FIG. 9A, the driver CJG is gripped by the robot hand HD. Next, as shown in FIGS. 9B and 9C, the driver CJG is moved so that the tip CP of the gripped driver CJG contacts the tip WP of the screw SC.

  Then, a state where the tip CP of the driver CJG is in contact with the tip WP of the screw SC is imaged by the imaging unit, and marker recognition processing of the work marker WMK and the calibration marker CMK is performed. Next, as described above, the second relative position / posture relationship and the third relative position / posture relationship are specified based on the result of the marker recognition process. The second relative position / posture relationship at this time is the relative position / posture relationship between the calibration marker CMK and the tip WP of the screw SC (or the tip CP of the driver CJG), and the third relative position / posture relationship is the calibration. This is the relative position and orientation relationship between the work marker CMK and the work marker WMK.

  Next, the relative position and orientation relationship between the work marker WMK and the tip WP of the screw SC is specified based on the specified second relative position and orientation relationship and the third relative position and orientation relationship. The relationship is stored in the storage unit in association with the jig ID of the work marker WMK. This is the calibration process.

  As described above, when the robot actually performs the work of attracting the screw SC to the tip CP of the driver CJG, the marker recognition processing of the work marker WMK is performed and stored in association with the recognized jig ID. Identify the relative position and orientation relationship. Then, the position of the tip WP of the screw SC in the robot coordinate system is specified based on the specified relative position and orientation relationship, the robot is operated based on the position of the tip WP of the specified screw SC, and the tip of the driver CJG Adhere the screw SC to the CP.

  Finally, as shown in FIG. 9D, the screw SC attracted to the tip CP of the driver CJG is assembled to the work WK. In this example, since the driver CJG functions as both a calibration jig and a tool, the driver CJG is used during actual work. Here, the tool is a tool for performing work such as machining on a workpiece or work target, and refers to a tool such as a drill or a screwdriver, for example. Note that the tool may be gripped by the gripper 310 or may be provided directly at the end point of the arm of the robot 300.

  Furthermore, another example is shown in FIG. In this example, E-rings ER are stacked side by side along the axis of the E-ring feeder WJG. At this time, the E ring setter CJG is pressed against the E ring ER to cause the robot to perform an operation of acquiring the E ring ER.

  In this case, since the E ring setter CJG is used as a calibration jig, a calibration marker CMK is attached to the E ring setter CJG. On the other hand, the E-ring supply machine WJG is used as a work jig, and a work marker WMK is attached to the E-ring supply machine WJG.

  Then, with the position of the bottom E-ring as the work reference point WP, the work marker WMK and the work marker WMK are calibrated in the state where the tip CP of the E-ring setter CJG is pressed. The relative position and orientation relationship of the reference point WP is specified and stored.

  As a result, when performing actual work, by performing marker recognition processing of the work marker WMK attached to the E-ring supply machine WJG, an accurate work reference point WP in the robot coordinate system is recognized, and the E-ring It is possible to cause the robot to perform an operation of acquiring the ER.

3. Next, the robot 300 of this embodiment will be described. As shown in FIG. 4 described above, the robot 300 includes a control device 100 (having a captured image receiving unit 110 and a processing unit 130), a robot control unit 350, and a robot mechanism 380. The robot system includes a calibration jig 600, a work jig 700, an imaging unit 200, and a robot 300.

  The robot mechanism 380 includes an end effector 310, an arm 320, and the like. Here, the end effector 310 is a part attached to the end point of the arm in order to grip, lift, lift, adsorb, or process the work (work object). Say. The end effector 310 may be, for example, a hand (gripping unit), a hook, a suction cup, or the like. Further, a plurality of end effectors may be provided for one arm. The arm is a part of the robot 300 and refers to a movable part including one or more joints.

  Next, the processing unit 130 specifies the work reference in the robot coordinate system by performing each process described above, and outputs the specified work reference to the robot control unit 350. Then, the robot control unit 350 controls the robot mechanism 380 based on the acquired work standard in the robot coordinate system.

  In this case, the robot 300 may be configured such that the robot body and the control device 100 are integrated as shown in FIG. That is, the robot 300 may include the control device 100. Specifically, as illustrated in FIG. 11A, the robot 300 includes a robot mechanism 380 and a base unit portion that supports the robot mechanism 380, and the control device 100 and the robot control portion 350 are stored in the base unit portion. It may be a thing. The robot 300 in FIG. 11A is provided with wheels and the like in the base unit portion so that the entire robot can move. The robot 300 may be moved manually, or may be moved by providing a motor that drives wheels and controlling the motor by the robot control unit 350. 11A is an example of a double arm type, the robot 300 may be a single arm type or other multi-arm type robot as shown in FIG. 11B described later.

  Furthermore, as illustrated in FIG. 11A, the control device 100 is not necessarily provided in the base unit portion provided under the robot 300. For example, as shown in FIG. 11B, the robot 300 of the present embodiment may be configured such that the robot body 300 (robot) and the control device 100 are separate. In this case, some or all of the functions of the control device 100 are realized by, for example, a PC (Personal Computer). In this case, the control device 100 and the robot 300 are communicatively connected via a network including at least one of wired and wireless.

  As shown in FIG. 12, the function of the control device 100 may be realized by a server 500 that is connected to the robot 300 via a network 400 including at least one of wired and wireless.

  Alternatively, in the present embodiment, a part of the processing of the control device of the present invention may be performed by the control device on the server 500 side. In this case, the processing is realized by distributed processing with the control device provided on the robot 300 side. The control device on the robot 300 side is realized by a terminal device 330 (control unit) installed in the robot 300, for example.

  In this case, the control device on the server 500 side performs a process assigned to the control device of the server 500 among the processes in the control device of the present invention. On the other hand, the control device provided in the robot 300 performs the process assigned to the control device of the robot 300 among the processes of the control device of the present invention. Each process of the control apparatus of the present invention may be a process assigned to the server 500 side or a process assigned to the robot 300 side.

  Thereby, for example, the server 500 having a higher processing capability than the terminal device 330 can perform processing with a large processing amount. Further, for example, the server 500 can collectively control the operations of the robots 300, and it is easy to cause a plurality of robots 300 to perform cooperative operations.

  The imaging unit (camera) 200 includes an imaging element such as a charge-coupled device (CCD) and an optical system. The imaging unit 200 is disposed at an angle such that a work marker, a calibration marker, a work reference, and the like fall within the angle of view of the imaging unit 200, for example, on a ceiling or a work table. Then, the imaging unit 200 outputs information of the captured image to the control device 100 (captured image reception unit 110). However, in the present embodiment, the information of the captured image is output as it is to the control device 100, but the present invention is not limited to this. For example, the imaging unit 200 can include a device (processor) used for image processing or the like.

  Moreover, in the above description, in order to control the robot 300, the process which specifies the relative position and orientation relationship by the control device 100 has been described, but the present embodiment is not limited to this. For example, the specified relative position and orientation relationship can be used for CAD (Computer Aided Design) matching (object detection) or the like.

  Note that a part or most of the processing of the control device, the robot system, the robot, and the like of the present embodiment may be realized by a program. In this case, a control device, a robot system, a robot, and the like according to the present embodiment are realized by a processor such as a CPU executing a program. Specifically, a program stored in a non-temporary information storage device is read, and a processor such as a CPU executes the read program. Here, an information storage device (device readable by a computer) stores programs, data, and the like, and functions as an optical disk (DVD, CD, etc.), HDD (hard disk drive), or memory (card type). It can be realized by memory, ROM, etc. A processor such as a CPU performs various processes according to the present embodiment based on a program (data) stored in the information storage device. That is, in the information storage device, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing a computer to execute the processing of each unit). Is memorized.

  Further, the control device or the like of the present embodiment may include a processor and a memory. The processor here may be, for example, a CPU (Central Processing Unit). However, the processor is not limited to the CPU, and various processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) can be used. The processor may be a hardware circuit based on ASIC (Application Specific Integrated Circuit). The memory stores instructions that can be read by a computer. When the instructions are executed by the processor, each unit of the control device according to the present embodiment is realized. The memory here may be a semiconductor memory such as an SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory), or may be a register or a hard disk. In addition, the instruction here may be an instruction of an instruction set constituting the program, or an instruction for instructing an operation to the hardware circuit of the processor.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described at least once together with a different term having a broader meaning or the same meaning in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. Further, the configurations and operations of the control device and the robot are not limited to those described in the present embodiment, and various modifications can be made.

100 control device, 110 captured image reception unit, 130 processing unit, 200 imaging unit,
300 robot, 310 end effector, 320 arm, 330 terminal device, 350 robot controller, 380 robot mechanism, 400 network,
500 servers, 600 calibration jigs,
610 Calibration marker, 700 Work jig, 710 Work marker

Claims (12)

A captured image reception unit that receives a captured image obtained by imaging a calibration marker provided in a calibration jig and a work marker provided in a work jig;
A processing unit that causes the robot to perform work using the work jig based on the captured image;
The control apparatus characterized by including. In claim 1,
The processor is
Relative position of a work reference that is at least one of a reference point, a reference line, and a reference surface of the work using the work jig and the work marker by performing image recognition processing on the captured image A control device characterized by specifying an attitude relationship. In claim 2,
The captured image reception unit
Accepting the captured image obtained by imaging the calibration marker and the work marker in a state where the calibration marker and the work reference are in a known second relative position and orientation relationship;
The processor is
A control apparatus that performs marker recognition processing of the calibration marker and the work marker based on the captured image to identify the relative position and orientation relationship. In claim 3,
The processor is
Based on the result of the marker recognition process, a third relative position / posture relationship between the calibration marker and the work marker is specified, and the second relative position / posture relationship and the third relative position / posture relationship are determined. A control device that identifies the relative position and orientation relationship based on the relationship. In any of claims 2 to 4,
The processor is
A control device that identifies the work reference in a robot coordinate system based on a result of marker recognition processing of the work marker and the relative position and orientation relationship, and controls the robot based on the work reference. . In any of claims 2 to 5,
The working marker is
A control device characterized by being a marker affixed to the work jig by an operator. In any one of Claims 2 thru | or 6.
The captured image reception unit
A control apparatus that receives the captured image obtained by imaging the calibration marker and the work marker in a state where the calibration jig and the work jig are in contact with each other. In any one of Claims 2 thru | or 7,
At least one of the calibration marker and the working marker is:
A control device having information on jig properties. In claim 8,
The processor is
A control apparatus that recognizes the jig property of the calibration marker by the marker recognition processing of the calibration marker and identifies the second relative position and orientation relationship based on the jig property. A calibration jig provided with a calibration marker;
A working jig provided with a working marker;
An imaging unit for imaging the calibration jig and the work jig;
And a robot that performs work using the work jig based on a captured image captured by the image capturing unit.   A robot that performs an operation using the work jig by using a captured image obtained by an image pickup unit of a calibration marker provided on the calibration jig and a work marker provided on the work jig. . Receiving a captured image showing a calibration jig provided with a calibration marker and a work jig provided with a work marker;
Relative position of a work reference that is at least one of a reference point, a reference line, and a reference surface of the work using the work jig and the work marker by performing image recognition processing on the captured image Identifying posture relationships;
A calibration method comprising:

Images