JP2015157342A - Robot, robot system, control device and control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot that makes it possible to accurately perform calibration between an imaging unit and the robot.SOLUTION: A robot comprises a hand, a force sensor, and a control unit that operates the hand, wherein the control unit positions the hand by a dent at a predetermined position of a calibration tool and then an imaging unit captures an image of the predetermined position.

Description

  The present invention relates to a robot, a robot system, a control device, and a control method.

Calibration of the coordinate system of the captured image of the imaging unit and the coordinate system that controls the hand of the robot is performed.
As an example, in the “coordinate system coupling method for determining the relationship between a sensor coordinate system and a robot tip in a robot-visual sensor system” described in Patent Document 1, a touch-up provided on a calibration board, which is a calibration jig. The reference point (touch-up point) is visually aligned with the tip of the robot arm (see Patent Document 1).
As for the accuracy evaluation of the recognition position of the marker, for example, after removing the calibration board, the marker is placed and the amount of deviation between the marker and the tip of the robot arm is visually measured.

JP-A-8-210816

However, in the technique according to Patent Document 1, since the touch-up point provided on the calibration board and the tip of the robot arm are visually aligned, there is no reading error due to the position or angle viewed by a person (operator). Occur. In addition, variations caused by people (individuals) such as different positions for different workers also occur.
In addition, since the accuracy evaluation of the marker recognition position is performed independently of calibration using the calibration board, it is difficult to place the marker recognition reference point at the same position as the touch-up point. Even if the accuracy is low, the quality of the touch-up cannot be reflected and the evaluation result cannot be utilized.

  SUMMARY An advantage of some aspects of the invention is that it provides a robot, a robot system, a control device, and a control method that can accurately calibrate an imaging unit and a robot.

One aspect of the present invention is made to solve the above-described problem, and includes a hand, a force sensor, and a control unit that operates the hand. The control unit includes a calibration jig. In the robot, the imaging unit images the predetermined position after the hand is positioned by the depression disposed at the predetermined position.
With this configuration, in the robot, the control unit positions the hand with the depression disposed at a predetermined position of the calibration jig, and then the imaging unit images the predetermined position. Thereby, in the robot, the positioning of the hand can be performed with high accuracy, and the calibration between the imaging unit and the robot can be performed with high accuracy.

One aspect of the present invention may be configured such that, in the robot, the control unit performs impedance control based on a value detected by the force sensor and positions the hand.
With this configuration, in the robot, the control unit performs impedance control based on the value detected by the force sensor, and positions the hand. As a result, the robot can accurately perform positioning of the hand by performing an operation of searching for the depression with the hand.

In one aspect of the present invention, in the robot, the control unit may be configured to position the hand in a state where a gripping jig having a shape corresponding to the shape of the depression is gripped by the hand. Also good.
With this configuration, in the robot, the control unit positions the hand in a state where the gripping jig having a shape corresponding to the shape of the depression is gripped by the hand. As a result, the robot can accurately position the hand by fitting the recesses having shapes corresponding to each other and the holding jig.

In one aspect of the present invention, in the robot, a configuration in which the calibration jig is a jig in which a board having a checker pattern and a transparent board having the depressions are combined may be used.
With this configuration, the robot performs calibration using a calibration jig in which a board having a checker pattern and a transparent board having depressions are combined. As a result, the robot can accurately calibrate the imaging unit and the robot.

In one embodiment of the present invention, in the robot, the calibration jig is a jig in which a board having a checker pattern and a board on which a marker is attached to the transparent board having the depression are combined. May be.
With this configuration, the robot performs calibration using a calibration jig in which a board having a checker pattern and a transparent board having a depression and a board with a marker are combined. As a result, the robot can accurately calibrate the imaging unit and the robot.

One aspect of the present invention is a robot, wherein a positioning projection is provided on one of the two boards combined with the calibration jig, and a positioning hole is provided on the other to pass the projection. May be used.
With this configuration, the robot performs calibration using a calibration jig in which two boards are positioned and combined. As a result, the robot can accurately calibrate the imaging unit and the robot.

One aspect of the present invention is made to solve the above-described problem, and includes a robot including an imaging unit, a hand and a force sensor, a control unit that operates the robot, a calibration jig, And the control unit causes the imaging unit to image the predetermined position after positioning the hand by a depression disposed at a predetermined position of the calibration jig, or sets the predetermined position before the positioning. It is a robot system which makes the said imaging part image.
With this configuration, in the robot system, the control unit causes the imaging unit to take an image of the predetermined position after positioning the hand by the depression disposed at the predetermined position of the calibration jig, or the predetermined position before the positioning. To the imaging unit. Accordingly, in the robot system, the hand can be positioned with high accuracy, and the imaging unit and the robot can be calibrated with high accuracy.

One aspect of the present invention is made to solve the above-described problem, and includes an image receiving unit that receives a captured image captured by the imaging unit, and a processing unit that operates a robot. The controller is configured to image the predetermined position after the robot hand is positioned by a depression disposed at a predetermined position of the calibration jig.
With this configuration, in the control device, the processing unit positions the robot hand with the depression disposed at a predetermined position of the calibration jig, and then the imaging unit images the predetermined position. As a result, in the system, the control device can accurately position the hand, and can accurately calibrate the imaging unit and the robot.

One aspect of the present invention is made to solve the above-described problem. The robot hand is positioned by a depression disposed at a predetermined position of a calibration jig, and the predetermined position is set by an imaging unit. A control method including imaging.
With this configuration, in the control method, the hand of the robot is positioned by the depression disposed at the predetermined position of the calibration jig, and the predetermined position is imaged by the imaging unit. Thereby, in the control method, the positioning of the hand can be performed with high accuracy, and the calibration between the imaging unit and the robot can be performed with high accuracy.

  As described above, according to the robot, the robot system, the control apparatus, and the control method according to the present invention, the imaging unit images the predetermined position after the hand is positioned by the depression disposed at the predetermined position of the calibration jig. To do. Thereby, in the robot, robot system, control apparatus, and control method according to the present invention, the hand can be accurately positioned, and the imaging unit and the robot can be accurately calibrated.

It is a figure showing an example of rough composition of a robot system concerning one embodiment of the present invention. It is a figure which shows the schematic functional block of the control part which concerns on one Embodiment of this invention. It is a figure which shows an example of the jig | tool for calibration which concerns on one Embodiment of this invention. (A) is a figure which shows the example of a one part structure of an example of the jig | tool for calibration which concerns on one Embodiment of this invention, (B) is the sectional drawing. It is a figure which shows another example of the jig | tool for calibration which concerns on one Embodiment of this invention. (A) is a figure which shows the example of a structure of a part of other example of the marker precision evaluation board which concerns on one Embodiment of this invention, (B) is the sectional drawing. (A), (B), (C) is a figure which shows an example of the positioning using the impedance control which concerns on one Embodiment of this invention. It is a flowchart which shows an example of the procedure of the process performed in the robot system which concerns on one Embodiment of this invention. (A), (B), (C) is a figure showing other examples of positioning using impedance control concerning one embodiment of the present invention. It is a figure which shows schematic structure of the robot which concerns on the 1st modification of embodiment of this invention. It is a figure which shows schematic structure of the robot system which concerns on the 2nd modification of embodiment of this invention. (A) is a figure which shows the positioning with respect to the touchup point which concerns on a comparative example, (B) is a figure which shows the image recognition using the marker which concerns on a comparative example.

Embodiments of the present invention will be described in detail with reference to the drawings.
[Outline of Robot System According to this Embodiment]
FIG. 1 is a diagram illustrating a schematic configuration example of a robot system 1 according to an embodiment of the present invention.
The robot system 1 according to the present embodiment includes a robot 11, a camera 12 that is an imaging unit, and a line 13 that connects these to enable communication. The robot 11 is a single-arm robot including one manipulator (arm), and includes a hand (robot hand) 21 attached to the tip of the manipulator and a control unit 41. The hand 21 is provided with a force sensor 31.
As the manipulator, a 6-axis manipulator is used. As another configuration example, a 7-axis manipulator or a 5-axis or less manipulator may be used.
As the line 13, for example, a wireless line may be used instead of a wired line.

FIG. 1 shows a calibration jig 62 placed (arranged) on a table (working desk) 61 and a gripping jig 63 gripped by the hand 21. In FIG. 1, a checker pattern is shown as an example of a schematic appearance of the calibration jig 62, but this is a schematic and may actually have various appearances.
In the present embodiment, the robot 11 is installed in an arrangement that allows the gripping jig 63 gripped by the hand 21 to contact (touch up) the calibration jig 62. The calibration jig 62 is installed in an arrangement capable of imaging.
The hand 21 may have an arbitrary shape. In the present embodiment, the hand 21 has two openable / closable fingers (bar-shaped portions) that can grip the gripping jig 63. The number of fingers that the hand 21 has may be arbitrary. Moreover, the finger | toe which the hand 21 has may be linear form, for example, or one part or all part may be curvilinear form or broken line form.
The control unit 41 controls the operation of the robot 11 (including the hand 21) and the camera 12.

FIG. 2 is a diagram showing a schematic functional block of the control unit 41 according to the embodiment of the present invention.
The control unit 41 according to the present embodiment includes a robot motion processing unit 101, an image information receiving unit 102, a force sensor value receiving unit 103, a storage unit 104, an input unit 105, and an output unit 106.
The robot motion processing unit 101 includes a positioning unit 121, an image processing unit 122, and a calibration unit 123. The robot operation processing unit 101 performs, for example, image processing and control processing for the robot 11.

The image information receiving unit 102 receives and acquires information on an image output from the camera 12 via the line 13, thereby receiving and acquiring the information on the image. Further, the image information receiving unit 102 may control the operation of the camera 12 by communicating a signal with the camera 12 via the line 13.
In the present embodiment, the camera 12 captures an image and outputs information of the captured image to the image information receiving unit 102 via the line 13. As the image information, for example, two-dimensional (planar) information may be used, or three-dimensional (space) information may be used by using two or more cameras. .

The force sensor value receiving unit 103 receives and acquires information on the force sensor value output from the force sensor 31 via a line (not shown). The force sensor value receiving unit 103 may control the operation of the force sensor 31 by communicating a signal with the force sensor 31 via a line.
In the present embodiment, the force sensor 31 detects a force applied to the hand 21 and outputs a value indicating the detected force as a force sensor value to the force sensor value receiving unit 103 via a line. The force sensor 31 may detect a force and a moment (torque) for each of the three-dimensional orthogonal coordinate axes, for example, and the value of the moment may be included in the force sensor value (force value) in the present embodiment.
As the line, for example, a wireless line may be used instead of a wired line.

The storage unit 104 includes a storage medium that stores various types of information. The storage unit 104 stores, for example, information on programs used in the robot motion processing unit 101 and information on parameters (for example, numerical values) used in various processes.
The input unit 105 receives an input from the outside. The input unit 105 includes, for example, a keyboard and a mouse that receive an operation input operated by a user. Further, the input unit 105 may be configured to include a function of receiving input from an external device, for example.
The output unit 106 performs output to the outside. The output unit 106 includes, for example, a screen (for example, a liquid crystal display) that displays various types of information as images to the user, a speaker that outputs audio information to the user, and the like. The output unit 106 may be configured to include a function of outputting information to an external device, for example.

The robot operation processing unit 101 is configured using, for example, a CPU (Central Processing Unit) and performs various processes.
The positioning unit 121 positions the hand 21 with respect to the calibration jig 62.
The image processing unit 122 processes the image of the calibration jig 62.
The calibration unit 123 calibrates the camera 12 and the hand 21.

[Example of calibration jig according to this embodiment]
FIG. 3 is a diagram illustrating an example of the calibration jig 62-1 according to the embodiment of the present invention. The calibration jig 62-1 is an example of the calibration jig 62 shown in FIG.
The calibration jig 62-1 according to the present embodiment is configured by combining the calibration board 201 and the transparent board 301. Each of the calibration board 201 and the transparent board 301 has a plate shape having a square (square or rectangular) surface, for example, the same shape. The material of the transparent board 301 is made of a transparent material (a transparent material) such as glass (for example, silicate glass) or a resin (for example, acrylic glass or vinyl chloride).

The calibration board 201 includes positioning pins (side pins) 221-1 to 221-4 at the four corners.
The transparent board 301 is provided with positioning holes (positioning holes) 321-1 to 321-4 at each of its four corners.
The side pins 221-1 to 221-4 provided in the calibration board 201 and the positioning holes 321-1 to 321-4 provided in the transparent board 301 are provided at positions corresponding to each other. The calibration pins 201 and the transparent board 301 are positioned by passing the side pins 221-1 to 221-4 through the positioning holes 321-1 to 321-4.
Note that the number, position, shape, and the like of the side pins 221-1 to 221-4 of the calibration board 201 and the positioning holes 321-1 to 321-4 of the transparent board 301 are various as long as positioning is possible. Various configurations may be used.

The calibration board 201 has a pattern (checker pattern) 211 in which white squares and black squares are alternately arranged in directions orthogonal to each other on the inner surfaces of the four side pins 221-1 to 221-4. Prepare. The white square and the black square have the same shape.
The white portion and the black portion may have a shape other than a square, for example, or may have different shapes.
As another configuration example, any combination of two different colors other than the combination of white and black may be used.

  The transparent board 301 has a search hole (in FIG. 3, a plurality of probe holes) that form a depression at each of the points (cross points) at which the white square corners and the black square corners of the checker pattern 211 of the calibration board 201 contact each other. A single search hole 311 is provided with a reference numeral).

FIG. 4A is a diagram illustrating a partial configuration example of an example of the calibration jig 62-1 according to the embodiment of the present invention. FIG. 4B is a cross-sectional view (a cross-sectional view taken along the line PP).
The search hole 311 is a hemispherical hole opened on one surface of the transparent board 301. The transparent board 301 and the calibration board 201 are arranged so that the other face of the transparent board 301 (the face opposite to the one face described above) and the face of the calibration board 201 provided with the checker pattern 211 are opposed to each other. Are superimposed.
The shape of the search hole 311 may be other shapes, for example, a conical shape.

Here, in the present embodiment, a jig having a spherical (or hemispherical as another configuration example) portion at the tip of the rod-shaped portion is used as the gripping jig 63 gripped by the hand 21. By using the gripping jig 63 having a spherical tip, it is possible to increase the probability that the hand 21 finds the search hole 311 of the transparent board 301.
Note that the shape of the tip of the gripping jig 63 may be various shapes, and for example, an appropriate shape is used for searching the search hole 311.

Thus, in the present embodiment, the transparent board 301 is overlaid on the upper surface of the calibration board 201 (here, the surface on which the checker pattern 211 is provided), and the upper surface of the transparent board 301 (here, the calibration board 201). And the touch-up point of the checker pattern 211 are matched with each other. In this case, the calibration board 201 and the transparent board 301 are positioned without being displaced by the side pins 221-1 to 221-4 standing on the calibration board 201 and the positioning holes 321-1 provided in the transparent board 301. Is done.
In this embodiment, since the transparent board 301 is used, the calibration board 201 can be imaged from the upper surface side of the transparent board 301 by the camera 12, and when the touch-up point image is image-processed, a search hole is used. The influence of having 311 can be eliminated. When the calibration board 201 is imaged from the upper surface side of the transparent board 301 by the camera 12, both the checker pattern 211 and the search hole 311 can exist without the search hole 311 hiding the touch-up point.

[Another example of calibration jig according to this embodiment]
FIG. 5 is a view showing another example of the calibration jig 62-2 according to the embodiment of the present invention.
The calibration jig 62-2 according to the present embodiment is configured by combining the calibration board 201 and the marker accuracy evaluation board 401.
In the present embodiment, a common calibration board 201 is used for the calibration jig 62-1 shown in FIG. 3 and the calibration jig 62-2 shown in FIG.
Each of the calibration board 201 and the marker accuracy evaluation board 401 has a plate shape having a square (square or rectangular) surface, for example, the same shape.

The marker accuracy evaluation board 401 includes positioning holes (positioning holes) 421-1 to 421-4 at each of the four corners.
The side pins 221-1 to 221-4 provided in the calibration board 201 and the positioning holes 421-1 to 421-4 provided in the marker accuracy evaluation board 401 are provided at positions corresponding to each other. The calibration pins 201 and the marker accuracy evaluation board 401 are positioned by passing the side pins 221-1 to 221-4 through the positioning holes 421-1 to 421-4.
Note that the number, position, shape, etc. of the side pins 221-1 to 221-4 of the calibration board 201 and the positioning holes 421-1 to 421-4 of the marker accuracy evaluation board 401 can be determined as long as they can be positioned. Various configurations may be used.

The marker accuracy evaluation board 401 includes a transparent board 431 and a plurality of markers attached to the transparent board 431 (in FIG. 5, one of the plurality of markers 412 is provided with a reference numeral).
The transparent board 431 has a search hole that forms a depression at each of the points where the white square corner and the black square corner of the checker pattern 211 of the calibration board 201 touch each other (in FIG. The hole 411 is provided with a reference numeral).
The transparent board 431 has a plurality of probe holes 411 provided in respective directions orthogonal to each other, and has a surface opposite to each other of the search holes existing in every other direction (here, the calibration board 201). The marker 412 is provided on the surface facing the surface.

FIG. 6A is a diagram illustrating a partial configuration example of another example of the marker accuracy evaluation board 401 according to an embodiment of the present invention. FIG. 6B is a cross-sectional view thereof (QQ cross-sectional view).
The search hole 411 is a hemispherical hole opened on one surface of the transparent board 431. The marker 412 is provided at a position corresponding to the search hole 411. For example, a predetermined position (reference position) in the surface of the marker 412 and a predetermined position (for example, the center position) of the search hole 411 are provided. It is made to face. In the marker 412, a pattern for image recognition provided in the marker 412 is arranged on the search hole 411 side. The marker 412 is attached to the transparent board 431 by pasting or printing. As this pattern, for example, a pattern in which a predetermined one or more shapes, an arbitrary color, an arbitrary pattern, and the like are combined is used.
The marker accuracy evaluation board 401 and the calibration board 201 are arranged so that the surface on the marker 412 side of the transparent board 431 of the marker accuracy evaluation board 401 and the surface provided with the checker pattern 211 of the calibration board 201 are opposed to each other. Overlaid.
The shape of the search hole 411 may be other shapes, for example, a conical shape.

  Here, in the present embodiment, a jig having a spherical portion at the tip of a rod-shaped portion is used as the gripping jig 63 gripped by the hand 21. By using the gripping jig 63 having a spherical tip, it is possible to increase the probability that the hand 21 finds the search hole 411 of the marker accuracy evaluation board 401.

  In the present embodiment, after the calibration using the calibration jig 62-1 is completed, the transparent board 301 is removed from the calibration board 201, and the marker accuracy evaluation board 401 is overlaid on the calibration board 201. The jig 62-2 is realized, and calibration using the calibration jig 62-2 is performed. The calibration jig 62-2 is created so that the reference point (reference position) of the marker 412 and the touch-up point coincide with each other, and the upper part of the touch-up point of the marker accuracy evaluation board 401 (here, A search hole 411 is provided on the side of the surface not facing the calibration board 201. Thereby, for example, when the position error at the reference point of the marker 412 is large, it is possible to quickly determine the location of which calibration of the touch-up point is a problem.

In this embodiment, the calibration board 201 is shared by the two calibration jigs 62-1 to 62-2. However, as another configuration example, the two calibration jigs 62-1 are used. ~ 62-2, a separate calibration board 201 (e.g., of the same configuration) may be used.
In the present embodiment, the transparent board 301 of the calibration jig 62-1 and the transparent board 431 of the marker accuracy evaluation board 401 of the calibration jig 62-2 are separately configured. However, as another configuration example, These transparent boards 301 and 431 may be shared.

[Calibration using first calibration jig 62-1]
Calibration using a calibration jig 62-1 as the calibration jig 62 shown in FIG. 1 will be described.
In the robot operation processing unit 101, the positioning unit 121 positions the touch-up point in the checker pattern 211 of the calibration board 201 while holding the gripping jig 63 by the hand 21 of the robot 11. In the present embodiment, the positioning unit 121 performs positioning by performing impedance control.

FIG. 7A, FIG. 7B, and FIG. 7C are diagrams showing an example of positioning using impedance control according to an embodiment of the present invention.
As shown in FIG. 7A, the positioning unit 121 grips the gripping jig 63 by the hand 21 of the robot 11 so that a spherical portion exists at the tip of the hand 21. Then, the positioning unit 121 controls the position and posture of the hand 21 so that the spherical portion is in contact with the upper surface of the transparent board 301 (here, the surface where the probe hole 311 is provided). The position of the search hole 311 is searched by performing impedance control based on the value of the force detected (detected) by the force sensor 31. In the present embodiment, the positioning unit 121 controls the hand 21 so that the hand 21 (and the bar-shaped portion of the gripping jig 63) is perpendicular (or substantially perpendicular) to the upper surface of the transparent board 301. .

FIG. 7B shows a state where the spherical portion of the gripping jig 63 is in contact with a part of the search hole 311.
FIG. 7C shows a state where the spherical portion of the gripping jig 63 is accommodated in the search hole 311.
Here, for example, when the positioning unit 121 determines that the spherical portion of the gripping jig 63 gripped by the hand 21 is in a state of being accommodated in the search hole 311, the positioning portion 121 has a spherical shape at the tip of the gripping jig 63. The position of the gripping jig 63 is changed to various postures by controlling the position and posture of the manipulator and the hand 21 while keeping the portion in the search hole 311, so that the spherical shape of the gripping jig 63 is changed. It is possible to increase the accuracy with which the portion fits in the search hole 311.

  Thus, as a result of the impedance control, the spherical portion of the gripping jig 63 (in this embodiment, the central axis of the gripping jig 63) and the touch-up point of the checker pattern 211 coincide with each other. Calibration can be carried out reliably by just searching. As a result, it is possible to suppress reading errors and touch-up point position variations caused by people.

FIG. 8 is a flowchart illustrating an example of a procedure of processing performed in the robot system 1 according to an embodiment of the present invention.
(Step S1)
First, the positioning unit 121 performs impedance control based on the force value (force sensor value) received from the force sensor 31 by the force sensor value receiving unit 103 while holding the gripping jig 63 by the hand 21. Thus, the search hole 311 is positioned. As a result, the positioning unit 121 uses the grasping jig 63 to search for a coordinate system for controlling the manipulator and the hand 21 of the robot 11 (referred to as a robot coordinate system in this embodiment). The coordinate value (for example, the coordinate value of the position or orientation) when the button is pointed (when touched up) is acquired.

(Step S2)
Next, the image processing unit 122 acquires image information received from the camera 12 by the image information receiving unit 102. Then, the image processing unit 122 processes the acquired image information, so that the image processing coordinate system (referred to as a camera coordinate system in the present embodiment) is positioned in the search hole 311 (touch) that has been positioned by the positioning unit 121. The coordinate value (for example, the coordinate value in the image) of the position of the up point) is acquired.
Here, the image processing unit 122 searches and specifies the position of the search hole 311 (touch-up point) based on, for example, the difference between the luminance values of white and black in the checker pattern 211.
As the image information processed by the image processing unit 122, for example, image information captured after the positioning timing by the positioning unit 121 may be used, or captured before the positioning timing by the positioning unit 121. The image information may be used, or the image information captured at the same time (or substantially at the same time) as the positioning timing by the positioning unit 121 may be used.

(Step S3)
Next, the calibration unit 123 performs a calibration process by associating the coordinate value in the robot coordinate system acquired by the positioning unit 121 with the coordinate value in the camera coordinate system acquired by the image processing unit 122. In this calibration process, for example, the calibration unit 123 may associate a coordinate value in the robot coordinate system with a coordinate value in the camera coordinate system for each of a plurality of different search holes 311 (touch-up points). . Further, the calibration unit 123 performs smoothing (for example, derivation of an approximate expression by fitting with a predetermined function) for the association of coordinate values performed for each of a plurality of different search holes 311 (touch-up points). May be performed. In addition, the calibration unit 123 is, for example, a position between adjacent search holes 311 (touch-up points) based on the association of coordinate values performed for each of a plurality of different search holes 311 (touch-up points). In addition, coordinate values may be associated by interpolation for positions where the search holes 311 (touch-up points) do not exist.

Here, in the present embodiment, the hemispherical search hole 311 is used, but as another configuration example, a search hole of another shape may be used.
FIG. 9A, FIG. 9B, and FIG. 9C are diagrams showing another example of positioning using impedance control according to an embodiment of the present invention. In this example, a search hole 511 having a conical shape is used.
9 (A), 9 (B), and 9 (C) show states similar to those shown in FIGS. 7 (A), 7 (B), and 7 (C), respectively. . 9A, FIG. 9B, and FIG. 9C, components other than the search hole 511 and the transparent board 501 are shown in FIG. 7A, FIG. 7B, and FIG. The same reference numerals as in C) are attached.

[Calibration using second calibration jig 62-2]
Calibration using a calibration jig 62-2 as the calibration jig 62 shown in FIG. 1 will be described.
Similarly to the calibration using the calibration jig 62-1, the calibration using the calibration jig 62-2 is performed according to the processing procedure shown in FIG.
(Process corresponding to step S1)
First, the positioning unit 121 performs impedance control based on the force value (force sensor value) received from the force sensor 31 by the force sensor value receiving unit 103 while holding the gripping jig 63 by the hand 21. Thus, the search hole 411 is positioned. Thereby, the positioning unit 121 points to the probe hole 411 (touch-up point) by the gripping jig 63 with respect to the coordinate system (robot coordinate system) for controlling the manipulator and the hand 21 of the robot 11 (touch-up). Coordinate values (for example, coordinate values of position and orientation).

(Process corresponding to step S2)
Next, the image processing unit 122 acquires image information received from the camera 12 by the image information receiving unit 102. Then, the image processing unit 122 processes the acquired image information, so that the image processing coordinate system (camera coordinate system) exists at the position of the search hole 411 (touch-up point) positioned by the positioning unit 121. The coordinate value (for example, the coordinate value in an image) regarding the marker 412 to be acquired is acquired.
Here, the image processing unit 122 recognizes an image based on a pattern that is an object of image recognition provided in the marker 412 and acquires a coordinate value of a predetermined position. In this image recognition, for example, template matching (pattern matching) processing may be used. In this embodiment, color image processing is used for the recognition of the image.

(Process corresponding to step S3)
Next, the calibration unit 123 performs a calibration process by associating the coordinate value in the robot coordinate system acquired by the positioning unit 121 with the coordinate value in the camera coordinate system acquired by the image processing unit 122.

[Calibration using two types of calibration jigs 62-1 to 62-2]
In this embodiment, the robot motion processing unit 101 performs calibration using the second calibration jig 62-2 after performing calibration using the first calibration jig 62-1. Here, if these two types of calibration are both accurate, the results of these two types of calibration match. Further, even if there is an error in these two types of calibration, if the error is within an allowable range, for example, the calibration unit 123 may perform processing that employs one of the two types of calibration results, or You may perform the process etc. which employ | adopt the average result of the result of two types of calibration. In addition, when there is an error in an unacceptable range for these two types of calibration, the robot motion processing unit 101 performs calibration using the first calibration jig 62-1 and the second calibration jig 62-2. A configuration may be used in which the calibration using the redo is performed again.
In the present embodiment, the error in calibration using the first calibration jig 62-1 includes an error in associating the robot coordinate system and the camera coordinate system at each touch-up point, The error in calibration using the second calibration jig 62-2 includes a pattern recognition error of the marker 412 in image processing. In these two types of calibration, for example, different image processing is performed, and errors may be different.

Further, when the calibration using the second calibration jig 62-2 is regarded as the verification of the calibration using the first calibration jig 62-1, the first calibration jig 62-2 1 may be used, and a configuration in which calibration using the second calibration jig 62-2 is not performed may be used.
Further, in the case where calibration using the first calibration jig 62-1 has already been performed, the calibration using the first calibration jig 62-1 is not performed, and the second calibration jig is not used. A configuration in which calibration using the tool 62-2 is performed may be used.

  Here, the calibration using the first calibration jig 62-1 and the calibration using the second calibration jig 62-2 are performed at a plurality of crosspoints (white and black contacts of the checker pattern 211). Of the points provided with the search holes 311), and a configuration performed for four points may be used as an appropriate example.

  In this embodiment, on the surface of the table 61, the surface of the board (the calibration board 201 or the transparent board 301) of the first calibration jig 62-1 and the second calibration jig 62-2. The board (the calibration board 201 and the marker accuracy evaluation board 401) is placed so that its surface is horizontal. As another example of the configuration, the board surface of the first calibration jig 62-1. Alternatively, a configuration in which the board surface of the second calibration jig 62-2 is placed vertically or obliquely may be used.

  Further, in this embodiment, a configuration is used in which the probe holes 311 and 411 are searched for by the spherical portion at the tip of the gripping jig 63 while the gripping jig 63 is gripped by the hand 21. As an example, a configuration in which the search holes 311 and 411 are searched for by the portion of the hand having a spherical or hemispherical portion at the tip may be used.

[Summary of this embodiment]
As described above, in the robot system 1 according to the present embodiment, the coordinate system (the robot coordinate system) of the vision for the robot (the camera 12 for image processing) and the coordinate system (the robot) that controls the manipulator and the hand 21 of the robot 11. In the calibration operation between the imaging unit (in this embodiment, the camera 12) and the robot 11 that calibrates the association with the coordinate system), the robot 21's hand 21 with respect to the calibration jigs 62-1 to 62-2 is corrected. When positioning is performed, an error caused by a person can be reduced (ideally, eliminated). In the robot system 1 according to the present embodiment, the pattern of the marker 412 is obtained by performing calibration using the first calibration jig 62-1 and calibration using the second calibration jig 62-2. Can be evaluated (for example, accuracy of position recognition).

According to the robot system 1 according to the present embodiment, for example, it has been difficult to position the calibration board with respect to the touch-up point of the calibration board, the reference position of the marker, and the like. Regardless, it is possible to accurately perform touch-up work and marker accuracy evaluation.
According to the robot system 1 according to the present embodiment, for example, the alignment with respect to the touch-up point is not based on visual observation, so that reading errors and variations due to the operator can be suppressed, and the alignment (positioning) is performed mechanically. Is possible.

  According to the robot system 1 according to the present embodiment, for example, when the hand 21 is stopped by the control based on the force after the force is detected by the force sensor 31, ) Even when control is performed so as to go too far, gripping is performed from the search holes 311 and 411 by the pressing force in the vertical direction (in this embodiment, the direction perpendicular to the surface of the transparent board 301 or the marker accuracy evaluation board 401). The spherical portion at the tip of the jig 63 can be prevented from being removed, and the center deviation can be eliminated.

According to the robot system 1 according to the present embodiment, for example, the accuracy evaluation result regarding the pattern of the marker 412 can be reflected in calibration. Thereby, according to the robot system 1 which concerns on this embodiment, it can be useful for the analysis of a factor about whether a cause which produces an error is a mechanical factor or a soft factor, for example, and a problem (factor) Efficiency up to the specific can be achieved.
According to the robot system 1 according to the present embodiment, for example, an operation of moving the hand 21 between touch-up points and an operation of positioning the hand 21 with respect to the touch-up points are performed by an apparatus (in this embodiment, for example, a control unit 41) can be automated.
As described above, according to the robot system 1 according to the present embodiment, calibration of the imaging unit (the camera 12 in the present embodiment) and the robot 11 can be performed with high accuracy.

Here, with reference to FIG. 12 (A) and FIG. 12 (B), the effect acquired by the robot system 1 which concerns on this embodiment is demonstrated concretely.
FIG. 12A is a diagram illustrating positioning with respect to the touch-up point 1012 according to the comparative example.
FIG. 12A shows a case where a person visually positions the hand 1051 with respect to the touch-up point 1012 of the checker pattern 1011 on the calibration board 1001.
Although an error occurs visually by human eyes as in this comparative example, such an error can be reduced (ideally eliminated) by the robot system 1 according to the present embodiment.

FIG. 12B is a diagram illustrating image recognition using the marker 1101 according to the comparative example.
In FIG. 12B, after the positioning shown in FIG. 12A is performed, the marker 1101 (in the example of FIG. 12B, one marker 1101 out of a plurality of markers 1101 to 1103). A case is shown in which image recognition is performed and positioning is evaluated.
In this comparative example, the above-described human visual error is evaluated. However, according to the robot system 1 according to the present embodiment, such an error can be reduced (ideally eliminated).

  As described above, in the comparative example shown in FIGS. 12A and 12B, the hand 1051 and the touch-up point 1012 are visually aligned, whereas in the robot system 1 according to the present embodiment, The probe hole 311 or 411 is provided at a position corresponding to the touch-up point, and the probe 21 is searched for by the hand 21 (in this embodiment, the holding jig 63 held by the hand 21). By aligning the position 21 and the touch-up point, calibration errors between the imaging unit (in this embodiment, the camera 12) and the robot 11 can be reduced.

[Outline of Robot System According to Modified Example of Embodiment]
<First Modification>
FIG. 10 is a diagram showing a schematic configuration of a robot 601 according to the first modification of the embodiment of the present invention.
The robot 601 includes a storage unit 611, a body portion 612 to 614, a left manipulator 621, a hand 631 and a force sensor 641, a right manipulator 622, a hand 632 and a force sensor 642, a left wheel 651, and a right wheel 651. A wheel 652, a camera 661, and a control unit 671 are provided.

  The robot 601 is a dual-arm robot including two manipulators 621 and 622 that respectively constitute arms (arms). The robot 601 is configured integrally with the camera 661 and the control unit 671, and its operation is controlled by a control signal input from the control unit 671. Further, the robot 601 may output a signal indicating its own state or the like to the control unit 671.

A body member 612, a body member 613, and a body member 614 are sequentially attached to the upper surface of the storage unit 611, and a manipulator 621 constituting a left arm is attached to the left side of the uppermost body member 614. A manipulator 622 constituting a right arm is attached to the right side of the body member 614, a wheel 651 is attached to the left side of the bottom surface of the storage unit 611, and a wheel 652 is attached to the right side of the bottom surface of the storage unit 611. A camera 661 is attached to the upper surface of the storage unit 611.
The robot 601 can be moved by rotating the left wheel 651 and the right wheel 652 by applying external force manually or automatically by the apparatus.
The storage unit 611 stores a control unit 671.
Each of the manipulators 621 and 622 is a kind of vertical articulated robot, for example, and functions as a robot arm. Each manipulator 621 and 622 includes a hand 631 and 632 at its tip. The hands 631 and 632 are provided with force sensors 641 and 642, respectively.

In the example of FIG. 10, the camera 661 is provided on the upper surface of the storage unit 611. However, as another configuration example, the camera 661 may be provided in any other location. It may be provided in an arbitrary place, or may be provided as a place other than the robot by being fixed to a floor surface, a ceiling, a wall surface or the like.
Further, the camera 661 may be configured to be able to change the imaging direction, the imaging angle, and the like by applying an external force manually, or automatically by the apparatus.

The control unit 671 controls the left arm manipulator 621 (and the hand 631 and the force sensor 641) and the right arm manipulator 622 (and the hand 632 and the force sensor 642). The control unit 671 and each of the manipulators 621 and 631 are connected to be able to transmit a control signal or the like via, for example, a wired or wireless line.
The control unit 671 may control the left arm manipulator 621 (and the hand 631 and the force sensor 641) and the right arm manipulator 622 (and the hand 632 and the force sensor 642) in association with each other at the same time, or The left arm manipulator 621 (and hand 631 and force sensor 641) and the right arm manipulator 622 (and hand 632 and force sensor 642) may be controlled separately.

  Here, the left arm manipulator 621 (and the hand 631 and force sensor 641) and the right arm manipulator 622 (and the hand 632 and force sensor 642) included in the robot 601 shown in FIG. 10 are provided in the robot 11 shown in FIG. It corresponds to the manipulator (and the hand 21 and the force sensor 31). Further, the camera 661 provided in the robot 601 shown in FIG. 10 corresponds to the camera 12 shown in FIG. Further, the control unit 671 provided in the robot 601 shown in FIG. 10 corresponds to the control unit 41 provided in the robot 11 shown in FIG.

<Second Modification>
FIG. 11 is a diagram showing a schematic configuration of a robot system 701 according to a second modification of the embodiment of the present invention.
The robot system 701 connects the robot 711, the camera 712, the control device 713, connects the camera 712 and the control device 713 to enable communication, and connects the robot 711 and the control device 713 to enable communication. A line 715 is provided. The robot 711 is a single-arm robot including one manipulator (arm), and includes a hand (robot hand) 721 attached to the tip of the manipulator. The hand 721 is provided with a force sensor 731. The control device 713 has the same function as the control unit 41 shown in FIG.
FIG. 11 shows a calibration jig 62 placed (arranged) on a table (work desk) 61 and a gripping jig 63 gripped by a hand 721. The table 61, the calibration jig 62, and the gripping jig 63 are the same as those shown in FIG. 1, and the same reference numerals as those in FIG. 1 are used.

  Here, compared with the robot system 1 shown in FIG. 1, in the robot system 701 shown in FIG. 11, the robot 711 and the control device 713 (corresponding to the control unit 41 shown in FIG. 1) are separated. It is configured. The control device 713 has a function of communicating with the camera 712 via the line 714 and a function of communicating with the robot 711 via the line 715.

As in the example of FIG. 11, even in a robot system 701 in which the robot 711 and the control device 713 are separately formed, a robot system (for example, the robot system 1 shown in FIG. An effect similar to that of the robot system including the robot 601 shown in FIG. 10 can be obtained.
As for another example of the configuration of the robot 601 illustrated in FIG. 10, a portion other than the control unit 671 of the robot 601 and the control unit 671 may be configured separately.

[Configuration example of the above embodiment]
As one configuration example, a hand (the hand 21 in the example of FIG. 1 and the hands 631 to 632 in the example of FIG. 10) and a force sensor (the force sensor 31 in the example of FIG. 1) and the example of FIG. Then, force sensors 641 to 642) and a control unit (the control unit 41 in the example of FIG. 1 and the control unit 671 in the example of FIG. 10) that operates the hand, and the control unit includes: A predetermined position of a calibration jig (the calibration jig 62 in the example of FIG. 1, for example, the calibration jig 62-1 shown in FIG. 3 or the calibration jig 62-2 shown in FIG. 5). After positioning the hand by a depression (in the example of FIGS. 3 and 5, the search holes 311 and 411 in the examples of FIGS. 3 and 5), the imaging unit (example of FIG. 1) is positioned. In the example of FIG. 10, the camera 661) is the predetermined camera. Imaging the location (in the example of FIG. 1, a robot 11, in the example of FIG. 10, the robot 601) a robot is.
The calibration jig is, for example, a jig for calibrating the imaging unit and the robot, and the control unit is based on, for example, the result of hand positioning and the result of imaging a predetermined position. And calibrate.

As one configuration example, in the robot, the control unit performs impedance control based on a value detected by the force sensor, and positions the hand.
As an example of the configuration, in the robot, the control unit uses the hand to have a shape corresponding to the shape of the depression (semispherical in the examples of FIGS. 3 and 5) (spherical in the examples corresponding to FIGS. 3 and 5). Alternatively, the hand is positioned in a state where a gripping jig having a hemispherical shape (the gripping jig 63 in the example of FIG. 1) is gripped.
As one configuration example, in the robot, the calibration jig includes a board (a calibration board 201 in the example of FIG. 3) having a checker pattern (in the example of FIG. 3) and a transparent having the recess. It is a jig combined with a board (transparent board 301 in the example of FIG. 3).
As one configuration example, in the robot, the calibration jig includes a board (a calibration board 201 in the example of FIG. 5) having a checker pattern (in the example of FIG. 5) and a transparent having the recess. A jig in which a board (a marker accuracy evaluation board 401 in the example of FIG. 5) is combined with a board (a transparent board 431 in the example of FIG. 5) with a marker (a marker 412 in the example of FIG. 5). is there.
As an example of the configuration, in the robot, positioning protrusions (side pins 221-1 to 221-4 in the examples of FIGS. 3 and 5) are provided on one of the two boards to which the calibration jig is combined. Positioning holes are provided on the other side (positioning holes 321-1 to 321-4, 421-1 to 421-4 in the examples of FIGS. 3 and 5) through which the protrusions are passed.

As one configuration example, the imaging unit (the camera 12 in the example of FIG. 1, the camera 661 in the example of FIG. 10, the camera 712 in the example of FIG. 11), and the hand (the hand in the example of FIG. 1). 10, the hands 631 to 632 in the example of FIG. 10, the hand 721 in the example of FIG. 11, and the force sensor (the force sensor 31 in the example of FIG. 1, and the force sensor in the example of FIG. 11, and in the example of FIG. 11, the robot is the robot 11 in the example of FIG. 11. In the example of FIG. 11, the robot is the robot 601. 711) and a control unit for operating the robot (in the example of FIG. 1, the control unit 41, in the example of FIG. 10, the control unit 671, in the example of FIG. 11, the function of the control device 713), Calibration jig The control unit causes the imaging unit to image the predetermined position after positioning the hand by a depression disposed at a predetermined position of the calibration jig, or causes the imaging unit to capture the predetermined position before the positioning. A robot system to be imaged (the robot system 1 in the example of FIG. 1, the robot system including the robot 601 in the example of FIG. 10, and the robot system 701 in the example of FIG. 11).
As one configuration example, an image receiving unit (a function of the image information receiving unit 102 in the example of FIG. 2) that receives a captured image captured by the imaging unit, and a processing unit that operates the robot (the robot operation process in the example of FIG. 2). The control unit is configured to image the predetermined position after the processing unit has positioned the hand of the robot by the depression disposed at the predetermined position of the calibration jig. (In the example of FIG. 11, the control device 713).
As one configuration example, a control method (for example, FIG. 1 or FIG. 1) including positioning a robot hand by a depression disposed at a predetermined position of a calibration jig and imaging the predetermined position by an imaging unit. 10 is a control method performed by the control units 41 and 671 shown in FIG. 10 and the control device 713 shown in FIG.

[About the above embodiments]
The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  It should be noted that a program for realizing the function of an arbitrary component (functional unit) in the devices described above (for example, the control units 41 and 671 and the control device 713) is recorded on a computer-readable recording medium. The program may be loaded into a computer system and executed. Here, the “computer system” includes hardware such as an OS (Operating System) and peripheral devices. “Computer-readable recording medium” means a portable disk such as a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a CD (Compact Disk) -ROM, or a hard disk built in a computer system. Refers to the device. Further, the “computer-readable recording medium” means a volatile memory (RAM: Random Access) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Memory that holds a program for a certain period of time, such as Memory).

In addition, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Further, the above program may be for realizing a part of the functions described above. Further, the program may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

DESCRIPTION OF SYMBOLS 1,701 ... Robot system, 11, 601, 711 ... Robot, 12, 661, 712 ... Camera, 13, 714-715 ... Line, 21, 631-632, 721, 1051 ... Hand, 31, 641-642, 731 ... force sensor, 41, 671 ... control unit, 61 ... table, 62, 62-1 to 62-2 ... calibration jig, 63 ... gripping jig, 101 ... robot motion processing unit, 102 ... image information receiving unit , 103 ... force sensor value receiving unit, 104 ... storage unit, 105 ... input unit, 106 ... output unit, 121 ... positioning unit, 122 ... image processing unit, 123 ... calibration unit, 201, 1001 ... calibration board, 211, 1011: checker pattern, 221-1 to 221-4 ... side pins, 301, 431, 501 ... transparent board, 311, 411, 511 ... Holes, 321-1 to 321-4, 421-1 to 421-4 ... Positioning holes, 401 ... Marker accuracy evaluation board, 412, 1101 to 1103 ... Marker, 611 ... Storage part, 612 to 614 ... Body part, 621 -622 ... Manipulator, 651-652 ... Wheel, 713 ... Control device, 1012 ... Touch-up point

Claims (9)

Hands and
A force sensor;
A control unit for operating the hand,
The controller is configured to image the predetermined position after the hand is positioned by a depression disposed at a predetermined position of the calibration jig,
robot. The control unit performs impedance control based on a value detected by the force sensor, and positions the hand.
The robot according to claim 1. The control unit positions the hand in a state where a gripping jig having a shape corresponding to the shape of the depression is gripped by the hand.
The robot according to claim 1 or 2. The calibration jig is a jig in which a board having a checker pattern and a transparent board having the depression are combined.
The robot according to any one of claims 1 to 3. The calibration jig is a jig in which a board having a checker pattern and a board with markers attached to the transparent board having the depressions are combined.
The robot according to any one of claims 1 to 3. A positioning projection is provided on one of the two boards combined with the calibration jig, and a positioning hole through which the projection is passed is provided on the other.
The robot according to any one of claims 4 and 5. An imaging unit;
A robot with a hand and a force sensor;
A control unit for operating the robot;
A calibration jig, and
The control unit causes the imaging unit to take an image of the predetermined position after positioning the hand by a depression disposed at a predetermined position of the calibration jig, or the imaging unit takes the predetermined position before the positioning. Let me image
Robot system. An image receiving unit that receives a captured image captured by the imaging unit;
A processing unit for operating the robot,
The processing unit positions the hand of the robot by a depression disposed at a predetermined position of a calibration jig, and then the imaging unit images the predetermined position.
Control device. Positioning the robot's hand by a recess placed at a predetermined position of the calibration jig;
Imaging the predetermined position by an imaging unit;
Control method.

Images