JP2016097474A - Robot and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate calibration of a coordinate system of an imaging section and a coordinate system of a robot.SOLUTION: A robot comprises: an instruction reception section receiving a calibration start instruction; and an arm changing a positional relation between a marker indicating a reference point and an imaging section when the calibration start instruction is received. Calibration of a coordinate system of the imaging section and a coordinate system of the robot is performed on the basis of an image of the marker imaged by the imaging section after the positional relation between the imaging section and the marker is changed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a robot and a robot system.

Conventionally, in order to control a robot using robot vision, it is necessary to calibrate the robot coordinate system and the coordinate system of the imaging unit. At the time of calibration, the position of the reference point in the robot installation space is specified by each of the robot coordinate system and the coordinate system of the imaging unit, and the position of the reference point expressed by one coordinate system is expressed by the other coordinate system. A determinant for conversion to the position of the reference point is obtained. According to the technique described in Patent Document 1, the reference point is taught in the robot coordinate system by moving the arm so as to touch the three reference points, and then the reference point is moved by the imaging unit moved to a predetermined position by the arm. Calibration is performed by imaging the marker to be shown and detecting the reference point in the coordinate system of the imaging unit.

JP-A-8-210816

According to the technique described in Patent Document 1, it is necessary to teach the position of the reference point by the operator operating the arm so as to touch the three reference points. However, it is not easy to accurately operate the arm while visually identifying the boundary state whether the arm touches or does not touch the reference point. That is, the technique described in Patent Document 1 has a problem that it is not easy to accurately teach the position of the reference point. If calibration is performed while accurately teaching the positions of a plurality of reference points, there is a problem that the time required for calibration is prolonged, and this problem becomes more serious as the number of robots to be calibrated increases.

The present invention was created to solve these problems, and an object thereof is to facilitate calibration of the coordinate system of the imaging unit and the coordinate system of the robot.

A robot for achieving the above object includes an instruction receiving unit that receives a calibration start instruction, and an arm that changes a positional relationship between a marker indicating a reference point and an imaging unit when the calibration start instruction is received, Calibration of the coordinate system of the imaging unit and the coordinate system of the robot is performed based on an image obtained by imaging the marker by the imaging unit after the positional relationship between the imaging unit and the marker changes.

In addition, the robot for achieving the above object has a positional relationship between a marker indicating three or more reference points and an imaging unit, and a plane including the three or more reference points and an optical axis of the imaging unit are perpendicular to each other. A first state, a second state in which the imaging unit is focused on the marker indicating at least one of the three or more reference points, and a third state in which the optical axis of the imaging unit passes the first point on the plane The optical axis of the imaging unit is automatically changed to at least one of a fourth state passing through the second point on the plane and a fifth state where the optical axis of the imaging unit passes through the third point on the plane. An arm is provided, and the coordinate system of the imaging unit and the coordinate system of the robot are calibrated based on images obtained by imaging the marker by the imaging unit in the third state, the fourth state, and the fifth state. .

According to these inventions, the positional relationship between the imaging unit and the marker can be changed by a calibration start instruction so that an image necessary for calibrating the coordinate system of the imaging unit and the coordinate system of the robot can be captured. Therefore, it is easy to calibrate the coordinate system of the imaging unit and the coordinate system of the robot.

Here, the calibration start instruction is a timing at which the arm starts the operation of changing the positional relationship between the imaging unit and the marker so that an image necessary for calibrating the coordinate system of the imaging unit and the coordinate system of the robot can be captured. The instruction may be an instruction to be specified, and may not include a target value that specifies the amount of change in the position or posture of the arm. Moreover, the marker should just point out the reference point of a robot coordinate system. For example, a two-dimensional figure such as a point or a circle written on a flat plate can be used as a marker, or an object having a three-dimensional shape (for example, the tip of an arm) can be used as a marker. And a marker may be provided in an arm, and may be provided in the thing (for example, work bench) fixed to the space where a robot is installed. When the marker is provided on the arm (including the case where the arm becomes the marker), the imaging unit is fixed to the robot coordinate system. When the marker is fixed in the robot coordinate system, the imaging unit is provided on the arm.

Note that the function of each means described in the claims is realized by hardware resources whose function is specified by the configuration itself, hardware resources whose function is specified by a program, or a combination thereof. The functions of these means are not limited to those realized by hardware resources that are physically independent of each other.

FIG. 1A is a schematic perspective view according to an embodiment of the present invention. FIG. 1B is a block diagram according to an embodiment of the present invention. The top view concerning embodiment of this invention. The typical perspective view concerning the embodiment of the present invention. The flowchart concerning embodiment of this invention. The flowchart concerning embodiment of this invention. The flowchart concerning embodiment of this invention. The schematic diagram concerning embodiment of this invention. The figure which shows the image concerning embodiment of this invention. The flowchart concerning embodiment of this invention. The typical perspective view concerning the embodiment of the present invention. The flowchart concerning embodiment of this invention. The typical perspective view concerning the embodiment of the present invention. The typical top view concerning the embodiment of the present invention. The schematic diagram concerning embodiment of this invention. The schematic diagram concerning embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, the same code | symbol is attached | subjected to the corresponding component in each figure, and the overlapping description is abbreviate | omitted.
1. First Example 1-1. Outline As shown in FIG. 1, a robot system according to a first embodiment of the present invention includes a robot 1,
An imaging unit 2 and a PC (Personal Computer) 3 as a calibration device are provided.

The robot 1 is a six-axis robot including six rotary shaft members 121, 122, 123, 124, 125, and 126 on an arm. The center of the tip of the rotating shaft member 126 on which various tools for operating the workpiece are mounted is referred to as a tool center point (TCP). The position and orientation of the TCP is a reference for the position and orientation of various tools. The coordinate system of the robot 1 used when controlling the robot 1 is a horizontal x-axis and a y-axis, respectively, and z with a vertical downward direction as a positive direction.
This is a three-dimensional orthogonal coordinate system determined by the axis. Further, the rotation around the z axis is represented by u, the rotation around the y axis is represented by v, and the rotation around the x axis is represented by w. The unit of length of the robot coordinate system is millimeter and the unit of angle is degree.

The imaging unit 2 has a workpiece size in a reference plane perpendicular to the optical axis of the lens 201,
It is an imaging part for recognizing a shape and a position. The coordinate system of the imaging unit 2 is a coordinate system of an image output from the imaging unit 2, and is determined by a B-axis having a horizontal right direction of the image as a positive direction and a C-axis having a vertical downward direction of the image as a positive direction. . The unit of length of the coordinate system of the imaging unit 2 is pixel, and the unit of angle is degree. The coordinate system of the imaging unit 2 is a plane coordinate system in real space perpendicular to the optical axis of the imaging unit 2, and the optical characteristics (focal length, distortion, etc.) of the lens 201 and the area image sensor 20.
2 is a two-dimensional orthogonal coordinate system that is nonlinearly converted according to the number and size of pixels. Therefore, in order to recognize the size, shape, or position of the workpiece based on the image output from the imaging unit 2 and control the robot 1 based on the recognition result, the coordinate system of the imaging unit 2 is set to the coordinate of the robot 1. A process relating to the system, that is, calibration is required.

The PC 3 is connected to the robot 1 and the imaging unit 2. A calibration program for calibrating the coordinate system of the robot 1 and the coordinate system of the imaging unit 2 is installed in the PC 3. The operator activates a calibration program to execute calibration, and inputs a calibration start instruction to the PC 3. The calibration start instruction is merely a trigger for the calibration program to start the calibration process, and does not include a target value for operating the robot 1. In order to calibrate the coordinate system of the image pickup unit 2, conventionally, the optical axis of the image pickup unit 2 is set to be in a state of being accurately perpendicular to a plane including three or more reference points, or touched with TCP. It was necessary to make precise preparations such as teaching three or more reference points accurately by uploading. And if these preparations are inaccurate, the calibration will fail, and the operator knows the calibration failure,
Know that your preparation was inaccurate. Therefore, in the past, a great deal of time was required for calibration.

According to an embodiment of the present invention described below, calibration preparation is automatically performed using a robot vision (image captured by the imaging unit 2) in a state where calibration is not performed. In preparation for calibration performed automatically, the optical axis of the imaging unit 2 is perpendicular to a reference plane including three or more reference points, and the imaging unit 2 is in focus on the reference plane. Includes operations. For this reason, it is very easy to calibrate the coordinate system of the robot 1 and the coordinate system of the imaging unit 2 only by inputting a calibration start instruction in a state where the exact positional relationship between the reference point and the imaging unit is not fixed. Can do.

1-2. Configuration As shown in simplified form in FIG. 1A, the robot 1 includes a base 110 and arms 111, 112,
113, 114, and 115. The base 110 supports the rotating shaft member 121 of the first arm 111. The first arm 111 rotates with respect to the base 110 together with the rotary shaft member 121 around the central axis of the rotary shaft member 121. The first arm 111 is the second arm 1
Twelve rotating shaft members 122 are supported. The second arm 112 rotates with respect to the first arm 111 together with the rotary shaft member 122 about the central axis of the rotary shaft member 122. The second arm 112 supports the rotating shaft member 123 of the third arm 113. The third arm 113 rotates with respect to the second arm 112 together with the rotation shaft member 123 around the central axis of the rotation shaft member 123. The third arm 113 supports the rotating shaft member 124 of the fourth arm 114. The fourth arm 114 rotates with respect to the third arm 113 together with the rotation shaft member 124 about the central axis of the rotation shaft member 124. The fourth arm 114 supports the rotating shaft member 125 of the fifth arm 115. The fifth arm 115 rotates with respect to the fourth arm 114 together with the rotation shaft member 125 about the central axis of the rotation shaft member 125. Fifth arm 115
Supports the rotating shaft member 126. Rotating shaft member 12 at the tip of the manipulator
6 is provided with a tool chuck 1261 whose tool mounting surface is shown in FIG.
Various tools for operating the workpiece are attached to the tool chuck 1261. As shown in FIG. 2, the attachment surface of the tool chuck 1261 is divided into four parts, and the tool shaft is inserted into the central part thereof. The center of the mounting surface of the tool chuck 1261 corresponds to TCP.

As shown in FIG. 1B, the robot 1 includes a motor 131 that drives the rotary shaft member 121,
A motor 132 that drives the rotating shaft member 122 and a motor 1 that drives the rotating shaft member 123.
33, a motor 134 for driving the rotary shaft member 124, a motor 135 for driving the rotary shaft member 125, a motor 136 for driving the rotary shaft member 126, and motors 131-13.
6 is provided. Motors 131 to 136 are arms 111 to 1
15 components. The motors 131 to 136 are servo motors that are feedback controlled so that the difference between the target value and the current value becomes zero. The control unit 14 acquires a target value indicating the position and orientation of the TCP from the PC 3, and derives target values of the motors 131 to 136 based on the target value indicating the position and orientation of the TCP.

The imaging unit 2 is a digital imaging unit including a lens 201, an area image sensor 202, an AD converter (not shown), and the like. As shown in FIG. 1A, the imaging unit 2 is provided at a predetermined position on the table 9 on which a workpiece is placed so as to be able to capture an image vertically upward. Note that the robot vision calibrated in the present embodiment is a two-dimensional robot vision for a predetermined reference plane. For this reason, the lens 201 has a shallow depth of field (short focal length).
It is preferable to use a single focus lens having a small F value.

The PC 3 includes a processor (not shown), a main memory (not shown) including a DRAM, an input / output mechanism (not shown), an external storage (not shown) including a nonvolatile memory, a display, and an instruction receiving unit 30.
It is a computer equipped with a keyboard that functions as a computer. The PC 3 functions as an instruction receiving unit 30, an image acquisition unit 31, a target value deriving unit 32, an output unit 33, and a calibration unit 34 by executing a calibration program stored in an external storage by a processor.

The image acquisition unit 31 instructs the imaging unit 2 to perform imaging, and acquires an image in which a marker indicating a reference point is captured from the imaging unit 2 in accordance with the instruction. As shown in FIG. 3, the reference points used in the present embodiment are nine lattice points P1, P2, P on a plane perpendicular to the rotary shaft member 126 at an arbitrary position and posture.
3, P4, P5, P6, P7, P8, and P9. In this embodiment, a plane including the reference points P1 to P9 determined with respect to the rotary shaft member 126 having an arbitrary position and posture is referred to as a reference plane. Reference point P1
The relative positional relationship between P9 to P9 is fixed in the robot coordinate system, but the reference points P1 to P9
Moves within the fixed visual field of the imaging unit 2 by the movement of the rotary shaft member 126 during execution of automatic calibration. That is, in the calibration process, the position and orientation of the rotating shaft member 126 (x, y, z, u, v
, W), the X axis and the Y axis that define the reference plane are determined, and the coordinates of the reference point are determined by the X axis coordinate and the Y axis coordinate. The X axis and the Y axis are perpendicular to the central axis of the rotating shaft member 126 and perpendicular to each other, and the TCP when defining the reference plane is the origin of the reference plane. The reference plane at the time of final calibration is the target of robot vision. Since the reference plane is a coordinate system fixed to the rotating shaft member 126, the relationship between the reference plane coordinate system and the robot coordinate system is linear. In the present embodiment, the mounting surface of the tool chuck 1261 shown in FIG. 2 is used as a marker indicating the reference points P1 to P9.

The target value deriving unit 32 holds a template of the mounting surface of the tool chuck 1261. The template of the mounting surface of the tool chuck 1261 is used for analyzing an image acquired from the imaging unit 2 and detecting the coordinates of the reference point in the coordinate system of the imaging unit 2. The target value deriving unit 32
A target value for changing the positional relationship between the imaging unit 2 and the reference points P1 to P9 to a predetermined state is derived based on an image captured by the imaging unit 2. Specifically, the target value deriving unit 32 derives the positions and orientations of the reference points P1 to P9 corresponding to a predetermined state as TCP target values based on the image captured by the imaging unit 2. Here, the predetermined state is the following state.
・ Vertical state: A state in which the optical axis of the reference plane is perpendicular to the imaging unit 2 ・ In-focus state: A state in which the imaging unit 2 is focused on the reference plane

The vertical state includes a low-precision vertical state determined by a target value derived based on an image captured in a state where the reference plane is not focused, and a high-accuracy vertical state determined by a threshold value of an “inclination index” described later. included. The in-focus state is determined by the sharpness of the image, which is an in-focus index. In this embodiment, the state where the sharpness of a predetermined region (partial image) of the image is maximized is set to the in-focus state. However, by providing a threshold for the in-focus index, a certain range is set in the in-focus state. You may have it.

The output unit 33 outputs the target value derived by the target value deriving unit 32 to the control unit 14 of the robot 1.
To change the positional relationship between the imaging unit 2 and the reference points P1 to P9 between the vertical state and the in-focus state.

1-3. Automatic calibration 1-3-1. Overall Flow Next, the overall flow of automatic calibration using the robot system described above will be described with reference to FIG.

The automatic calibration by the robot system is started when the operator inputs a calibration start instruction to the PC 3, and thereafter is completed without requiring any operation from the operator. The operator is required to input the environment information of the robot system to the PC 3 before inputting the calibration start instruction, and the state where the optical axis of the imaging unit 2 passes through the rotary shaft member 126 at the tip of the manipulator by so-called jog feed. It is only to move TCP. That is, no precise operation is required for the operator before inputting the calibration start instruction. The environmental information includes the number and size of pixels of the area image sensor 202 and the focal length of the lens 201. In this embodiment, the mounting surface of the tool chuck 1261 itself is used as a marker indicating the reference point. However, a plate or the like on which the marker indicating the reference point is represented is used as the tool chuck 12.
When attaching to 61, it is necessary to input the relative position and orientation of the marker with respect to TCP.

When a calibration start instruction is input to the PC 3 (S1), the PC 3 executes low-precision tilt correction (
S2). In the low-accuracy tilt correction, a reference plane corresponding to the TCP position and orientation at the time when the calibration start instruction is input is set. Subsequently, TCP is set to nine reference points on the set reference plane.
In this state, the imaging unit 2 performs imaging, and the TCP positions indicating the nine reference points are detected in the coordinate system of the imaging unit 2. Based on the detection result, a target value for deriving the reference plane and the optical axis of the imaging unit 2 in a vertical state (vertical state) is derived. The derived target value is the robot 1
When the arms 111 to 115 move based on the target value, the reference plane and the optical axis of the imaging unit 2 are in a low-precision vertical state in which the optical axis is approximately vertical.

Next, the PC 3 performs focus adjustment (S3). In the focus adjustment, imaging is performed by the imaging unit 2 while moving the TCP in a direction perpendicular to the reference plane in the low-precision vertical state, and a focused state in which the sharpness of the tool chuck 1261 in the image is maximized is searched. The Subsequently, a target value for achieving the in-focus state is derived based on the search result. When the derived target value is input to the robot 1 and the arms 111 to 115 move based on the target value, the focused state is obtained.

Next, the PC 3 executes high-precision tilt correction (S4). In the high-precision tilt correction, the imaging unit 2 performs imaging while tilting the reference plane in the low-precision vertical state and the focused state by a minute angle. An inclination index of the reference plane is derived based on a plurality of images obtained by imaging the tool chuck 1261 as a marker indicating a plurality of reference points in the same reference plane. The inclination index of the reference plane is a value having a positive correlation with the difference between the angle formed by the optical axis of the imaging unit 2 and the reference plane and 90 degrees. Based on the derived inclination index, a target value for obtaining a vertical state is derived. When the derived target value is input to the robot 1 and the arms 111 to 115 move based on the target value, a highly accurate vertical state is obtained.

Next, the PC 3 executes nine-point calibration (S5). In the nine-point calibration, the coordinate system of the robot coordinate system and the imaging unit 2 is based on the coordinates of the coordinate system of the reference plane of the nine reference points belonging to the reference plane in the high-precision vertical state and the coordinates of the coordinate system of the imaging unit 2. A final conversion formula for calibrating is derived.

As described above, the PC3 as a calibration device is a low-precision tilt correction, focus adjustment,
By sequentially performing high-precision tilt correction, the robot 1 is automatically controlled so that the reference plane is perpendicular to the optical axis of the imaging unit 2 and the imaging unit 2 is in focus with respect to the reference plane. After setting the reference plane that is the target of the robot vision, nine-point calibration is executed for the set reference plane.

1-3-2. Low-precision tilt correction Next, details of the low-precision tilt correction will be described with reference to FIG.
First, the target value deriving unit 32 sets a reference plane, and derives a target value for moving TCP as a marker to nine reference points belonging to the set reference plane. Subsequently, the output unit 33
Outputs the target value to the control unit 14 of the robot 1, and the image acquisition unit 31 acquires nine images captured in a state where the TCP is located at each reference point from the imaging unit 2 (S21). The reference point that is the movement destination of the TCP is derived based on the position and posture (the position and posture of the TCP) of the tool chuck 1261 at the time when the calibration start instruction is input. Specifically, for example, the central reference point P5 is set to the TCP position when the calibration start instruction is input, and the plane perpendicular to the rotating shaft member 126 at the time when the calibration start instruction is input is the reference plane. The other reference points are derived as follows. The output unit 33 determines the coordinates of the nine reference points thus derived in the robot coordinate system as the control unit 1.
4 is output as a target value.

Here, nine images picked up by the image pickup unit 2 will be described with reference to FIGS.
FIG. 7 schematically shows the position and orientation of the reference plane F (n) with respect to the imaging unit 2. F (0
) Is a reference plane imaged in step S21. When an image is captured every time the TCP as a marker is moved to the nine reference points P1 to P9, the position of the tool chuck 1261 in each image acquired from the imaging unit 2 varies depending on the positions of the reference points P1 to P9. . When the tool chucks 1261 appearing in each of the nine images are collectively represented as one image, for example, as shown in FIG. 8A, the tool chucks 1261 appear at non-grid points. In FIG. 8, the degree of focusing is not reflected. The robot is moved so that the marker indicates each reference point set in the same plane in the robot coordinate system, and the marker at each reference point is obtained from n images captured by the imaging unit fixed in the robot coordinate system. The reference point information
A board with a marker indicating n reference points is held at a specific position and posture by a robot,
In this state, the reference point information obtained from one image obtained by imaging the marker attached to the plate is almost the same. The former information reflects the correspondence between the exact robot coordinate system and the coordinate system of the imaging unit, and the latter information captures the coordinate system and the image obtained by combining the robot coordinate system and the coordinate system of the plate with the markers. The correspondence of the coordinate system of the part is reflected. Since the robot coordinate system and the coordinate system of the plate with the marker are not necessarily in a linear relationship, it is preferable to use the former information in order to calibrate the robot coordinate system and the coordinate system of the imaging unit strictly. .

Subsequently, the target value deriving unit 32 analyzes nine images captured with the TCP positioned at each reference point, and detects the center position of the mounting surface of the tool chuck 1261 in each image (S2).
2). That is, the target value deriving unit 32 detects the positions of nine reference points in the coordinate system of the imaging unit 2.

Subsequently, the target value deriving unit 32 determines whether the coordinate system of the imaging unit 2 and the robot coordinate system are based on the coordinates of the reference point in the coordinate system of the reference plane and the coordinates of the reference point in the coordinate system of the imaging unit 2. A correspondence relationship is derived (S23). Specifically, a transformation matrix for transforming the coordinate system of the imaging unit 2 into the robot coordinate system is derived. Obtaining such a transformation matrix is nothing but calibrating the coordinate system of the imaging unit 2 and the robot coordinate system with respect to the reference plane set in step S21. However, since the transformation matrix derived here is derived based on an image captured in a state where the imaging unit 2 is not focused on the reference points P1 to P9, the transformation matrix is calculated with respect to the optical axis of the imaging unit 2. It does not exactly correspond to the orientation of the reference plane or the positions of the reference points P1 to P9.

Subsequently, the target value deriving unit 32, based on the transformation matrix derived in step S23, the imaging unit 2
A target value for deriving the reference plane to a position and orientation in which the reference plane is perpendicular to the optical axis of the image pickup unit 2 and the optical axis of the imaging unit 2 passes through the central reference point P5 is derived. The derived target value is output from the output unit 3
3 is output to the control unit 14 of the robot 1 (S24). As the arms 111 to 115 move in accordance with the input target value, the reference plane becomes approximately perpendicular to the optical axis of the image pickup unit 2 and the optical axis of the image pickup unit 2 moves to the reference point P5 at the center. It will be the position and posture to pass through. That is, when the low-accuracy inclination correction is executed and terminated, the positional relationship between the imaging unit 2 and the reference points P1 to P9 becomes a low-precision vertical state.

1-3-3. Focus Adjustment Next, details of the focus adjustment will be described with reference to FIG.
First, the target value deriving unit 32 stores the current TCP position and orientation (S31). That is, as shown in FIG. 7, the position and orientation of the TCP corresponding to the reference plane F (1) at the time when the low-precision inclination correction is completed are stored.

Next, the target value deriving unit 32 performs the focus adjustment based on the coordinates of the reference point in the coordinate system of the reference plane derived during the low-precision tilt correction and the coordinates of the reference point in the coordinate system of the imaging unit 2. An area where the tool chuck 1261 is reflected in the acquired image and a distance from the imaging unit 2 to the tool chuck 1261 are derived (S32).

Subsequently, the image acquisition unit 31 acquires an image from the imaging unit 2 (S33). At this time, the image acquisition unit 31 acquires an image in which the tool chuck 1261 is reflected in the region derived in step S32.

Subsequently, the target value deriving unit 32 derives the focus index of the imaging unit 2 based on the image acquired from the imaging unit 2 (S34). As the focus index, a value obtained by standardizing a differential integrated value (sharpness) of a region where the tool chuck 1261 is captured with a certain area can be used. The target region from which the focus index is derived is set to a region where the tool chuck 1261 appears in the image, such as a region surrounded by a dotted line in FIG. 8B. That is, the tool chuck 12 as a marker by the imaging unit 2.
A focus index is derived based on a partial image cut out from the image obtained by capturing 61.

Subsequently, the target value deriving unit 32 determines whether or not the number of times that the imaging unit 2 has taken an image in the focus adjustment has reached a specified number (S35).

When the number of times the imaging unit 2 has taken the image in the focus adjustment has not reached the specified number, the target value deriving unit 32 moves the TCP by a predetermined distance in a direction perpendicular to the reference plane and approaching the imaging unit 2 (S36). The process is repeated from step S33. That is, until the number of times of imaging reaches the specified number of times, the reference plane is F (2), F (
3), F (4),..., The imaging unit 2 performs imaging on each reference plane, and a focus index is derived for each captured image. The closer the tool chuck 1261 is to the imaging unit 2, the larger the area where the tool chuck 1261 appears, and eventually the tool chuck 1261.
Will not fit in the image as shown in FIG. 8C. Therefore, the target region from which the focus index is derived is set to be larger as the tool chuck 1261 approaches the imaging unit 2.
The entire image is displayed after the region where 1 appears in contact with the edge of the image. When the focus index is derived for the region set in this way, if the focus index is sharpness, the focus index usually increases gradually and then gradually decreases.

When the number of times the image pickup unit 2 has taken an image in the focus adjustment reaches a specified number, the target value deriving unit 32 derives a target value for achieving the in-focus state based on the focus index, and the output unit 33 is derived. The target value is output to the robot 1 (S36). Specifically, for example, a target value for moving the TCP to the position where the maximum focus index is obtained among the plurality of focus indices derived in step S34 is derived. When the derived target value is output to the robot 1, the reference plane is in focus with respect to the imaging unit 2. Normally, the arms 111 to 115 move so that the TCP moves in a direction perpendicular to the reference plane and away from the imaging unit 2 so that the reference plane moves from F (7) to F (5) in FIG. To bring it into focus.

As described above, in the focus adjustment, after the reference point is approached in the first direction that is perpendicular to the reference plane and approaches the imaging unit 2, the partial image cut out from the image in which the tool chuck 1261 as the marker is imaged. Based on this, a target value for achieving the in-focus state is derived. Based on the derived target value, the arms 111 to 115 change the positional relationship between the reference point and the imaging unit 2 to a focused state. Here, since the focus adjustment is performed in a low-precision vertical state, the optical axis of the reference plane and the imaging unit 2 is vertical. Therefore, moving the reference point in the direction perpendicular to the reference plane is substantially the same as changing the imaging unit 2 and the reference point in a direction parallel to the optical axis of the imaging unit 2.

1-3-4. High-precision tilt correction Next, high-precision tilt correction will be described in detail with reference to FIG.
^First , the target value deriving unit 32 derives the current inclination index H1 (S401). The inclination index includes a central reference point P5 belonging to the same reference plane and four reference points P2, P4, P in the vicinity thereof.
6 and P8 are derived using the coordinates of the coordinate system of the reference plane of P8 and the coordinates of the coordinate system of the imaging unit 2. The tool chuck 1261 shows an image captured by the imaging unit with the reference plane perpendicular to the optical axis of the imaging unit 2 and the optical axis of the imaging unit 2 passing through the central reference point P5 (see FIG. 8D). The intervals d1, d2, d3, d4 between the reference points are equal. At this time, if the coordinates of the reference point Pn are represented by p (n), the following equation is established.

u ₁ = u ₂
v ₁ = v ₂
However,
u ₁ = p (5) −p (4)
u ₂ = p (6) −p (5)
_{v 1 = p (5) -p} (2)
_{v 2 = p (8) -p} (5)

Then, when the reference plane a straight line connecting the reference point P2 and the reference point P8 from the vertical state to the axis rotation, the difference between v ₁ and v ₂ is increased according to the rotation angle. Further, when the reference plane to the axis of the straight line connecting the reference point P4 and the reference point P6 from the vertical state to the rotation, the difference between u ₁ and u ₂ is increased according to the rotation angle. Therefore, the following formulas H _x and H _y can be used as the inclination index.
H _x = | u ₁ −u ₂ |
H _y = | v ₁ −v ₂ |

Next, the target value deriving unit 32 compares the derived inclination indexes H _x and H _y with a predetermined threshold value, and determines whether or not the inclination indexes H _x and H _y are equal to or less than the predetermined threshold value ( S402). When the inclination indexes H _x and H _y are equal to or less than a predetermined threshold value, the high-precision inclination correction is completed successfully.

If the slope indices H _x and H _y are not less than or equal to a predetermined threshold value, the target value deriving unit 32 determines whether or not the number of times of slope correction performed in step S411, which will be described later, is less than a predetermined threshold value ( S403). If the number of times of tilt correction is not less than a predetermined threshold, the high-precision tilt correction fails and ends. When the high-precision tilt correction fails and ends, the automatic calibration may be terminated, or the automatic calibration may be performed again from the low-precision tilt correction.

If the number of times of inclination correction is less than a predetermined threshold value, the target value deriving unit 32 stores the current TCP posture as the posture (1) (S404).

Next, the target value deriving unit 32 derives a target value for rotating the reference plane around the Y axis by δ degrees, and the output unit 33 outputs the derived target value to the robot (S405). δ is a minute angle determined in advance according to an error allowed in the high-precision vertical state. Subsequently, the target value deriving unit 3
2 sets a new reference point for the reference plane rotated with TCP according to the target value,
By analyzing an image captured in a state where the TCP is located on each new reference point, to derive the gradient index H ² (S406). Specifically, the target value deriving unit 32 derives a target value for moving the TCP to the reference points P2, P4, P5, P6, and P8 belonging to the new reference plane after rotation, and the output unit 33 is derived. The obtained target value is output to the robot 1, and the image acquisition unit 31 acquires an image captured with the TCP located at each reference point from the imaging unit 2. Target value deriving unit 32, by analyzing the acquired image thus similar to step S401, to derive the gradient index H ^2. As a result, the index H ² slope for the reference plane obtained by rotating δ degrees from a reference plane corresponding to the position (1) around the Y-axis is derived. Subsequently, the output unit 33 changes the TC to the posture (1).
A target value for returning P is output to the robot 1 (S407).

Next, the target value deriving unit 32 derives a target value for rotating the reference plane around the X axis by δ degrees, and the output unit 33 outputs the derived target value to the robot (S408). Subsequently, the target value deriving unit 32 sets a new reference point for the reference plane rotated together with the TCP according to the target value, and analyzes the image captured with the TCP positioned at each new reference point. ,
To derive a gradient index ^{H 3} (S409). As a result, from the reference plane corresponding to the posture (1), X
Indicator H ³ tilt the reference plane obtained by rotating δ degrees around the axis is derived.

Next, the target value deriving unit 32 uses the inclination correction amounts ΔV, ΔW based on the inclination indexes H ¹ , H ² , H ^3.
Is derived (S410). The inclination correction amounts ΔV and ΔW are derived by the following equations.

Next, the target value deriving unit 32 derives a target value based on the derived inclination correction amounts ΔV and ΔW,
The output unit 33 outputs the derived target value to the robot (S411). As a result, arm 1
By moving 11 to 115, the reference plane rotates by ΔV around the X axis and ΔW around the Y axis.

In the high-accuracy inclination correction, the processing from step S401 to S411 described above is ended in a high-precision vertical state with an affirmative determination made in step S402, or step S403.
Repeat until negative determination is made.

1-3-5.9 Point Calibration Next, 9 point calibration will be described. In a state where the high-precision tilt correction is successful, the reference plane is exactly perpendicular to the optical axis of the imaging unit 2. This means that the robot coordinate system and the coordinate system of the imaging unit 2 can calibrate the rotation component and the enlargement / reduction component in the reference plane based on the coordinates of the reference point obtained until the high-precision tilt correction is completed. . In the nine-point calibration, the robot coordinate system and the coordinate system of the imaging unit 2 are accurately calibrated at any point on the reference plane, taking into account the distortion of the lens 201. That is, in the nine-point calibration, a parameter for correcting the linear relationship between the robot coordinate system and the coordinate system of the imaging unit 2 nonlinearly is 9 in the robot coordinate system.
It is derived using the coordinates of one reference point and the coordinates of nine reference points in the coordinate system of the imaging unit 2. Specifically, the output unit 33 outputs a target value for moving the TCP to nine reference points to the robot 1 to move the arms 111 to 115, and the image acquisition unit 31 is a tool chuck located at each reference point. Nine images obtained by imaging 1261 are acquired. The calibration unit 34 analyzes each image and detects a reference point indicated by the tool chuck 1261. Subsequently, the calibration unit 34 derives a correction parameter for final calibration based on the coordinates of the target value output to the robot 1 by the output unit 33 and the coordinates of the reference point obtained by analyzing the image. . Note that the greater the number of reference points used for deriving parameters used for such nonlinear correction, the higher the calibration accuracy.

1-4. Modification Examples It goes without saying that if the position and orientation of the imaging unit 2 are fixed in the robot coordinate system, the coordinate system of the imaging unit 2 and the robot coordinate system can be calibrated by the method described above. For example, the imaging unit 2 may be fixed to an indoor wall so that the optical axis is horizontal, and a vertical reference plane may be set. Further, for example, the horizontal reference plane may be set by fixing the imaging unit 2 to the indoor ceiling vertically downward.

2. Second Example In the second example, a robot system in which the imaging unit 22 is mounted on a movable unit of the robot 1 will be described. In this embodiment, it is assumed that the imaging unit 22 is fixed to the arm 111 with a stay or the like so that the optical axis of the imaging unit 22 is parallel to the rotation shaft member 126 as shown in FIG. In the first embodiment, in order to calibrate the robot vision of the imaging unit 2 fixed to the robot coordinate system, the reference plane to be calibrated is set while moving the reference point in the robot coordinate system. In contrast, in the second embodiment, since the reference plane to be calibrated is fixed from the beginning with respect to the robot coordinate system, imaging is performed so that the optical axis is perpendicular to the reference plane to be calibrated. Part 22
Calibration is performed with the robot 1 moved by the robot 1.

Hereinafter, automatic calibration of the robot system in which the imaging unit 22 is mounted on the movable unit of the robot 1 will be described with reference to FIG.
The automatic calibration of the present embodiment is performed on the assumption that the optical axis of the imaging unit 22 is approximately parallel to the rotating shaft member 126. It is necessary to set the imaging position and imaging posture of the imaging unit 22 with respect to the arm 115 as environment information at a preparation stage before inputting the calibration start instruction. The imaging position is a position where the optical axis of the imaging unit 22 and the area image sensor 202 intersect. The imaging posture is the optical axis posture. However, the imaging position and the imaging orientation of the imaging unit 22 with respect to the arm 115 set as environment information do not need to correspond strictly to the attachment state of the imaging unit 22 to the arm 115 at the time of starting automatic calibration. .

Further, the automatic calibration of the present embodiment is fixed to the robot coordinate system as shown in FIG.
This is performed for the two reference points P21 to P29. The reference points P21 to P29 are lattice points included in one reference plane that can be arbitrarily set. Therefore, in the preparation stage, the marker board 4 with the markers 41 to 49 shown in FIG. 13 is fixed to the table 9 as shown in FIG. The marker board 4 is a plate in which nine markers 41 to 49 having a circular outline are attached to predetermined lattice points on a flat surface. The center position of each of the markers 41 to 49 on the marker board 4 is a target value deriving unit 32 in a two-dimensional coordinate system (marker coordinate system) fixed to the marker board 4.
Hold. The centers of the markers 41 to 49 are defined as reference points P21 to P29. The marker may be in any form that can identify the reference point by detecting the marker from the image of the marker. In other words, any form can be used as long as the position of a specific point can be geometrically defined with respect to the marker figure. A point separated from the marker may be defined as a reference point, a plurality of reference points may be defined from one marker, or a single reference point may be defined by a plurality of markers separated from each other.

When the instruction receiving unit 30 receives a calibration start instruction (S1), low-precision inclination correction is executed (S2). Since the markers 41 to 49 attached to the marker board 4 indicate nine reference points,
In the low-precision inclination correction of this embodiment, it is not necessary to move the robot 1 for each reference point. When a calibration start instruction is received in a state in which the entire marker board 4 can be imaged by the imaging unit 22, the image acquisition unit 3.
1 acquires from the imaging unit 22 one image in which nine markers 41 to 49 are captured. The target value deriving unit 32 analyzes one acquired image and detects nine reference points indicated by the markers 41 to 49 in the coordinate system of the imaging unit 22. Subsequently, the target value deriving unit 32 has the coordinates of the coordinate system of the imaging unit 22 of the reference point, the imaging position and imaging orientation of the imaging unit 22 input as environment information, and the coordinates of the marker coordinate system of the markers 41 to 49. Based on the above, the correspondence between the coordinate system of the imaging unit 22 and the robot coordinate system is derived. Specifically, a conversion matrix for converting the coordinate system of the imaging unit 22 to the robot coordinate system is derived for the reference plane on which the markers 41 to 49 are located. However, since the transformation matrix derived here is derived based on an image captured in a state where the imaging unit 22 is not focused on the markers 41 to 49, the imaging unit for the reference points P21 to P29. 2
2 does not accurately correspond to the imaging posture or the imaging position. Subsequently, the target value deriving unit 32 makes the optical axis of the imaging unit 22 perpendicular to the marker board 4 based on the derived transformation matrix,
In addition, a target value for moving the arm 115 to a position and posture in which the optical axis of the imaging unit 22 passes through the center of the central marker 45 is derived, and the derived target value is output to the control unit 14 of the robot 1 via the output unit 33. Output to. As the arms 111 to 115 move according to the input target value, the optical axis of the imaging unit 22 is approximately perpendicular to the marker board 4, and the optical axis of the imaging unit 22 is approximately the center marker 45. Go through the center. That is, when the low-accuracy inclination correction is executed and finished, the positional relationship between the imaging unit 22 and the reference plane becomes a low-precision vertical state.

Next, focus adjustment is executed (S3). In this embodiment, the focus adjustment is performed using the tool chuck 1261.
Instead of moving the TCP so that the sharpness of the area where the image appears is maximized, the arm 115 is moved so that the sharpness of the area where the marker 45 in the center of the marker board 4 is reflected is maximized. Same as focus adjustment. When the focus adjustment is completed, the imaging unit 2
2 and the reference point P25 are in a focused state.

Next, the imaging unit posture correction is executed (S6). The imaging unit posture correction is a process of correcting a deviation between the imaging posture of the imaging unit 22 set as environment information and the actual imaging posture of the imaging unit 22 by correcting the environmental information. Details will be described later.

Next, high-precision tilt correction is executed (S5). The high-precision inclination correction is the same as the high-precision inclination correction of the first embodiment except for the following points. The center reference point P5 among the five reference points used in the first embodiment is set as the current imaging position of the imaging unit 22. Further, the remaining four reference points P2, P4, P6, and P8 of the five reference points used in the first embodiment are set as the imaging positions of the imaging unit 22, and the imaging unit 2 is used.
On the plane that is perpendicular to the optical axis 2 and includes the current imaging position, an interval of 90 degrees centered on the imaging position of the current imaging unit 22 is set. For the setting of these five imaging positions, the imaging position and imaging posture of the imaging unit 22 and the current position and posture of the arm 115, which are the environment information corrected in step S6, are used. The image acquisition unit 31 images the marker board 4 with the imaging unit 22 that is at the five imaging positions set in this manner and whose optical axis is perpendicular to the plane including the five imaging positions, and acquires five images. The target value deriving unit 32 analyzes the five images and detects the center of the marker 45 at the center of the marker board 4 in the coordinate system of the imaging unit 22. Marker 4 detected in this way
Using the five center coordinates 5 in place of the coordinates of the five reference points detected in the high-accuracy correction of the first embodiment, high-precision tilt correction similar to that of the first embodiment is executed. By executing such high-precision tilt correction, the optical axis of the imaging unit 22 is perpendicular to the marker board 4 and the imaging unit 2
A target value corresponding to a state in which the two optical axes pass through the center of the marker 45 is derived. When the derived target value is input to the robot 1, the arms 111 to 115 move based on the input target value, and the optical axis of the imaging unit 22 passes through the center of the marker 45 and is perpendicular to the marker board 4. Become. That is, when the high-precision inclination correction is completed successfully, the reference points P21 to 29 and the imaging unit 22 are completed.
The positional relationship of is in a highly accurate vertical state.

Here, the imaging unit posture correction in step S6 will be described in detail. For example, it is assumed that an imaging position and an imaging posture in which the optical axis of the imaging unit 22 corresponds to the dotted line in FIG. 14 at the imaging positions P2, P5, and P8 are set as environment information. And actually, the imaging positions P2, P5, P
8, the imaging unit 22 is attached to the arm 115 so as to correspond to the solid lines A2, A5, and A8 whose optical axes are not perpendicular to the reference plane F. In this case, the imaging positions P2, P5
, It is assumed that high-precision tilt correction is executed by positioning the imaging unit 22 at P8 and imaging the marker board 4. However, based on the target value derived by high-precision tilt correction, the arm 1
Even if 11 to 115 operate, the optical axis of the imaging unit 22 does not become perpendicular to the marker board 4. The high-precision tilt correction is based on the premise that the center marker 45 is imaged five times by the imaging unit 22 from five imaging positions on a plane perpendicular to the “real optical axis of the imaging unit 22”. On the other hand, the inclination index is derived based on the five images picked up by the image pickup unit 22 from the five image pickup positions on the plane perpendicular to the “optical axis corresponding to the environment information with error”. is there. Therefore, it is necessary to execute the correction for matching the “real optical axis of the imaging unit 22” with the “optical axis corresponding to the environment information” before the high-precision correction. This correction necessary before the high-accuracy correction is the imaging unit posture correction.

In the imaging unit attitude correction, an imaging attitude in which the optical axis is perpendicular to a plane including these imaging positions from the five imaging positions P2, P4, P5, P6, and P8 (imaging attitude based on environment information to be corrected). The image is picked up by the image pickup unit 22 and the marker board 4 is obtained from each of the five images.
The center of the center marker 45 is detected. Subsequently, the marker 45 detected in this way
Are used instead of the coordinates of the five reference points detected in the high accuracy correction of the first embodiment, and the same inclination correction amounts ΔV and ΔW as in the first embodiment are derived. Derived correction amount ΔV,
When the imaging posture of the imaging unit 22 as environment information is corrected using ΔW, “real imaging unit 22 is obtained.
The “optical axis” and “optical axis corresponding to environmental information” can be matched.

After the high-precision tilt correction is completed successfully, nine-point calibration is performed (S5). In the nine-point calibration of the present embodiment, the imaging unit 22 is moved while moving the TCP to the nine reference points P1 to P9 of the first embodiment.
Instead of detecting the position of each reference point from the image captured by the image capturing unit 22 using the coordinate system of the image capturing unit 22, nine markers 41 to 41 are acquired from the image acquired from the image capturing unit 22 in the current image capturing position and image capturing posture in a high-precision vertical state. The center of 49 is detected as reference points 21-29. Then, a final conversion formula for converting the coordinates of the reference points 21 to 29 detected in the coordinate system of the imaging unit 22 into the coordinates of the reference points 21 to 29 in the robot coordinate system is derived.

3. Third Example The state of the robot 1 is controlled by inputting to the robot 1 target values indicating the position and posture of a tool chuck 1261 on which various tools are mounted (that is, the TCP position and posture). Therefore, in order to control the position and posture of the tool mounted on the tool chuck 1261 and other parts, the position and posture of the tool with respect to the TCP are set in advance as environment information, and the position and posture of the tool are changed to the TCP. It is necessary to input the target value to the robot 1 by converting it into the corresponding position and posture. In the present embodiment, a method for automatically setting the positions and postures of various tools with respect to the TCP using a robot vision will be described. Before detecting the positions and orientations of various tools with respect to TCP using robot vision, it is assumed that the robot coordinate system and the coordinate system of the imaging unit have been calibrated with respect to the reference plane. Therefore, in this embodiment, as described in the first embodiment, the reference plane F perpendicular to the optical axis of the imaging unit 2 fixed to the table 9 is set, and nine references on the set reference plane F are set. Assume that the calibration is complete for a point. In this state, a method for detecting the position and posture of the tool whose position and posture with respect to the TCP with respect to the TCP using the robot vision will be described with reference to FIG. In this embodiment, “the position and orientation of the tool relative to the TCP” refers to the position and orientation of the tool on the reference plane, and the TCP position and orientation (x, y, z, u).
, V, w) is represented by an offset (ΔX, ΔY, ΔU) for identification. Here, (x, y, z, u, v, w) is a robot coordinate system, and (ΔX, ΔY, ΔU) is a reference plane coordinate system. As already described, the robot coordinate system and the reference plane coordinate system are used. The relationship is linear. Therefore, there is no geometric difference in whether the position and orientation of the tool are determined by the coordinate system of the robot coordinate system or the coordinate system of the reference plane. ΔU is a rotation component around the Z axis perpendicular to the reference plane.

First, the reference point of the tool indicated by the tool marker is moved on the reference plane by moving the TCP parallel while maintaining the posture in which the central axis is perpendicular to the reference plane. At this time, the center axis of the tool chuck 1261 may be moved in parallel.
261 may be rotated around its central axis (because ΔU is included in the offset). The tool marker only needs to indicate a reference point, and the tool itself may be used as a tool marker. However, in this embodiment, a label is attached to the tool as a tool marker. It is sufficient that the label has a known shape and size. The reference point may be a point fixed with respect to the tool. Further, when the tool reference point is moved to the reference plane, the TCP may be moved by jog feed, but the TCP may be automatically moved using a robot vision. Specifically, since the shape and size of the tool marker are known, the position of the tool marker before movement relative to the reference plane is specified based on the image of the tool marker imaged before movement. Then, a TCP target value for moving the tool marker to the reference plane is derived from the position of the tool marker specified with respect to the reference plane, and the derived target value is output to the robot 1. Furthermore, by repeating this imaging and movement process, the tool reference point indicated by the tool marker can be accurately positioned on the coordinate plane.

Next, the tool marker is imaged by the imaging unit 2 in a state where the reference point of the tool is located on the reference plane, and the position of the reference point indicated by the tool marker is detected by the coordinate system of the imaging unit 2.
Then, the position and orientation (x ₀ , y ₀ , z ₀ , TCP position in the robot coordinate system of the TCP are determined based on the coordinates of the perpendicular foot drawn from the TCP to the reference plane at the time of imaging and the coordinates of the detected reference point. u ₀ , v
₀ , w ₀ ) offset (ΔX, Δ for specifying the position and orientation of the tool on the reference plane
Y, ΔU) is derived.

For example, by deriving an offset (ΔX, ΔY, ΔU) using the imaging unit 22 of the second embodiment as a tool, the position and orientation of the imaging unit 22 necessary as environment information are specified. Therefore, instead of allowing the user to set the position and orientation of the imaging unit 22, it is possible to automatically detect the position and orientation of the imaging unit 22, and to automatically set the detected position and orientation of the imaging unit 22 as environment information. is there. Then, it is possible to calibrate the coordinate system of the imaging unit 22 and the robot coordinate system using the imaging unit 22 in which the environment information is automatically set as described above. That is, the calibration of the respective coordinate systems of the imaging unit 2 fixed to the robot coordinate system and the imaging unit 22 fixed to the movable unit of the robot 1 and the robot coordinate system is completed automatically only by inputting a calibration start instruction. It is also possible to make it. Further, when an imaging unit different from the imaging unit 2 and the imaging unit 22 is added, the imaging unit 22 captures the image of the other imaging unit, thereby calibrating the coordinate system of the other imaging unit and the robot coordinate system. It is also possible to do. That is, for example, when the robot operates using the marker board 4, the coordinate system of the plurality of imaging units and the robot coordinate system can be automatically calibrated.

In addition, when environmental information is required for a plurality of tools T1 and T2, an offset for specifying each position and posture may be derived for each tool. That is, the tool T1 is moved to the reference plane F as shown in FIG. 15A to derive the offset for the tool T1, and the tool T2 is moved to the reference plane F as shown in FIG. 15B to derive the offset for the tool T2. do it. Further, the example in which one calibrated reference plane is finally set has been described so far, but two or more calibrated reference planes may be set. For example, Tool T
A reference plane corresponding to 1 and a reference plane corresponding to the tool T2 may be set, and each reference plane may be calibrated with the robot coordinate system.

4). Other Embodiments The technical scope of the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications can be made without departing from the scope of the present invention.

For example, the calibration start instruction may be received by a click operation with a mouse, a tap operation on a display, or a voice.

Further, the order of the low-accuracy inclination correction, the focus adjustment, and the high-accuracy inclination correction described in the above embodiments may be changed, and the focus adjustment may be executed before the low-accuracy inclination correction.
Any of the high-precision tilt correction may be omitted. Further, a target value derived based on an image in which the marker is imaged by the imaging unit in a state where calibration is not performed is acquired, and the positional relationship between the imaging unit and the reference point is determined in advance based on the target value. There may be at least one process to be changed. For example, the robot operation for focusing on the reference point is performed by jog feed.
Only high-precision tilt correction may be performed based on a target value derived based on an image.

Further, the robot coordinate system and the coordinate system of the imaging unit may be calibrated based on the coordinates of three or more reference points acquired when performing high-precision tilt correction without performing nine-point calibration. That is, the calibration of the coordinate system of the imaging unit and the robot coordinate system may be performed using three or more reference points on the reference plane set by high-precision tilt correction, and the calibration is executed using eight or less reference points. May be 10
Calibration may be executed using the above reference points.

Further, some target value for changing the robot to a predetermined posture may be included in the calibration start instruction. For example, if the position where the imaging unit or marker board is fixed to the table is determined in advance, the target value for moving the robot without using the robot vision to the position where the reference plane and the imaging unit almost face each other is instructed to start calibration May be included.

The present invention can also be applied to vertical articulated robots other than six axes, and can also be applied to a scalar robot in which all the rotation axes of the arms are parallel. In the present invention, two or more imaging units are used.
It can also be applied to three-dimensional robot vision. That is, in calibrating the coordinate system of a plurality of imaging units and the robot coordinate system for realizing a three-dimensional robot vision,
By applying the present invention, the positional relationship between each imaging unit and the marker is changed by a calibration start instruction so that an image necessary for calibrating the coordinate system of the imaging unit and the coordinate system of the robot can be captured. Can do.

131-136 ... motor, 111-115 ... arm, 1 ... robot, 2 ... imaging unit, 3 ...
PC, 4 ... marker board, 9 ... table, 14 ... control unit, 31 ... image acquisition unit, 31 ... image acquisition unit, 32 ... target value deriving unit, 33 ... output unit, 34 ... calibration unit, 41-49 ... marker ,
110 ... Base, 111 ... First arm, 112 ... Second arm, 113 ... Third arm, 114
... 4th arm, 115 ... 5th arm, 121-126 ... Rotating shaft member, 131-136 ... Motor, 201 ... Lens, 202 ... Area image sensor, 1261 ... Tool chuck, F ... Reference plane, P1-P9 ... Reference Point, P21 to P29 ... Reference point

Claims (15)

An instruction receiving unit for receiving a calibration start instruction;
When the calibration start instruction is accepted, an arm that changes the positional relationship between the marker indicating the reference point and the imaging unit;
With
Calibration of the coordinate system of the imaging unit and the coordinate system of the robot is performed based on an image in which the marker is captured by the imaging unit after the positional relationship between the imaging unit and the marker is changed.
robot. When the arm receives the calibration start instruction, the arm changes the positional relationship to a predetermined state,
The calibration is performed based on an image in which the marker is imaged by the imaging unit when the positional relationship is in the predetermined state.
The robot according to claim 1. The arm acquires a target value derived based on an image obtained by imaging the marker by the imaging unit, and changes the positional relationship to the predetermined state based on the target value.
The robot according to claim 2. The calibration start instruction does not include a target value for changing the positional relationship to the predetermined state.
The robot according to claim 2 or 3. The predetermined state includes a state where the imaging unit is focused on the marker.
The robot according to any one of claims 1 to 4. When the calibration start instruction is received, the arm acquires a target value derived based on a partial image cut out from an image obtained by capturing the marker by the imaging unit, and the position is determined based on the target value. The relationship is changed so that the imaging unit is focused on the marker.
The robot according to claim 5. The arm changes the positional relationship between the imaging unit and the marker in a first direction perpendicular to a plane including three or more reference points, and then moves the imaging unit in a second direction opposite to the first direction. And focusing the imaging unit on the marker by changing the positional relationship between the marker and
The robot according to claim 5 or 6. The predetermined state includes a state in which a plane including three or more reference points and an optical axis of the imaging unit are perpendicular to each other.
The robot according to any one of claims 1 to 7. In the predetermined state, the positions where the optical axes of the imaging units pass through the plane are different from each other.
Including more than one state,
The robot according to claim 8. The imaging unit is provided in the arm.
The robot according to any one of claims 1 to 9. The marker is provided on the arm;
The robot according to any one of claims 1 to 9. An instruction receiving unit for receiving a calibration start instruction;
When the calibration start instruction is accepted, an arm that changes the positional relationship between the marker indicating the reference point and the imaging unit;
A robot comprising:
A calibration device that calibrates the coordinate system of the imaging unit and the coordinate system of the robot based on an image of the marker imaged by the imaging unit after the positional relationship between the imaging unit and the marker has changed;
A robot system comprising: The positional relationship between the marker indicating three or more reference points and the imaging unit is
A first state in which a plane including the three or more reference points and an optical axis of the imaging unit are perpendicular to each other;
A second state in which the imaging unit focuses on the marker indicating at least one of the three or more reference points;
A third state in which the optical axis of the imaging unit passes the first point on the plane;
A fourth state in which the optical axis of the imaging unit passes through a second point on the plane;
A fifth state in which the optical axis of the imaging unit passes through a third point on the plane;
Equipped with an arm that automatically changes to at least one of
Calibration of the coordinate system of the imaging unit and the coordinate system of the robot is performed based on an image in which the marker is captured by the imaging unit in the third state, the fourth state, and the fifth state.
robot. The arm automatically changes the positional relationship to the first state, the second state, the third state, the fourth state, and the fifth state.
The robot according to claim 13. The arm acquires a target value derived based on an image of the marker imaged by the imaging unit, and automatically changes the positional relationship based on the target value;
The robot according to claim 13 or 14.

Images