JP7264763B2 - calibration device - Google Patents

Description

The present invention relates to a calibration device that updates various parameters indicating states such as the position and orientation of robots and sensors to more appropriate values.

As a conventional calibration device, one described in Patent Document 1 is known. In the abstract of this document, there is described a solution for ``setting a calibration range in advance in the image coordinate system for calibration of a single camera or a stereo camera and allowing calibration to be performed within an arbitrary range.'' "A target mark is placed on a robot, and by moving the robot and detecting the target mark at multiple locations within the field of view of the camera, the robot coordinate system of the robot and the image coordinate system of the camera are associated. A calibration device comprising: an image range setting unit for setting an image range in an image coordinate system of a camera; A calibration device is disclosed that includes a calibration range measuring unit that measures a range.

That is, in the calibration device of Patent Document 1, the tip of the arm of the robot is continuously moved, and target marks attached to the tip of the arm are detected at a plurality of locations, thereby calibrating the visual sensor (monocular camera, stereo camera). Is going.

JP 2018-111166 A

However, in the articulated robot arm used in Patent Document 1, manufacturing errors and control errors of each joint accumulate and appear in the target mark at the tip of the arm, so the actual position of the target mark deviates from the theoretical position without error. Therefore, even if the visual sensor is calibrated based on the actual position of the target mark, there is a high possibility that the calibration is not appropriate.

Therefore, the present invention aims to provide a calibration device that can update various parameters of the robot and sensors to more appropriate values without using an unstable reference such as a target mark attached to the tip of the arm of the robot. aim.

In order to solve the above problems, the calibration device of the present invention is a calibration device that uses a camera or a laser displacement meter as a sensor for observing a robot in front of a target mark whose installation position is known, as a sensor, A sensor data acquisition unit that generates sensor data obtained by measuring the robot and the target mark by the sensor , a robot control unit that controls the robot, and a data storage unit that stores sensor parameters related to the sensors and robot parameters related to the robot. a virtual data generating unit that generates virtual data obtained by measuring the robot and the target mark from the sensor in the virtual space based on the sensor parameter and the robot parameter; and comparing the sensor data and the virtual data. , an environment determination unit that determines the difference between the two; a relative position/orientation estimation unit that searches for the sensor parameter or the robot parameter that reduces the difference; and the sensor parameter or the robot parameter stored in the data storage unit. an updating unit that updates the sensor parameters or the robot parameters searched by the relative position/orientation estimating unit, wherein the data storage unit further stores environment information regarding the positions of the sensor, the robot, and the target mark. The virtual data generation unit observes the robot and the target mark from the sensor in a virtual space in which the sensor, the robot, and the target mark are arranged based on the environment information, thereby generating the virtual data should be generated.

According to the calibration apparatus of the present invention, various parameters of the robot and sensors can be updated to more appropriate values without using an unstable reference such as the target mark attached to the tip of the arm of the robot.

Functional block diagram of the calibration device of the first embodiment Coordinate space representation diagram of the robot and sensor of Example 1 Processing flowchart of the virtual data generation unit of the first embodiment Explanatory diagram of matching processing between virtual imaging data and sensor data Processing flowchart of imaging environment determination unit and relative position/orientation estimation unit An example of robot posture change when posture information is changed Data Representation of Parameter Updates and Differences Example of output screen for updating parameters in Example 2

An embodiment of the sensor position/orientation calibration device of the present invention will be described below with reference to the drawings.

A calibration device 1 according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 7. FIG.

FIG. 1 is a diagram showing functional blocks of a calibration device 1 of this embodiment together with a schematic arrangement of a robot 2, a sensor 3, and a target mark 4. As shown in FIG. Below, the robot 2, the sensor 3, and the target mark 4 will be outlined, and then the details of the calibration device 1 will be explained.

<Robot 2, Sensor 3, Target Mark 4>
The robot 2 is a multi-joint robot arm that grips an object with a robot hand at the tip of the arm. etc. is controlled. This embodiment can be applied not only to robot hands but also to various end effectors such as suction hands. Although the robot 2 having three joints will be described below as an example, it goes without saying that the application of the present invention is not limited to the robot 2 having three joints.

The sensor 3 is a camera, a laser displacement meter, or the like that observes the robot 2 in front of the target mark 4 whose installation position is known. In the following description, it is assumed that the sensor 3 is a camera.

The target mark 4 is formed by arranging a plurality of marks on a plane. When the relative positions and orientations of the sensor 3 and the target mark 4 are different, the relative position and orientation of the sensor 3 and the target mark 4 are detected by utilizing the fact that the appearance of the target mark 4 observed by the sensor 3 is different. can do. It should be noted that the plurality of marks arranged on the target mark 4 may be individually equivalent, or may be unique and individually identifiable.

Here, a coordinate system related to this embodiment will be described with reference to FIG. In the figure, W denotes a world coordinate system, which is a coordinate system that serves as a reference when specifying the position of the robot 2, the sensor 3, the target mark 4, or an object grasped by the robot 2. FIG. R is a robot coordinate system whose origin is a fixed point (for example, base) of the robot 2, and is a coordinate system translated while maintaining each axial direction of the world coordinate system W. FIG. S is a sensor coordinate system whose origin is the fixed point of the sensor 3 (for example, a light receiving element of a camera), the line-of-sight direction of the sensor 3 is the Sz axis, and the Sx axis and the Sy axis are set so as to be orthogonal to the Sz axis. coordinate system.

_CRS is a coefficient for coordinate transformation between the coordinates on the sensor coordinate system S and the coordinates on the robot coordinate system R mutually. By using this coefficient _CRS , the position and shape of the object on the sensor coordinate system S detected by the sensor 3 can be recognized as the position and shape of the object on the robot coordinate system R. Using coordinates on the robot coordinate system R, the robot 2 can be commanded to grip a desired object.

When coordinate transformation is performed using such a coefficient _CRS , the detection result of the sensor 3 can be accurately transmitted to the robot 2 if the coefficient _CRS is appropriate _. In such a case, the detection result of the sensor 3 cannot be accurately transmitted to the robot 2, and there is a possibility that, for example, the robot 2 cannot correctly grip a desired object.

Since the accuracy of the coefficient _CRS depends on the sensor parameter _PS and the robot parameter _PR , which will be described later, if these parameters can be optimized, the coefficient _CRS will also be optimized. Therefore, in the calibration device 1 of the present embodiment, the sensor parameter _PS and the robot parameter _PR can be automatically optimized, so that the coefficient _CRS can be easily optimized regardless of the skill level of the operator. made it possible.

<Calibration device 1>
Returning now to FIG. 1, the detailed structure of the calibration device 1 required for optimizing the coefficient _CRS will now be described. Note that the calibration device 1 is actually a computer such as a personal computer equipped with hardware such as an arithmetic device such as a CPU, a main storage device such as a semiconductor memory, an auxiliary storage device such as a hard disk, and a communication device. . Each function shown in FIG. 1 is realized by having the arithmetic unit execute the program loaded into the main memory while referring to the database recorded in the auxiliary memory. Although the details of each function shown here will be described later, well-known techniques in the computer field will be omitted as appropriate.

As shown in FIG. 1, the calibration device 1 of this embodiment includes a robot control unit 1a, a data storage unit 1b, a virtual imaging data generation unit 1c, a sensor data acquisition unit 1d, an imaging environment determination unit 1e, a relative position/posture estimation It includes a unit 1f, a robot parameter updating unit 1g, and a sensor parameter updating unit 1h.

The robot control unit 1a communicates bidirectionally with the robot 2 and controls the attitude and behavior of the robot 2. FIG. For example, the robot control unit 1a controls the posture of the robot 2 by designating the rotation angles θ ₁ to θ ₃ of each joint of the robot 2 . A combination of the rotation angles θ ₁ to θ ₃ transmitted from the robot control unit 1a to the robot 2 is hereinafter referred to as posture information _IS .

The data storage unit 1b stores environment information I _E , sensor parameters P _S , and robot parameters _PR , which are used to generate virtual imaging data _DV by the virtual data generation unit 1 c, which will be described later. The environment information _IE is, for example, information such as the installation positions of the robot 2, the sensor 3, and the target mark 4 in the world coordinate system W. FIG. For example, when the sensor 3 is a camera, the sensor parameters _PS are internal parameters such as its focal length (fx, fy), distortion coefficient, optical axis center (cu, cv), and the translation of the sensor with respect to the world coordinate system. External parameters indicating movement (Tx, Ty, Tz) and rotation (3x3 matrix of r11 to r33). The robot parameter P _R is, for example, a parameter of the posture of each arm of the robot 2 (rotational angle θ of each joint) and the open/closed state of the robot hand.

The virtual imaging data generation unit 1c generates virtual imaging data _DV based on the environment information I _E , the sensor parameters P _S , and the robot parameters P _R obtained from the data storage unit 1b. The processing for generating the virtual image data _DV here will be described with reference to the flowchart of FIG.

First, in step S31, the virtual imaging data generation unit 1c acquires the environment information _IE from the data storage unit 1b, and based on it, arranges the robot _2V , the sensor _3V , and the target mark _4V in the virtual space. . Next, in step S32, the virtual imaging data generation unit 1c provides the robot _2V in the virtual space with the rotation angle θ of each joint and the open/closed state of the robot hand based on the robot parameters _PR acquired from the data storage unit 1b. set. Next, in step S33, the virtual imaging data generation unit 1c sets internal parameters for the sensor _3V (camera) in the virtual space based on the sensor parameters P _S acquired from the data storage unit 1b. As a result, the robot 2 _V , the sensor 3 _V , and the target mark 4 _V are arranged in the virtual space, as exemplified in the virtual space data display area 5b of FIG. 8, which will be described later.

Finally, in step S34, the virtual image data generating unit 1c creates virtual image data DV corresponding to images of the robot _2V and the target mark _4V in the virtual space taken from the viewpoint of the sensor _3V in the virtual space _. to generate

To project an arbitrary point (x, y, z) on the virtual space onto the virtual imaging data _DV , for example, Equation 1 is used. In Expression 1, u and v are coordinates in a uv coordinate system (virtual coordinate system) virtually set on the virtual image data _DV . fx and fy are the focal length per pixel of the camera, and are values obtained by dividing the focal length of the camera by the vertical and horizontal physical intervals of the pixels of the camera. cu and cv are the coordinates of the optical axis center in the virtual coordinate system where the optical axis of the camera and the virtual image data _DV intersect.

The sensor data acquisition unit 1d acquires sensor data _DS based on the output of the actual robot 2 and the target mark 4 observed from the viewpoint of the actual sensor 3. FIG. In this embodiment, since the sensor 3 is a camera, the sensor data _DS obtained here are image data.

The imaging environment determination unit 1e compares the virtual imaging data _DV output by the virtual imaging data generation unit 1c and the sensor data _DS output by the sensor data acquisition unit 1d to determine the imaging environment. Details of the processing here will be described with reference to FIGS. 4 and 5. FIG.

4A and 4B are explanatory diagrams of the matching processing of the virtual image data _DV and the sensor data _DS , and FIG. The imaging data D _V , (b) is the sensor data D _S generated based on the actual output of the sensor 3, and (c) is the matching processing of the virtual imaging data D _V and the sensor data D _S .

As is clear from the large difference between the virtual imaging data _DV and the sensor data _DS , FIG. is equivalent to

Parameters can be optimized by choosing parameters that can produce virtual imaging data _DV that match the sensor data _DS . Specifically, as shown in (c) matching processing, parameter adjustment equivalent to rotation processing and movement processing required to match the virtual image data _DV with the sensor data _DS may be performed.

FIG. 5 is a flowchart showing an example of imaging environment determination processing executed by the imaging environment determination unit 1e.

First, in step S51, the imaging environment determination unit 1e acquires the sensor data _DS and the virtual imaging data _DV . Next, in step S52, the imaging environment determination unit 1e measures the reference marker position of the target mark 4 included in both data, and in step S53, the imaging environment determination unit 1e matches corresponding points between both data. .

By the processing of steps S51 to S53 described above, the imaging environment determination unit 1e can calculate the amount of rotation and the amount of movement necessary to match the virtual imaging data _DV with the sensor data _DS . The determination unit 1e determines whether the amount of rotation or the amount of movement is equal to or greater than a predetermined threshold. Then, when the amount of rotation and the amount of movement required for parameter calibration are less than the threshold values, the imaging environment determination unit 1e determines that the currently stored parameters are effective in accordance with the actual situation, and the data storage unit 1b. The processing of this flow chart ends without updating the parameters of .

On the other hand, when the amount of rotation or the amount of movement required to match the virtual imaging data _DV with the sensor data _DS is equal to or greater than the threshold, the imaging environment determination unit 1e determines that the parameters currently stored in the data storage unit 1b are It is determined that it does not conform to the actual situation and correction is necessary, and shifts to relative position/orientation estimation processing for parameter correction.

When shifting to the relative position/orientation estimation processing, first, in step S55, the relative position/orientation estimation unit 1f searches for the sensor parameter _PS that minimizes the difference between the virtual image data _DV and the sensor data _DS between the target marks 4. . For example, if there are four types of sensor parameters P _S , P _S1 , P _S2 , P _S3 and P _S4 , optimum values for minimizing the difference between the target marks 4 are obtained in order from the sensor parameters P _S1 . Next, in step S56, the relative position/orientation estimation unit 1f searches for robot parameters _PR that minimize the difference of the robot 2 between the virtual image data _DV and the sensor data _DS . For example, if there are four types of robot parameters P _R , P _R1 , P _R2 , P _R3 , and P _R4 , optimum values for minimizing the difference of the robot 2 are obtained in order from the robot parameters P _R1 .

Then, in step S57, the robot parameter updating unit 1g or the sensor parameter updating unit 1h updates each parameter stored in the data storage unit 1b to the optimum value of each parameter obtained in steps S55 and S56. In FIG. 5, the search _for the optimum value of the robot parameter P _R was performed after the search for the optimum value of the sensor parameter P _{S .} is larger than the difference on the target mark 4 side, the optimum value of the sensor parameter _PS may be searched after searching for the optimum value of the robot parameter _PR .

Next, a method for evaluating the posture accuracy of the robot 2 will be described with reference to FIGS. 6 and 7. FIG.

When evaluating the posture accuracy of the robot 2, first, the robot control unit 1a transmits arbitrary posture information I _S (θ ₁ , θ ₂ , θ ₃ ) to the robot 2 . At this time, the robot 2 assumes the posture shown in FIG. 6A, so the error amount d corresponding to the posture information _IS is measured and recorded.

Next, the robot control unit 1a transmits to the robot 2 posture information I _S '(θ ₁ ', θ ₂ , θ ₃ ) in which only the rotation angle θ ₁ of the posture information I _S is changed to the rotation angle θ ₁ '. do. At this time, the robot 2 assumes the posture shown in FIG. 6B, so the error amount d' corresponding to the posture information I _S ' is measured and recorded.

Then, when the error amount d and the error amount d' are different and the error amount d' is large, the posture accuracy of the joint corresponding to the rotation angle θ1 is poor, or the accuracy of the link on the tip side of the joint is poor. can be determined.

By executing the posture accuracy evaluation work as described above for a large number of posture information _IS , it is possible to create a correspondence table between the posture information _IS and the amount of error d as shown in FIG. Then, from the relationship between the attitude information I _S and the error amount d shown here, an element of the attitude information I _S that increases or decreases the error amount d by a general analysis method such as multiple regression analysis or logistic regression analysis, or the attitude information I Since we can find a combination of the elements of _S , we can identify it as the optimal robot parameter P _R .

Then, _the coefficient _CRS is calculated based on the sensor parameter _PS and the robot parameter _PR corrected based on FIGS. By performing transformation, the position and shape of an object in the vicinity of the robot 2 can be correctly recognized, so that the robot 2 can appropriately grip the object.

As described above, according to the calibration apparatus of the present embodiment, the robot parameters P _R that define the state of the robot and the , the sensor parameter _PS defining the state of the sensor can be calibrated to a more appropriate value. By using those parameters, the control of the robot becomes more accurate, and for example, an object can be more accurately grasped by the robot hand.

Next, with reference to FIG. 8, a calibration device 1 according to a second embodiment of the invention will be described. Duplicate descriptions of common points with the first embodiment will be omitted.

In the first embodiment, the calibration device 1 automatically updates the sensor parameters _PS and the robot parameters _PR , but in the present embodiment, the operator is allowed to participate in the processing in the relative position/orientation estimation unit 1f. .

FIG. 8 is an example of the display screen 5 of the calibration tool application software displayed on the display device connected to the calibration device 1 .

As shown here, the display screen 5 displays an imaging data display area 5a, a virtual space data display area 5b, a sensor parameter setting area 5c, a robot parameter setting area 5d, an enter button 5e, and a pointer 5f. .

In the imaging data display area 5a, the sensor data D _S actually acquired by the sensor 3 and the virtual imaging data D _V generated by the virtual imaging data generation unit 1c using the current sensor parameters P _S and robot parameters P _R are displayed. are displayed side by side. Note that if the two are different, the current sensor parameter _PS or robot parameter _PR is inappropriate, and if they match, the current sensor parameter _PS and robot parameter _PR are appropriate. Same as 1.

The virtual space data display area 5b displays the arrangement of the robot _2V , the sensor _3V , and the target mark _4V in the virtual space. By adjusting the parameters, the state of the virtual imaged data _DV displayed in the imaged data display area 5a and the state of the robot _2V displayed in the virtual space data display area 5b are changed.

When the operator operates the pointer 5f on the display screen 5 to appropriately set each parameter in the sensor parameter setting area 5c and the robot parameter setting area 5d, the sensor data _DS and the virtual imaging data _{DV are} displayed. approximately match. At this time, when the operator presses the decision button 5e, the robot parameter updating section 1g and the sensor parameter updating section 1h update each parameter stored in the data storage section 1b based on each parameter value being displayed.

According to the configuration of the second embodiment described above, not only can an effect equivalent to that of the first embodiment be obtained, but the operator can easily grasp which parameter has a problem by an intuitive operation. be able to.

1 ... calibration device,
1a ... robot control unit,
1b ... data storage unit,
1c ... virtual imaging data generation unit,
1d ... sensor data acquisition unit,
1e ... imaging environment determination unit,
1f: relative position/orientation estimator,
1g ... robot parameter update unit,
1h ... sensor parameter update unit,
2, 2 _V ... Robot,
3, 3 _V ... sensor,
4, 4 _V ... target mark,
5 ... display screen,
5a... Imaging data display area 5a,
5b... Virtual space data display area 5c... Sensor parameter setting area 5d... Robot parameter setting area 5e... Decision button 5f... Pointer

Claims (5)

A calibration device that uses a camera or a laser displacement meter as a sensor for observing a robot in front of a target mark whose installation position is known,
a sensor data acquisition unit that generates sensor data obtained by measuring the robot and the target mark with the sensor;
a robot control unit that controls the robot;
a data storage unit storing sensor parameters related to the sensor and robot parameters related to the robot;
a virtual data generation unit that generates virtual data obtained by measuring the robot and the target mark from the sensor in the virtual space based on the sensor parameter and the robot parameter;
an environment determination unit that compares the sensor data and the virtual data and determines a difference between the two;
a relative position and orientation estimator that searches for the sensor parameter or the robot parameter that reduces the difference;
an updating unit that updates the sensor parameters or the robot parameters stored in the data storage unit to the sensor parameters or the robot parameters searched by the relative position/orientation estimating unit;
and
the data storage unit further stores environmental information regarding the positions of the sensor, the robot, and the target mark;
The virtual data generation unit generates the virtual data by observing the robot and the target mark from the sensor in a virtual space in which the sensor, the robot, and the target mark are arranged based on the environment information. A calibration device characterized by: In the calibration device according to claim 1,
The environment determination unit obtains an amount of rotation or an amount of movement necessary to match the virtual data with the sensor data,
The relative position/orientation estimating unit searches for the sensor parameter or the robot parameter that reduces the difference when the amount of rotation or the amount of movement is equal to or greater than a threshold, case, the calibration device does not search for the sensor parameter or the robot parameter that reduces the difference. In the calibration device according to claim 1 or claim 2,
The robot is an articulated robot arm,
The calibration device, wherein the robot parameter is a rotation angle of each joint of the multi-joint robot arm or an operating state of an end effector attached to the robot. In the calibration device according to claim 1 or claim 2,
the sensor is a camera;
The calibration device, wherein the sensor parameter is the position and orientation, focal length, distortion coefficient, or optical axis center of the camera. In the calibration device according to claim 1,
A calibration device, wherein the sensor parameters or the robot parameters input by a worker are used for searching by the relative position/orientation estimating unit.

Images