JP2019052983A - Calibration method and calibrator - Google Patents

Abstract

To provide a calibration method for reducing a scale error included in the measurement result of a three-dimensional visual measuring device and improving measurement accuracy.SOLUTION: The shape of a functional component or the shape of a jig coupled with the functional component is three-dimensionally measured by a measuring probe 102 of a three-dimensional measuring instrument 101, and a second coordinate system having origin in a reference portion of a functional component equipped with a stereo camera 104 is defined. The coordinate values, of a specific portion of the measuring probe 102 positioned at three or more measurement points in a range imageable by the stereo camera 104, in a first coordinate system in which are expressed the three-dimensional measured values of the stereo camera 104, are visually measured in three-dimensions. Meanwhile, calibration data for performing coordinate conversion between the first coordinate system and the second coordinate system and a correction matrix with which the three-dimensional measured coordinate values of the stereo camera 104 are correctable are acquired on the basis of the measured coordinate values, in the first and second coordinate systems, of the measurement points in the specific portion of the measuring probe 102.SELECTED DRAWING: Figure 1

Description

  The present invention is between a first coordinate system in which a three-dimensional measurement value of a three-dimensional visual measurement device is expressed and a second coordinate system determined from a reference part of a functional component including the three-dimensional visual measurement device. The present invention relates to a calibration method and a calibration apparatus for acquiring calibration data for performing coordinate transformation.

  In recent years, there has been a demand for an assembly production apparatus that realizes assembly work by a robot. In an assembly production apparatus using a robot, a technique for measuring the three-dimensional position / orientation of a part using a three-dimensional visual measurement device such as a stereo camera is known.

  When performing such three-dimensional measurement, if the three-dimensional visual measurement device is an imaging device such as a stereo camera, internal / external parameter calibration is performed on the camera in advance. Here, the internal parameter means optical characteristics such as the focal length and distortion characteristic of the lens, and the external parameter means the relative position and orientation of the two cameras in the stereo camera if the camera is a stereo camera. . If these internal / external parameters are determined and the measurement points in the captured images of the two cameras are associated, the three-dimensional coordinate values of the measurement points can be measured by the principle of triangulation.

  Here, the three-dimensional measurement when parallelizing the image captured by the stereo camera using the internal / external parameters is considered. This parallelization means that the lens distortion of the camera is corrected and converted into a captured image in a state where two cameras are installed in parallel. Here, if f is the focal length of the lens, B is the base length indicating the distance between the two lens principal points, and d is the parallax indicating the deviation of the measurement point position in the captured images of the two cameras, the Z direction (image The measurement result z in the depth direction) can be expressed as z = Bf / d. Of these, the focal length f corresponds to a part of the internal parameters, and the base line length (B) corresponds to a part of the external parameters.

  In order to measure a three-dimensional position, it is necessary to use highly accurate values as internal and external parameters. However, in order to directly measure the internal and external parameters with high accuracy, it is difficult to precisely measure the optical characteristics of the camera. Therefore, as described in Patent Document 1, the positions of a plurality of feature points are measured by two cameras, and an epipolar constraint condition is established for these feature points. A calibration method for calculating external parameters is generally used. Here, the epipolar constraint is a constraint condition that the corresponding points on the captured images of the two cameras of the stereo camera exist in the same plane (epipolar plane). The epipolar plane is a plane passing through the lens principal point and the measurement point of the two cameras.

JP-A-10-143666

  However, the internal / external parameters calculated by the method described in Patent Document 1 have an epipolar constraint that the positions of the feature points projected on the captured images of a plurality of cameras are on the same plane. It is a value calculated so as to hold.

  There may be an error between the calculated internal / external parameters and the actual true internal / external parameters. This is because an error occurs when the feature point position is image-measured at the time of calibration, or the lens distortion model that is a part of the internal parameters does not exactly match the actual lens distortion model. Therefore, there is a possibility that an error also occurs in the three-dimensional measurement using the three-dimensional visual measurement device due to the calculation error of the focal length of the internal parameter important in stereo measurement and the baseline length of the external parameter.

  In particular, if the calculated focal length is f ′ (≠ f) and the base line length is B ′ (≠ B), the measurement result z ′ in the Z direction is z ′ = B′f ′ / d = (B′f ′). / Bf) z. As described above, according to the conventional technique, in the three-dimensional measurement result of the three-dimensional visual measurement device, the Z component z ′ includes an error with respect to the actual true Z component z. The error in the depth (Z) direction (or photographing magnification) is hereinafter referred to as a scale error.

  In view of the above problems, an object of the present invention is to reduce a three-dimensional position measurement error including a scale error of a three-dimensional visual measurement device and to perform a three-dimensional visual measurement with high accuracy.

  In order to solve the above-described problem, in the present invention, a first coordinate system in which a three-dimensional measurement value of a three-dimensional visual measurement device is expressed, and a reference part of a functional component including the three-dimensional visual measurement device are determined. In a calibration method for acquiring calibration data for coordinate conversion between two coordinate systems, a position of the control device within the range that can be photographed by the three-dimensional visual measurement device and the position of the three-dimensional visual measurement device A mark installation step for positioning a mark whose coordinate value can be measured by the three-dimensional visual measurement device at a measurement position whose relationship is known in advance, and a control device in the first coordinate system by the three-dimensional visual measurement device A three-dimensional visual measurement step for measuring the coordinate value of the mark, and a control device calculates the coordinate value of the mark in a second coordinate system determined from a reference part of a functional component including the three-dimensional visual measurement device. A mark coordinate value calculating step, a coordinate value of the mark in the first coordinate system visually measured in the three-dimensional visual measurement step, and the second measured in the mark coordinate value measurement step. Using the coordinate value of the mark in the coordinate system, calibration data for performing coordinate conversion between the first coordinate system and the second coordinate system, and 3 measured by the three-dimensional visual measurement device A configuration including a correction matrix including an element for correcting the scale of the dimension measurement coordinate value and a calibration data calculation step for calculating the scale is adopted.

  According to the above configuration, the calibration data and the correction matrix including the element for correcting the scale of the three-dimensional measurement coordinate value are used to reduce the three-dimensional position measurement error including the scale error of the three-dimensional visual measurement device. A three-dimensional visual measurement device can accurately perform three-dimensional visual measurement.

It is explanatory drawing which showed the structure of the calibration apparatus concerning Embodiment 1-3 of this invention. It is explanatory drawing of the measurement probe concerning Embodiment 1-3 of this invention. It is explanatory drawing of the stereo camera and the positioning jig concerning Embodiment 1-3 of this invention. It is the flowchart figure which showed the whole calibration process of the three-dimensional visual measurement apparatus concerning Embodiment 1-3 of this invention. It is explanatory drawing of the stereo camera concerning Embodiment 1-3 of this invention. It is explanatory drawing of the parallel stereo camera concerning Embodiment 1-3 of this invention. (A)-(C) are explanatory drawings which showed the measuring method of the positioning coordinate system (M) concerning Embodiment 1-3 of this invention. It is explanatory drawing which showed the movement range of the measurement probe concerning Embodiment 1-3 of this invention. It is the flowchart figure which showed the calibration procedure of the three-dimensional visual measurement apparatus concerning Embodiment 1-3 of this invention. It is the flowchart figure which showed the calculation procedure of the correction | amendment matrix of the three-dimensional visual measurement apparatus concerning Embodiment 2 of this invention. It is explanatory drawing which showed the movement range of the measurement probe concerning Embodiment 2 of this invention. It is the flowchart figure which showed the calculation procedure of the correction | amendment matrix of the three-dimensional visual measurement apparatus concerning Embodiment 3 of this invention. (A), (B) is explanatory drawing which showed the end effector (hand) of the robot apparatus as a functional component provided with the stereo camera concerning Embodiment 5 of this invention. It is the flowchart figure which showed the calibration procedure of the three-dimensional visual sensor concerning Embodiment 5 of this invention. (A)-(C) are explanatory drawings which showed the measuring method of the finger coordinate system concerning Embodiment 5 of this invention. It is the block diagram which showed the structure of the calibration control system concerning each embodiment of this invention. It is explanatory drawing which showed the structure of the calibration apparatus using the mark installation apparatus concerning Embodiment 6 of this invention.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings. In addition, the structure shown below is an example to the last, For example, it can change suitably for those skilled in the art in the range which does not deviate from the meaning of this invention about a detailed structure. Moreover, the numerical value taken up by this embodiment is a reference numerical value, Comprising: This invention is not limited.

<Embodiment 1>
In the present embodiment (the same applies to the following embodiments), a three-dimensional visual measurement device is considered as the calibration target structure. This three-dimensional visual measurement apparatus is arranged at a predetermined part (predetermined position) of the robot apparatus or its working environment. The three-dimensional visual measurement device as a calibration target structure is a stereo camera (104) in the following embodiment, and is a functional component having a positioning unit for mounting on a robot device or a predetermined part (predetermined position) of its working environment. It is.

  In the calibration method of the present invention, the first coordinate system (C) in which the three-dimensional measurement value of the three-dimensional visual measurement device (stereo camera 104) is expressed, and the reference of the three-dimensional visual measurement device (functional component provided). Calibration data for coordinate transformation between the second coordinate system (M) determined from the part is acquired. That is, in the calibration systems of the first to sixth embodiments, the coordinate system used for the measurement of the three-dimensional visual measurement device and the positioning (mount) portion of the three-dimensional visual measurement device as the calibration target structure are used as a reference. Calibrate the coordinate system relationship (relative position and orientation).

  In particular, the first to fifth embodiments are premised on the configuration using the three-dimensional measuring machine 101 including the contact-type measurement probe 102. As will be described later in Embodiment 6, the measurement probe 102 photographed by the stereo camera 104 (three-dimensional visual measurement device) can be considered as a “mark” for calibration. That is, this measurement probe 102 may be read as a mark (7021: FIG. 17) in Embodiment 6 described later in Embodiments 1 to 5.

  In the first to fifth embodiments, the following control process is executed using the measurement probe of the coordinate measuring machine as the “mark”.

  The measurement probe 102 measures the shape of the functional component or the shape of the jig that can be mechanically coupled to the functional component, and defines the second coordinate system based on the three-dimensional measurement result (probe measurement) Process).

  A mark in which coordinate values can be measured by the camera (104) at a measurement position where the positional relationship with the camera (104) is known in advance within a range that can be photographed by the stereo camera 104 (three-dimensional visual measurement device). Then, the measurement probe 102 is positioned (mark setting step).

  The coordinate value of the mark in the first coordinate system is measured by the stereo camera 104 (three-dimensional visual measurement device) (three-dimensional visual measurement process).

  Calculate the coordinate value of the mark in the second coordinate system defined in the probe measurement step, which is determined from the reference part of the functional component including the stereo camera 104 (three-dimensional visual measurement device) (mark coordinate value calculation step).

  Using the respective coordinate values of the marks in the first coordinate system and the second coordinate system, the following correction data (calibration data) is calculated (calibration data calculation step). The correction data (calibration data) includes calibration data for coordinate conversion between the first and second coordinate systems, and three-dimensional measurement coordinate values measured by the stereo camera 104 (three-dimensional visual measurement device). And a correction matrix including an element for correcting the scale.

  FIG. 1 shows an example of the overall configuration of a calibration system for a three-dimensional visual measurement apparatus. In FIG. 1, the calibration system includes a three-dimensional measuring machine 101 using a contact-type measuring probe 102 on a surface plate 103.

  The three-dimensional measuring machine 101 can move the contact-type measurement probe 102 in the direction along the three axes (XYZ) by using an XYZ slider (not shown) arranged inside. The three-dimensional measuring machine 101 can measure the amount of movement of the XYZ slider with high accuracy by a position sensor (not shown) using, for example, laser light, and thereby measure the position of the measurement probe 102 with high accuracy. Is possible. A total of 21 geometric errors including straightness error (vertical, horizontal), rotation error (roll, pitch, yaw), indication error and right angle error of each axis of the measurement probe 102 (or its XYZ slider) are preliminarily determined by laser. A correction coefficient can be obtained by calculation using an interferometer or a gauge. Thereby, the position of the measurement probe 102 can be measured with high accuracy. In FIG. 1, the three-dimensional measuring machine 101 in which the holding portion of the measurement probe 102 (or the XYZ slider that supports the measuring probe) is configured in a gate shape is illustrated, but the mechanical configuration of the three-dimensional measuring machine 101 is illustrated. Is not limited to such a portal type. As long as the three-dimensional measuring machine 101 can measure the position of the measurement probe 102 with high accuracy, another mechanical structure such as an articulated arm type can be used.

  FIG. 2 shows the configuration of the measurement probe 102. The measurement probe 102 can be composed of a stylus 201 and a tip sphere 202. The tip sphere 202 to be brought into contact with the measurement object is attached to the tip of the stylus 201. For example, the stylus 201 can be configured to be detachable from the coordinate measuring machine 101. With such a detachable and replaceable configuration, for example, the stylus 201 (or the tip sphere 202) suitable for the optical performance of the three-dimensional visual sensor (stereo camera 104) and the shape of the positioning reference portion can be used.

  The tip sphere 202 is preferably made of a material such as zirconia that is non-transmissive and less glossy, and that can easily stabilize illumination conditions. However, the tip sphere 202 may be made of a material such as ruby, silicon nitride, ceramic, or other super hard material as long as it has characteristics suitable for measurement by the three-dimensional visual sensor as described above. The tip sphere 202 is preferably a sphere, for example, and its roundness and dimensions are preferably controlled in advance to a specific accuracy range at the time of manufacture. In the present embodiment, only the case where the tip sphere 202 is a sphere will be described. However, the calibration control described later can be performed with other shapes such as a hemisphere or a disk shape.

  In the three-dimensional measuring machine 101 of the present embodiment, a pressure sensor 203 is built in the mounting portion of the measurement probe 102. For example, the timing at which the tip sphere 202 contacts the object can be detected via the pressure sensor 203. By using this contact timing signal as a trigger signal, the reference position (for example, the center position) of the tip sphere 202 at the moment when the tip sphere 202 contacts the object can be measured.

  In this calibration system, a positioning jig 105 and a ring illumination 106 are installed on the surface plate 103 of the three-dimensional measuring machine 101, and a stereo camera 104 is installed on the positioning jig 105 as a three-dimensional visual measurement device. The three-dimensional measuring machine 101 may be dedicated to the calibration of the three-dimensional visual measuring device (stereo camera 104), and can be used for other purposes than calibrating the three-dimensional visual measuring device (stereo camera 104). It may be configured as a product.

  FIG. 3 shows a configuration of a stereo camera 104 and a positioning jig 105 as a three-dimensional visual measurement device (three-dimensional visual sensor). As shown in FIG. 3, the stereo camera 104 includes, for example, monocular cameras 301 and 302 arranged with their imaging optical axes spaced apart by a predetermined baseline length. In this embodiment, it is considered that the passive stereo camera 104 is used as the three-dimensional visual sensor. However, a three-dimensional visual sensor using a measurement method such as an active optical laser method or an active stereo method may be used.

  On a predetermined portion of the stereo camera 104, for example, on the back side of the casing, a plurality of positioning pins 303, 303 serving as positioning reference portions of the structure to be calibrated (in this embodiment, the three-dimensional visual sensor, that is, the stereo camera 104), At least two are arranged. Three or more positioning pins may be arranged. In this case, the positions and the number of positioning holes 304, 304... Of the positioning jig 105 described below are determined corresponding to the positions and the number of positioning pins of the stereo camera 104.

  On the other hand, in the positioning jig 105 arranged on the surface plate 103, positioning holes 304 and 304 are drilled at positions corresponding to the positioning pins 303 of the stereo camera 104, respectively. With such a fitting structure of the positioning pin 303 and the positioning hole 304, the stereo camera 104 and the jig 105 can be mechanically coupled, and the stereo camera 104 can be positioned in the X and Y directions with high accuracy. Also, positioning in the Z direction is performed by bringing the lower surface of the housing of the stereo camera 104 into contact with the upper surface of the positioning jig 105. For this reason, it is preferable that the flatness of the bottom surface of the stereo camera 104 and the top surface of the positioning jig 105 is high.

  The stereo camera 104 can be arranged at a predetermined part of the robot apparatus or its working environment. In this case, the stereo camera 104 may have a positioning unit for mounting at a predetermined part (predetermined position) of the robot apparatus or its working environment. preferable. For example, when the stereo camera 104 is used as a vision system of a robot apparatus, a camera mount 1104 having the same positioning holes 304 and 304 as the positioning jig 105 shown in FIG. 3 is part of the robot apparatus R (robot arm). Deploy. The robot apparatus R is indicated by broken lines in FIG. 3 corresponds to, for example, a housing such as a link on the hand side or an end effector such as a hand, and a camera mount 1104 provided with positioning holes 304 and 304 is disposed in a part thereof. .

  In the present embodiment, it is assumed that the coordinate system determined from the shape of the structure to be calibrated is the positioning coordinate system M determined from the positioning reference portion. As shown in FIG. 3, the XY plane of the positioning coordinate system M is assumed to coincide with the lower surface of the stereo camera 104 housing. The origin of the positioning coordinate system M is the intersection of the central axis of the left positioning pin 303 and the lower surface of the stereo camera 104 housing in FIG. Further, the X axis is a direction from the left positioning pin 303 to the right positioning pin 303. The Y axis is orthogonal to the X axis at the origin of the positioning coordinate system M, and is a direction from the front of the drawing to the back. The origin of the positioning coordinate system M (second coordinate system) may be arranged at a position that is uniquely determined from the shape and size of the reference portion (positioning pin 303) of the stereo camera 104 that is a functional component. For example, the origin of the positioning coordinate system M (second coordinate system) may be arranged on the camera bottom surface at a position where the distance between the centers of the two positioning pins 303 is equally divided.

  In the present embodiment, the positioning reference portion is arranged directly on the housing (the lower surface thereof) of the stereo camera 104, but it may have a separate structure from the housing. In this embodiment, the positioning pins 303 and 303 are used as the positioning reference portion. However, a configuration in which the side surface of the case of the stereo camera 104 is used as a reference surface and pressed against an appropriate positioning wall surface is used. It may be used. In this case, the side surface of the housing of the stereo camera 104 can be directly measured by the measurement probe 102, and thus there is a possibility that the positioning jig 105 may not be used.

  Although not shown, it is preferable to provide the positioning jig 105 with a mechanism that can fix the stereo camera 104 with screws or clamps so that the stereo camera 104 does not move during calibration. At this time, for example, by adopting a structure that is fixed while applying a force in one direction with a spring or the like, pin fitting error can be reduced, and more accurate calibration can be performed.

  The ring illumination 106 is used when photographing the tip sphere 202 of the stylus 201 with the stereo camera 104. Since the tip sphere 202 can be illuminated symmetrically by using the ring illumination 106, the measurement accuracy of the stereo camera 104 can be improved. In particular, low-angle ring illumination that can uniformly illuminate the outer periphery of the tip sphere 202 is preferable.

  An example of the configuration of the control system of the calibration system of FIG. 1 is shown on the right side of the figure. For the sake of convenience, in this embodiment, the control system of the calibration system is configured by a three-dimensional measuring machine controller 107, a vision controller 113, and a measurement controller 700. The measurement controller 700 mainly constitutes a user interface of the present calibration system, and includes at least a display unit 122 and an operation unit 123 necessary for calibration processing operation.

  However, the configuration of the control system of the calibration system is not necessarily limited to the configuration shown in FIG. For example, in the configuration of the present embodiment, the measurement controller 700, the three-dimensional measurement device controller 107, and the vision controller 113 are made independent from each other in consideration of a form in which the portion of the three-dimensional measurement device can be used for purposes other than calibration. It is. However, in the case of a system dedicated to calibration, the control system of the calibration system is a control device in which a measurement controller 700 having the following functions, a three-dimensional measuring machine controller 107, and a vision controller 113 are integrated. It may be configured.

  The three-dimensional measuring machine controller 107, the vision controller 113, and the measurement controller 700 are composed of the following functional blocks, for example. More specific configurations of these controllers will be described later with reference to FIG.

  The three-dimensional measuring machine controller 107 is a controller that controls the three-dimensional measuring machine 101 and the measurement probe 102. The three-dimensional measuring machine controller 107 includes a probe control unit 108, a nonvolatile memory 109, a slider drive unit 110, a probe position measurement unit 111, a pressure measurement unit 124, a temperature measurement unit 120, and a communication unit 112. The coordinate measuring machine controller 107 is a computer, and each unit in the coordinate measuring machine controller 107 may be configured by hardware or a circuit, or may be configured by software.

  The probe control unit 108 has a function of controlling each unit in the coordinate measuring machine controller 107 in accordance with a control program stored in the non-volatile memory 109 and a command from the slider operation unit 121. For example, the procedure of controlling the position of the measurement probe 102, reading the current position of the tip sphere 202 center, and transmitting the current position to the vision controller 113 is controlled.

  The nonvolatile memory 109 stores a stylus calibration result, a geometric tolerance correction function of the CMM 101, and a control program for calibration control described later. Naturally, the control program is stored in advance in the nonvolatile memory 109 prior to performing the calibration process.

  The slider driving unit 110 has a function of moving the sliders in the X, Y, and Z directions in the coordinate measuring machine 101 by an amount instructed by the probe control unit 108 and moving the measurement probe 102 to an arbitrary position.

  The probe position measuring unit 111 has a function of measuring the current position of the tip sphere 202 center of the stylus 201. The center position of the tip sphere 202 is calculated using the measurement results of the X, Y, and Z slider positions in the coordinate measuring machine 101, the geometric tolerance correction function of the coordinate measuring machine 101, and the stylus calibration result. Each slider is equipped with a temperature sensor, and the temperature of each slider can be measured via the temperature measurement unit 120. The probe position measurement unit 111 measures the temperature in the vicinity of each slider via the temperature measurement unit 120, and performs temperature compensation correction for correcting the center position of the tip sphere 202 using this temperature information. By this temperature compensation correction, the center position of the tip sphere 202 can be measured with higher accuracy.

  The pressure measurement unit 124 has a function of reading the measurement value of the pressure sensor 203 in the measurement probe 102. When the measured value exceeds a preset threshold value, a trigger signal is transmitted to the probe position measuring unit 111. Thereby, the current position of the center of the tip sphere 202 at the moment when the measurement probe 102 comes into contact can be measured.

  The communication unit 112 has a function of transmitting / receiving data to / from the communication unit 117 in the vision controller 113. The current position of the center of the tip sphere 202 of the measurement probe 102 is transmitted from the coordinate measuring machine controller 107 to the vision controller 113.

  The slider operation unit 121 is a controller for moving the measurement probe 102 to an arbitrary position. It is possible to operate the sliders in the XYZ directions separately.

  The vision controller 113 is a controller that controls the stereo camera 104 and the ring illumination 106 and performs a calibration calculation. The vision controller 113 is a computer, and each unit in the vision controller 113 may be configured by hardware or a circuit, or may be configured by software.

The vision control unit 115 in the vision controller 113 has a function of controlling each unit in the vision controller 113 in accordance with a control flow stored in the nonvolatile memory 116 and a command from the operation unit 123. Specifically, the procedure for controlling the shooting timing of the stereo camera 104, passing the shot image to the image processing unit 118, processing the image, and transmitting the current position of the tip sphere 202 center to the volatile memory 114 is controlled. To do. In the present embodiment, the vision controller 113 has a function of calculating a relative position / attitude between the camera coordinate system C and the positioning coordinate system M as described later. The vision controller 113 uses the calculation result (calibration data ( ^M T _C ) and correction matrix (A) of this embodiment) for performing coordinate conversion between the camera coordinate system C and the positioning coordinate system M as: For example, it can be stored in the nonvolatile memory 116. The nonvolatile memory 116 is used to store a calibration result between the camera coordinate system C and the positioning coordinate system M and a vision control procedure to be described later for performing control around the vision.

  The volatile memory 114 is composed of a storage device such as a DRAM or SRAM, and has a function of temporarily storing a measurement result of the center position of the tip sphere 202. The nonvolatile memory 116 and the volatile memory 114 are included in the storage device of the present invention. For example, in the present embodiment, data is temporarily stored in the volatile memory 114, but the same data may be stored in the nonvolatile memory 116.

  The imaging unit 119 has a function of transmitting a trigger command to the stereo camera 104 and the ring illumination 106 and acquiring a captured image from the stereo camera 104.

  The image processing unit 118 has a function of performing image processing on the image data transmitted from the stereo camera 104. Specifically, the center position of the tip sphere 202 is measured. In the present embodiment, the tip sphere 202 is, for example, a white sphere (white sphere).

  The outline of the measurement processing of the center position of the white sphere is as follows. First, the image processing unit 118 performs edge extraction processing on the captured image data of the monocular cameras 301 and 302 inside the stereo camera 104. Then, the edge of the tip sphere 202 is extracted from the length and roundness of each edge. Ellipse approximation is performed on the extracted edge to obtain image coordinates of the ellipse center. Stereo measurement is performed using the parallax obtained from the ellipse center of each image and the calibration result of the stereo camera 104, and the position of the tip sphere 202 center in the camera coordinate system C is obtained.

  The communication unit 117 has a function of communicating with the coordinate measuring machine controller 107. A movement command for the measurement probe 102 is transmitted from the vision controller 113 to the coordinate measuring machine controller 107. The measurement controller 700 includes a display unit 122 that displays captured images and calibration results, and an operation unit 123 that inputs information. The display unit 122 includes a display device using a display device such as an LCD, and the operation unit 123 includes a UI (user interface) device such as a mouse and a keyboard.

  Here, FIG. 16 shows an example of a more specific configuration for realizing the coordinate measuring machine controller 107, the vision controller 113, or the measurement controller 700 of FIG.

  In the configuration of FIG. 16, a ROM 602, a RAM 603, an HDD 604, and various interfaces 605 to 607 are arranged around a CPU 601 that functions as a main control unit.

  Connected to the CPU 601 are a ROM 602, a RAM 603, an HDD 604, and various interfaces 605 to 607. The ROM 602 stores basic programs such as BIOS. In the case of the coordinate measuring machine controller 107, the ROM 602 may include a device such as E (E) PROM that constitutes the nonvolatile memory 109 of FIG.

  The RAM 603 is a storage device that temporarily stores the calculation processing result of the CPU 601. The storage area of the volatile memory 114 of the vision controller 113 in FIG. 1 can be implemented by the RAM 603, for example.

  The HDD 604 can be arranged as an external storage device. Although not necessarily essential, the HDD 604 constitutes a storage unit that stores various data and the like that are calculation processing results of the CPU 601. The HDD 604 can also be used as the nonvolatile memory 109 used by the coordinate measuring machine controller 107 described above. Further, the HDD 604 can store a file in which a program for causing the CPU 601 to execute various arithmetic processes is recorded. The CPU 601 executes a control procedure described later by executing a program recorded (stored) in the ROM 602 or the HDD 604.

  When a program for executing a control procedure to be described later is recorded (stored) in the ROM 602 or the HDD 604, these recording media constitute a computer-readable recording medium storing a control procedure for carrying out the present invention. A program for executing a control procedure to be described later is stored in a fixed recording medium such as the ROM 602 or the HDD 604, and also on a removable computer-readable recording medium such as various flash memories or optical (magnetic) disks. It may be stored. Such a storage form can be used when a program for executing a control procedure for carrying out the present invention is installed or updated. In addition, when installing or updating a program for executing a control procedure, a method of downloading a program via a network (not shown) can be used in addition to the removable recording medium as described above.

  The CPU 601 can perform network communication via the interface 605. The interface 605 can be configured by various network communication methods such as a wired connection (such as IEEE 802.3) and a wireless connection (such as IEEE 802.xx). With the interface 605, for example, the CMM controller 107 of FIG. 1 or the communication units 112 and 117 of the vision controller 113 can be implemented.

  Further, the CPU 601 can communicate with other resources on a network (not shown) connected via the interface 605 and performing communication using a protocol such as TCP / IP. As another device (not shown) on such a network, an upper server for controlling the overall operation of the calibration system may be considered. In this case, the CPU 601 can transmit operation information such as log information related to the calibration processing in real time to the host server via the interface 605 to the network (not shown). Alternatively, a control program related to production control described later can be downloaded from the server and installed in the ROM 602 or the HDD 604, or an already installed program can be updated to a new version.

  The interfaces 606 and 607 can be configured based on various serial or parallel interface standards, for example. The interfaces 606 and 607 can be used in the coordinate measuring machine controller 107 of FIG. 1 to communicate with a controlled unit including a driving source such as the slider driving unit 110. The interfaces 606 and 607 can be used to read measurement information from measurement (sensor) units indicated by reference numerals 111, 120, and 124 in the coordinate measuring machine controller 107 of FIG. 1.

  In the vision controller 113 of FIG. 1, the interfaces 606 and 607 can be used to read measurement (image) information from an imaging sensor or the like included in the imaging unit 119.

  Further, in the case of the measurement controller 700 of FIG. 1 that mainly constitutes a user interface in this calibration system, the interfaces 606 and 607 are used for input / output of display information and operation information to the display unit 122 and the operation unit 123. it can.

  Hereinafter, a calibration method of the stereo camera 104 (three-dimensional visual measurement device) will be described with reference to the flowchart of FIG. In the calibration procedure of FIG. 4, first, calibration work for obtaining internal parameters and external parameters of the stereo camera 104 is performed (S401). Internal / external parameters are stored in a non-volatile memory 116 in the vision controller 113.

FIG. 5 shows the stereo camera 104 (three-dimensional visual measurement device) of this embodiment and each coordinate system related to the calibration process. In FIG. 5, a stereo camera 104 (three-dimensional visual measurement device) is composed of two monocular cameras 301 and 302 each having image sensors 501 and 502 as imaging means. The origins of the camera coordinate systems C ₁ and C _r are taken, for example, at the principal point positions of the imaging optical systems of the monocular cameras 301 and 302. Relative positions and postures of the sensor coordinate systems S _l and S _r determined on the image sensors 501 and 502 and the camera coordinate systems C _l and C _r are determined by internal parameters of the monocular cameras 301 and 302 constituting the stereo camera 104. . The internal parameters of the monocular cameras 301 and 302 include geometry information related to shooting characteristics such as focal length, image center, and lens distortion.

On the other hand, in this type of stereo imaging system, the external parameters are parameters that define the relative position / posture between the monocular cameras 301 and 302. Thus, the relative position and orientation between the camera coordinate system _{C l} and _{C r} of the monocular camera 301 and 302 is obtained.

  Calibration of the internal / external parameters of the stereo camera 104 is performed through photographing of a calibration subject in the case of a general method as described in Patent Document 1 described above, for example. For example, a plurality of calibration subjects to which an image pattern having a known size and shape is added are photographed by the stereo camera 104 while changing the position and orientation of the target. Then, image processing is performed on the captured images of the monocular cameras 301 and 302, and the position of the feature point on the calibration subject pattern on the image is obtained. At this time, in theory, the condition (epipolar constraint) that the same feature point exists on the same plane (epipolar plane) on the captured image of each monocular camera is satisfied. The internal / external parameters of the stereo camera 104 can be calculated, for example, through optimization calculation so that the epipolar constraint is satisfied. As the calibration subject, a chess board or a calibration plate provided with a checkered image pattern, which is prepared to have a certain accuracy, is used.

  However, errors may occur in internal and external parameters due to the following factors. First, when the pattern feature point position is measured by image processing, a reading error caused by a quantization error occurs. In the real world, a compound lens using a plurality of lenses is used, and it is extremely difficult to express the lens distortion only by a simple model such as a polynomial model. In particular, in the case of a lens having a large distortion such as a wide-angle lens, there is a possibility that the influence of a modeling error becomes large.

Using the internal / external parameters of the stereo camera 104, the position of the sensor coordinate system S ₁ , S _r based on the captured image is determined based on the principle of triangulation, for example, the three-dimensional position based on the left camera coordinate system C ₁ Stereo measurement, such as conversion to, becomes possible. As described above, the camera coordinate system C and the left camera coordinate system Cl of the stereo camera 104 may be treated as equivalent for convenience. For example, when the imaging optical axes of the monocular cameras 301 and 302 are arranged in parallel and a parallel stereo as shown in FIG. 6 is configured, the stereo measurement result of the Z coordinate of the measurement point P can be expressed by the following equation. .

  Here, f is a focal length, B is a baseline length that is a distance between lens principal points of the monocular cameras 301 and 302, and d is a parallax that is a difference in feature point positions on the captured images of the monocular cameras 301 and 302. . f is determined from an internal parameter, and B is determined from an external parameter. From the above formula (1), it can be seen that the calibration accuracy of the internal and external parameters greatly affects the stereo (three-dimensional) measurement accuracy.

  Next, in step S402 in FIG. 4, the stylus 201 of the three-dimensional measuring machine 101 is calibrated. Here, the stylus length, the spherical diameter of the tip sphere 202, and the like are calculated. By using the stylus calibration result and the amount of movement of each slider of the three-dimensional measuring machine 101, the current position of the tip sphere 202 center can be accurately measured.

  The stylus calibration in step S402 is performed as follows, for example. For example, a reference sphere having a desired sphericity is fixed on the surface plate 103, and contact measurement is performed by the measurement probe 102 on a plurality of locations on the surface of the reference sphere. The measurement result is compared with the design value of the reference sphere to obtain stylus calibration data.

  Next, in step S403 in FIG. 4, a measurement point by the measurement probe 102 for defining the positioning coordinate system M (second coordinate system) of the stereo camera 104 is determined (probe measurement process).

  As shown in FIG. 3, the stereo camera 104 of the present embodiment has a positioning pin 303 as a positioning reference portion, and is fixed to, for example, a robot arm (R) by the positioning pin 303. For example, the origin of the positioning coordinate system M (second coordinate system) is the intersection of the central axis of the left positioning pin 303 and the lower surface of the case of the stereo camera 104 as shown.

  In this case, the front surface of the housing of the stereo camera 104 is measured with the stylus 201, and the origin of the positioning coordinate system M (second coordinate system) arranged at the reference portion (one of the positioning pins 303) is specified. It may be a thing. In that case, the working accuracy of the housing of the stereo camera 104 and the positioning pins 303 needs to be sufficiently maintained.

  In this embodiment, when measuring the front surface of the case of the stereo camera 104 with the stylus 201, the stereo camera 104 is placed on the surface plate 103 of the three-dimensional measuring machine 101 via the positioning jig 105 as shown in FIG. Arrange at a predetermined position. This positioning jig 105 is the same as that actually used for mounting a specific stereo camera 104, or a reference member created with sufficient work accuracy.

  In the present embodiment, for example, as shown in FIGS. 7A to 7C, a method of measuring the positioning hole 304 of the positioning jig 105 with the measurement probe 102 is employed. In this case, it is a condition that the working accuracy of the positioning pin 303 and the positioning hole 304 of the positioning jig 105 is sufficiently maintained.

  In order to determine the XY plane of the positioning coordinate system M, the operator uses the slider operation unit 121 to place three or more tip spheres 202 of the measurement probe 102 on the upper surface of the positioning jig 105 as shown in FIG. (201a, 201b, 201c...) Are brought into contact. At this time, the probe control unit 108 in the coordinate measuring machine controller 107 stores the probe position at the moment of contact (probe measurement process).

  Next, as shown in FIGS. 7B and 7C, the tip sphere 202 of the measurement probe 102 is brought into contact with three or more points inside the two positioning holes 304 and 304, and the probe position at the moment of contact is determined. Remember. The points projected on the XY plane of the positioning coordinate system M for which the measured values at the time of contact are obtained are obtained, and circular approximation is performed on these projected points to measure the center position of the positioning hole 304. As described above, the origin of the positioning coordinate system M (FIG. 5) and the orientations of the X and Y axes can be acquired.

  In this embodiment, in step S403, only the measurement points are taught by the measurement probe 102, and the positioning coordinate system M based on the actual measurement of the measurement probe 102 is defined in step S901 (FIG. 9) described later. And However, in step S403, not only the measurement point teaching by the measurement probe but also the actual three-dimensional shape measurement may be performed up to the definition of the positioning coordinate system M.

  Next, in step S404 in FIG. 4, the position of the stylus 201 when the stereo camera 104 captures the tip sphere 202 of the stylus 201 is determined. Here, the movement area of the stylus 201 will be described with reference to FIG. The moving area of the stylus 201 preferably covers the common visual field area OV of the monocular cameras 301 and 302 of the stereo camera 104. The common visual field region OV is a region in which a subject can be photographed in a state where both the monocular cameras 301 and 302 are in focus.

  For example, the so-called depth of field of the monocular cameras 301 and 302 corresponds to a truncated pyramid-shaped space range, and the common visual field region OV is a region where stereo measurement can be performed with high accuracy. As described above, the use of the imaging information of the entire visual field common to the monocular cameras 301 and 302 reduces the lack of error information with respect to each measurement position, so that more accurate calibration can be realized.

  Note that the movement region of the stylus 201 of the measurement probe 102 may be automatically determined from the optical characteristics of the stereo camera 104 and the calibration result. Specifically, the camera coordinate system C is used as a reference based on the sensor size, focal length and work distance of each monocular camera 301 and 302 in the stereo camera 104, and external parameters which are relative positions and orientations between the monocular cameras 301 and 302. The common field of view. Further, a common visual field area based on the positioning coordinate system M is obtained using design values of relative positions and orientations between the camera coordinate system C and the positioning coordinate system M. In step S404, within the common visual field determined as described above, the probe control unit 108 automatically moves a plurality of styluses 201 of the measurement probe 102, preferably three or more moving positions. Then, the tip sphere 202 of the stylus 201 is photographed at each position. That is, the stylus 201 is moved to three or more measurement points within a range that can be satisfactorily photographed by the stereo camera 104, and the tip sphere 202 is photographed at each position.

  The processing so far corresponds to the advance preparation of this embodiment. Steps S405 (measurement of the position of the tip sphere 202 of the stylus 201 by the stereo camera 104) and S406 (calculation data and correction matrix calculation: calibration data acquisition step) in FIG. 4 correspond to narrowly defined calibration processing in the present calibration system. To do.

  Here, FIG. 9 shows a detailed flow of processing executed in the measurement of the center position of the tip sphere 202 in the camera coordinate system C and the positioning coordinate system M (S405) and the correction function calculation of the stereo measurement result (S406). In FIG. 9, steps S901 to S907 correspond to step S405 in FIG. 4, and step S908 corresponds to step S406 in FIG.

  The process of FIG. 9 is automatically performed when the operator presses the calibration start button on the operation unit 123, for example. First, in step S901 in FIG. 9, the coordinate measuring machine controller 107 automatically scans the measurement point taught in step S403 with the tip sphere 202 of the stylus 201 and performs contact measurement on the positioning jig 105 (probe measurement). Process). Using this measurement result, the positioning coordinate system M (second coordinate system) of the stereo camera 104 is defined. If the accuracy of both the positioning pin 303 and the positioning hole 304 is sufficient, the center position of the positioning hole 304 can be handled as the center position of the positioning pin 303. In this way, the probe control unit 108 of the coordinate measuring machine controller 107 can define the position of the origin of the positioning coordinate system M (FIG. 3: second coordinate system) and the direction of the X axis (Y axis). . Thereby, the coordinate measuring machine controller 107 can measure the center position of the tip sphere 202 of the stylus 201 with the positioning coordinate system M as a reference, for example.

  Next, in step S <b> 902, the operator installs the stereo camera 104 on the positioning jig 105 and the ring illumination 106 on the surface plate 103.

  In the loop of steps S903 to S907, the stylus 201 is moved to each of three or more measurement points arranged in the common visual field region OV of the stereo camera 104 in step S404 of FIG. Then, the three-dimensional visual measurement of the tip sphere 202 of the measurement probe 102 is performed by the stereo camera 104 at each measurement point.

  First, in step S903, the vision control unit 115 transmits a probe movement command to the coordinate measuring machine controller 107. Upon receipt of the command, the probe control unit 108 moves the tip sphere 202 of the stylus 201 to the measurement point set in advance in the stylus movement position determination process (S404 in FIG. 4). Subsequently, when the movement of the stylus 201 is completed in step S904, the current position of the center of the tip sphere 202 based on the positioning coordinate system M is measured via the probe position measuring unit 111. The measurement value is transmitted to the vision controller 113. In response to this, the vision control unit 115 in the vision controller 113 stores the received center position of the tip sphere 202 in the volatile memory 114.

  Next, in step S905, the vision control unit 115 of the vision controller 113 transmits a shooting command to the stereo camera 104 and the ring illumination 106 via the shooting unit 119 to cause the tip sphere 202 to be shot. As a result, the stereo image captured by the monocular cameras 301 and 302 is transmitted to the image processing unit 118. In step S906, the vision control unit 115 performs a three-dimensional measurement process, and acquires the coordinate value of the center position of the tip sphere 202 photographed in the camera coordinate system C. The coordinate value of the center position of the tip sphere 202 is stored in the volatile memory 114 (S906).

  Next, in step S907, it is determined whether or not the movement to all measurement points determined in the movement position determination process (S404) of the stylus 201 has been completed. If the movement to all measurement points and the photographing by the stereo camera 104 have not been completed, the process returns to step S903, the stylus 201 is moved to the next movement position, and the processes of steps S903 to S906 are performed again. If the movement to all the measurement points and the photographing have been completed in step S907, the process proceeds to the stereo measurement result correction function calculation process in step S908.

  In the correction function calculation process in step S908, data on the center position of the tip sphere 202 in the positioning coordinate system M obtained in step S904 and the center position of the tip sphere 202 in the camera coordinate system C obtained in step S906 are used. Ideally, the relationship of the following equation (2) is established for these data.

  Here, T is a homogeneous coordinate transformation matrix from the coordinate system of the upper left subscript to the coordinate system of the lower right subscript. The homogeneous coordinate transformation matrix is expressed by the following equation (3) using a rotation matrix (3 degrees of freedom) and a translation matrix (3 degrees of freedom).

  However, in reality, it is difficult to express the relationship between the positioning coordinate system M and the camera coordinate system C only by the relational expression such as Expression (2). This is because the Z coordinate scale of the positioning coordinate system M and the camera coordinate system C may be different.

  Here, the case of parallel stereo as shown in FIG. 6 will be described as an example. In general, the internal and external parameters of a stereo camera are values calculated so that epipolar constraints are established, and are different from the actual focal length and base line length determined optically. Here, it is assumed that the true focal length is f, the base length is B, the focal length obtained from internal and external parameters is f '(≠ f), and the base length is B' (≠ B).

  Since the captured image of the stereo camera 104 is optically projected onto the image sensor through the lens, the captured image is an image depending on the true focal length (f) and the base length (B). When image processing is performed on the captured image and the parallax at the measurement point P (x, y, z) is calculated, the parallax d can be ideally expressed as the following equation (4).

  When performing stereo measurement using the parallax obtained by the above equation, the focal length and the base line length are values obtained from internal and external parameters. Accordingly, the Z coordinate z ′ of the measurement point P obtained by stereo measurement is expressed by the following equation (5).

  From the above equations (4) and (5), the physical true Z coordinate z of the measurement point P and the Z coordinate z ′ obtained by stereo measurement have a relationship as shown in the following equation (6).

  As shown in the above equation (6), it can be seen that the scale of the physical true Z coordinate z and the Z coordinate z ′ of the stereo measurement result are different. Here, since the scale of the positioning coordinate system M can be measured with high accuracy using the three-dimensional measuring machine 101, it is considered to be equivalent to a physical scale. On the other hand, the coordinate system determined from the camera coordinate system C and the stereo measurement is the same coordinate system. As described above, it can be seen that the Z coordinate scales of the positioning coordinate system M and the camera coordinate system C are different.

  Further, the reason why it is difficult to express only by the relational expression such as Expression (2) is that it is difficult to accurately model lens distortion. In general, the lens distortion is approximated by a polynomial model. However, in the real world using a compound lens assembled with a plurality of lenses, it is difficult to represent the distortion curve with a monotonous model. Since the influence of the distortion modeling error changes depending on the measurement position, it is generally difficult to express only with the relational expression such as Expression (2). In particular, in a lens having a large lens distortion, this modeling error is large, and the influence on the stereo measurement accuracy increases.

  From the above situation, the relational expression between the center position of the tip sphere 202 in the positioning coordinate system M and the data of the center position of the tip sphere 202 in the camera coordinate system C is corrected as follows.

  This correction matrix (A) is a matrix for correcting the stereo measurement result in this embodiment, and this correction matrix (A) includes an element for correcting the scale of the three-dimensional measurement coordinate value. Here, when the above equation (7) is deformed, it can be expressed as the following equation (8).

  Here, the center position of the tip sphere 202 in the positioning coordinate system M and the center position of the tip sphere 202 in the camera coordinate system C are known because they are measured in steps S904 and S906 in FIG.

Next, the flow of calculating the correction matrix A for the stereo measurement result will be described with reference to FIG. In the present embodiment, a case where only the scale correction of the stereo measurement result is performed will be described. In step S1001 of FIG. 10, the correction matrix A is modeled as a scale conversion matrix. When the scale function of the Z coordinate is m _z , the correction matrix A can be expressed as the following equation (9).

In the above formula (9), only the scale function of the Z coordinate is considered, but the scale function of the X and Y coordinates may be incorporated. This is because the stereo measurement accuracy in the X and Y directions depends on the measurement accuracy in the Z direction, so that not only the Z direction but also the scales in the X and Y directions are different. X, function scale function of the Y coordinate _m x, When _{m y,} the correction matrix A can be expressed as the following equation (10). Since all the scales of the X, Y, and Z coordinates are corrected by the correction matrix A, calibration can be performed with higher accuracy.

The above m _x , m _y , and m _z may be constants, or variables depending on the distance from the center of the photographing optical axis and the stereo measurement result may be used. In the case of a constant, since correction is simple, the correction time of the stereo measurement result may be shortened.

  After defining the model of the correction matrix A in this way, in step S1002 of FIG. 10, the homogeneous coordinate transformation matrix is set so that the square sum of the norm of the equation (8) approaches 0 as shown in the following equation (11). The nonlinear optimization calculation is solved for the MTC and the correction matrix A. Thereby, the correction matrix A can be calculated with high accuracy. Here, N is the total number of movements of the measurement probe 102 (the number of imaging performed by moving the measurement probe 102).

The stereo measurement result correction matrix A and the homogeneous coordinate transformation matrix ^M T _C calculated in this way are stored in the nonvolatile memory 116 in the vision controller 113 (S1003). When the stereo camera 104 is actually installed and used in a robot or the like, highly accurate three-dimensional measurement is performed by performing scale correction using the correction matrix A of the present embodiment on the three-dimensional measurement result measured by the stereo camera 104. Is possible.

  In the present embodiment, the calibration system using the three-dimensional measuring machine 101 having the contact type measurement probe 102 has been described. However, a jig system processed with high accuracy may be used. Specifically, an object such as a mark (such as a checkerboard) that can be measured with a stereo camera with high accuracy is positioned with high accuracy, for example, at a different position in the Z direction with respect to the positioning reference portion. Or it may be adjusted.

  However, when the three-dimensional measuring machine 101 is used as in the present embodiment, the position of the functional component provided with the three-dimensional visual measurement device or the positioning portion of the positioning jig can be directly measured. There is an advantage that the jig may be set roughly. Further, when the three-dimensional measuring machine 101 is used, there is an advantage that calibration can be similarly performed with rough settings even if the shape of the functional component including the three-dimensional visual measuring device changes. That is, by using the three-dimensional measuring machine 101 as in the present embodiment, it is possible to reduce the number of steps for position adjustment between the functional component and the measurement probe, greatly shorten the calibration time, and acquire calibration data with high accuracy. effective.

  The method for calculating the correction matrix A of one stereo measurement result using the measurement position data of the tip sphere 202 of the measurement probe 102 of the three-dimensional measuring machine 101 has been described above. However, when the measurement range of the stereo camera 104 is wide, the influence of various error factors such as a lens distortion modeling error and an image quantization error becomes large, and high-precision correction becomes difficult.

  Therefore, as shown in FIG. 11, it is conceivable to divide the common visual field region OV of the monocular cameras 301 and 302 constituting the stereo camera 104 into a plurality of local regions 1101 and 1102. Then, using the position information of the tip sphere 202 in each of the local regions 1101 and 1102, a correction matrix A for performing a stereo measurement result for each local region is calculated. By such a local correction data calculation step, the correction matrix A optimized for each of the local regions 1101 and 1102 can be acquired, and a three-dimensional measurement result (three-dimensional measurement value) corrected with higher accuracy can be obtained. . In the example of FIG. 11, the common visual field OV is divided into two local regions 1101 and 1102. In the present embodiment, the common visual field OV is divided into two, but the common visual field OV is divided into a larger number than that. The correction matrix A may be obtained and used separately for each imaging range. In this case, for example, an experiment may be performed in which the division of the local regions 1101 and 1102 is repeated while verifying the stereo measurement accuracy, and the division number and the division position of the local regions 1101 and 1102 may be determined. By such a procedure, the division of the local area can be repeated, and the division mode of the local area can be determined so that acceptable stereo measurement accuracy can be obtained in the hardware of the stereo camera 104 concerned.

In addition, by using the homogeneous coordinate transformation matrix ^M T _C as a unit matrix, three-dimensional measurement in the camera coordinate system C with the scale corrected correctly becomes possible. In that case, even if the positioning reference portions (303, 303) are removed from the stereo camera 104 and used, highly accurate three-dimensional measurement can be performed.

<Embodiment 2>
The second embodiment shows an example in which a function depending on the distance from the center of the photographing optical axis is used as a scale function for correcting the scale of the camera coordinate system in the correction matrix calculation process (step S908 in FIG. 9) of the stereo camera.

  In the second embodiment, the hardware configuration and most of the control procedure are the same as those in the first embodiment. In this embodiment, only the correction matrix calculation process (step S908 in FIG. 9) of the stereo camera is different, and only the correction matrix calculation process will be described below.

  In general, the lens distortion amount changes according to the distance from the center of the photographing optical axis. There is almost no distortion near the center of the photographing optical axis, and the amount of distortion tends to increase as the distance increases. For example, the lens distortion correction parameter can be calculated in camera calibration (step S401 in FIG. 4), but it is extremely difficult to accurately correct the lens distortion as described above. As described above, the influence of lens distortion may remain uncorrected in image processing when calculating parallax. For example, it is considered that the parallax measurement error due to lens distortion tends to change according to the distance from the center of the optical axis of the lens. For example, parallax can be measured with high accuracy because it is hardly affected by lens distortion near the center of the optical axis, but it is affected by lens distortion correction errors in the peripheral field of view of the captured image, particularly near its edges. A larger error occurs during measurement.

As described above, the parallax measurement error Δd is considered to depend on the distances r ₁ and r ₂ from the center of the optical axis to the measurement point on the captured image in the monocular cameras 301 and 302. Therefore, the parallax measurement error Δd depending on the distances r ₁ and r ₂ can be modeled linearly or as a quadratic function as in the following equations (12) and (13). Here, α, β, γ, α ₁ , α ₂ , β ₁ , β ₂ , and γ ₁ are correction coefficients.

  Here, the position of the center of the optical axis of the lens may be an internal parameter value obtained by camera calibration, or a design value of the camera. In the present embodiment, modeling is performed linearly and in a quadratic function, but higher-order modeling may be performed. However, when modeling with a higher-order function, there is a possibility that the risk of time-consuming optimization calculations performed later or falling into a local solution may increase.

  Next, when the parallax measurement error Δd as described above exists, the Z coordinate z ″ of the measurement point P obtained by stereo measurement can be written as the following equation (14).

  When using the equations (4) and (14), the relationship between the physical true Z coordinate of the measurement point P and the Z coordinate z ″ of the measurement point P obtained by stereo measurement is as shown in the following equation (15). There is.

Thus, it can be seen that the scale of the physical true Z coordinate z of the measurement point P and the Z coordinate z ″ of the measurement point P obtained by stereo measurement change depending on the parallax measurement error Δd. Therefore, if the scale function m _z of the Z coordinate is modeled as shown in the following two expressions, the influence of the parallax measurement error Δd can be reduced.

Equation (16) is a right side denominator of a, _{a 1} is the correction factor (17). The above is the model setting of the scale function m _z of the correction matrix A. Using the model of the scale function m _z of the correction matrix A, the optimization calculation of Expression (11) shown in the first embodiment is performed, and each variable of the coordinate transformation matrix ^M T _C and the correction matrix A is calculated. Can do.

In the second embodiment, as an example, the scale function m _z is defined as in equations (16) and (17). However, as the scale function m _z , a polynomial such as a linear function or a quadratic function of the distances r ₁ and r ₂ from the center of each optical axis to the measurement point in the captured image of each monocular camera may be simply adopted. Good. As a result, the mathematical expression of the scale function m _z becomes simple, and there is a possibility that the time required for the optimization calculation of Expression (11) can be shortened and the risk of falling into a local solution can be reduced. As described above, in this embodiment, in order to correct a scale error due to lens distortion or the like, a model of the scale function m _z is used, and the function in which the scale function m _z depends on the distance from the optical axis center. This is a feature.

Further, in the above, it is possible but considering scale function m _z about the Z-axis direction is expressed by a polynomial, such as a linear function or a quadratic function similarly distance greater m _x and m _y r _1, r _2. Thus, not only the direction of the Z-axis scale function m _z only, by using the function model that depends on the distance r _1, r ₂ from also the optical axis center to m _x and m _y, is corrected with higher accuracy 3D measurement results can be obtained.

<Embodiment 3>
In the third embodiment, an example of using a function depending on a three-dimensional measurement coordinate value obtained by stereo measurement as a scale function for correcting the scale of the camera coordinate system in the correction matrix calculation process (step S908 in FIG. 9) of the stereo camera. Show.

  In the third embodiment, the hardware configuration and the control system configuration are the same as those in the first and second embodiments. In the third embodiment, only the correction matrix calculation process (step S908 in FIG. 9) of the stereo camera is different, and only the correction matrix calculation process will be described below.

  As described in the second embodiment, there is a possibility that the lens distortion cannot be corrected in the image processing when calculating the parallax, and the parallax measurement error tends to change according to the distance from the center of the optical axis of the lens. In addition, when the distance from the center of the optical axis of the lens is different on the captured image, it means that the three-dimensional coordinate value of the measured point has a measurement error different from the original value. This may be said that the parallax measurement error changes depending on the three-dimensional coordinate value of the measurement point.

As described above, when it is considered that the parallax measurement error Δd depends on the three-dimensional coordinate values (x, y, z) of the measurement point, the measurement error Δd is expressed by the following equations (18) and (19). Can be modeled linearly or as a quadratic function. Here, k, l, m, n, k ₁ , k ₂ , l ₁ , l ₂ , m ₁ , m ₂ , and n ₁ are correction coefficients.

  In the present embodiment, modeling is performed linearly and in a quadratic function, but higher-order modeling may be performed. However, when modeling with a higher-order function, there is a possibility that the risk of time-consuming optimization calculations performed later or falling into a local solution may increase.

Further, as shown in the equation (15) shown in the second embodiment, when a parallax measurement error Δd exists, the physical true Z coordinate of the measurement point P and the Z coordinate z of the measurement point P obtained by stereo measurement are obtained. The scale of '' changes. Therefore, if the scale function m _z of the Z coordinate is modeled as follows, the influence of the parallax measurement error Δd can be reduced.

Here, b and b ₁ are correction coefficients. The above is the model setting of the scale function mz of the correction matrix A. Using the model of the scale function m _z of the correction matrix A, the optimization calculation of the equation (11) shown in the first embodiment is performed to calculate the coordinate transformation matrix ^M T _C and the correction matrix A, respectively. it can.

In the present embodiment, m _z is defined as shown in equations (20) and (21) as an example. However, as the scale function m _z , a polynomial such as a linear function or a quadratic function of the three-dimensional coordinate values (x, y, z) of the measurement point may be simply adopted. As a result, the mathematical expression of the scale function m _z becomes simple, and there is a possibility that the time required for the optimization calculation of Expression (11) can be shortened and the risk of falling into a local solution can be reduced. As described above, in this embodiment, in order to correct a scale error due to lens distortion or the like, a model of the scale function m _z is used, and the scale function m _z depends on the three-dimensional coordinate value of the measurement point. It is characterized by being a function.

Further, in the above, considering scale function m _z about the Z-axis direction, m _x and m _y are similarly three-dimensional coordinate values of the measurement point (x, y, z) of the linear function and the like quadratic function It can be expressed by a polynomial. Thus, not only the direction of the Z-axis scale function m _z only, to m _x and m _y 3 dimensional coordinate values of the measurement point (x, y, z) by using a dependent function model, more accurate A three-dimensional measurement result corrected to can be obtained.

<Embodiment 4>
In the fourth embodiment, a process of generating a three-dimensional correction map in the stereo camera correction matrix calculation process (step S908 in FIG. 9) will be described.

  In the fourth embodiment, the hardware configuration and the control system configuration are the same as those in the first and second embodiments. In the fourth embodiment, only the correction matrix calculation process (step S908 in FIG. 9) of the stereo camera is different. Only the correction matrix calculation process will be described below.

Hereinafter, the calculation flow of the correction matrix of the stereo measurement result will be described with reference to FIG. The calibration process of the present embodiment includes a process (local correction matrix calculation step) of calculating a correction matrix B _i corresponding to a three-dimensional correction map after obtaining the homogeneous coordinate transformation matrix ^M T _C and the correction matrix A. Including.

  Hereinafter, i means the i-th local region when the measurement range is divided into N local regions, and 1 ≦ i ≦ N. First, the correction matrix A is modeled (step S1201). In modeling the correction matrix A, as shown in the equations (9) and (10) of the first embodiment, a model incorporating a scale function may be used, or a unit matrix that does not convert the scale may be used. In the present embodiment, description will be made assuming that the correction matrix A is a unit matrix.

Next, each variable of the homogeneous coordinate transformation matrix ^M T _C and the correction matrix A is calculated (step S1202). As for the calculation method of each variable of the homogeneous coordinate transformation matrix ^M T _C and the correction matrix A, the optimization calculation may be solved for the equation (11) as in the first embodiment.

Subsequently, in step S1203, an error of the stereo measurement result at each measurement position is calculated. Here, the homogeneous coordinate transformation matrix ^M T _C , the center position of the tip sphere 202 in the camera coordinate system C calculated in step S906 (FIG. 9), and the tip sphere 202 in the positioning coordinate system M calculated in step S904 (same). The center position of is used. This correction matrix B _i can be written as in the following equation (22).

This correction matrix B _i is an error of the stereo measurement result. This correction matrix B _i is calculated for each local region where i is 1 to N. The calculated correction matrix B _i is stored in the nonvolatile memory 116 in the vision controller 113 (S1204).

  The correction matrix for the stereo measurement result is calculated as follows. When the measurement position of the stereo camera 104 after calibration is determined, the center position of the tip sphere 202 is measured in the vicinity of the measurement position, and the error of the stereo measurement result is calculated. Then, this error may be used as it is as a correction matrix for the stereo measurement result. On the other hand, if the measurement position is not fixed or it is difficult to calculate an error in the entire range because the measurement range is wide, the following processing is performed.

Here, the center position of the tip sphere 202 when the error of the stereo measurement result is calculated is referred to as an error calculation point. First, N error calculation points q _i in the vicinity of the measurement point p are extracted. The number of error calculation points q _i may be two or more. Then, as the distance between the measuring points p and the error calculation point is short, to weight the weights w _i so the weight becomes heavy. Then, as shown in the following equation (23), a weighted average is calculated for the error calculation point correction matrix B (q _i ) to obtain the correction matrix B (p).

  As described above, the correction matrix B (p) of the stereo measurement result is calculated. According to such an interpolation method, correction with higher accuracy is possible if the error calculation point covers a wide measurement range. In the present embodiment, the method of performing interpolation using a weighted average has been described. However, other methods such as a simple average or a spline interpolation may be used.

<Embodiment 5>
For example, in the first embodiment, it can be considered that the housing of the stereo camera 104 constitutes a functional component including a three-dimensional visual measurement device (unit). However, when the stereo camera 104 as a three-dimensional visual measurement device is combined with the robot apparatus (R) or the like, for example, the stereo camera 104 is incorporated in an end effector or the like of the robot apparatus (R), and these are integrated into an assembly of functional parts. It may be configured. In this case, it can be considered that the end effector portion of the robot apparatus (R) to which the stereo camera 104 has already been assembled constitutes a functional component including the three-dimensional visual measurement apparatus (stereo camera 104).

  In the fifth embodiment, the end effector portion to which the stereo camera 104 has already been assembled constitutes a functional component including the three-dimensional visual measurement device (stereo camera 104), and the second coordinate system (F in this embodiment) is provided. The example arrange | positioned in the specific reference | standard site | part of an end effector is shown.

  In the fifth embodiment, as shown in FIG. 13A, it is assumed that the stereo camera 104 is already attached to the camera mount 1104 arranged on the side surface of the hand 1101a constituting the end effector of the robot apparatus. . The positioning and fixing method between the camera mount 1104 and the stereo camera 104 is arbitrary, and both may be easily detachable, or may be fixed by screwing or bonding.

  In short, in the fifth embodiment, it is assumed that the assembly of the hand 1101a as the end effector constitutes a functional component including the three-dimensional visual measurement device (stereo camera 104) as a unit.

  The hand 1101a includes, for example, two (or more) fingers 1102, 1102 (...) on the palm 1103 of the hand 1101a. The fingers 1102, 1102 (...) Are driven to open and close by a driving source (not shown) (motor, solenoid, other actuator) disposed inside the hand 1101 a, thereby allowing the workpiece to be gripped and operated.

  In the fifth embodiment, a finger coordinate system F (second coordinate system of the present embodiment) determined from (one of) fingers 1102 of the hand 1101a and a camera coordinate system C (first coordinate system) of the stereo camera 104 are described. Calibration data for coordinate conversion between and is acquired. FIG. 13B shows the hand 1101a from above, and the origin of the finger coordinate system F (second coordinate system of the present embodiment) is, for example, one corner of the finger 1102 (left side in the figure in this example). It is arranged in the part.

  In the fifth embodiment, an assembly of a hand 1101a as an end effector constitutes a functional component that integrally includes a three-dimensional visual measurement device (stereo camera 104). In this case, when performing an operation such as gripping the workpiece with the fingers 1102 and 1102 (...) Of the hand 1101a, the relative position and orientation of the component with respect to the finger 1102 is obtained by, for example, three-dimensional visual measurement of the stereo camera 104.

  In the fifth embodiment, calibration data that can perform coordinate conversion between the finger coordinate system F (second coordinate system) and the camera coordinate system C (first coordinate system) is acquired. According to the fifth embodiment, the calibration data is obtained by contact measurement of the hand 1101a or the finger portion corresponding to the functional component including the three-dimensional visual measurement device (stereo camera 104) with the three-dimensional measuring machine 101 (FIG. 1). Can be obtained with high accuracy.

  A camera coordinate system C (first coordinate system) measured three-dimensionally by the stereo camera 104 by using calibration data for coordinate conversion between the finger coordinate system F (second coordinate system) and the camera coordinate system C. Can be easily converted into the finger coordinate system F reference. Accordingly, it is possible to realize a highly accurate operation such as gripping small parts with the robot arm using the three-dimensional visual measurement result of the stereo camera 104.

  Also in the fifth embodiment, the hardware and control system of the calibration system are the same as those in the first embodiment, and only the parts different from the first embodiment will be described. FIG. 14 shows the entire calibration procedure of the three-dimensional visual sensor of the fifth embodiment. FIG. 14 is a diagram corresponding to FIG. 4 of the first embodiment. The difference from FIG. 4 is a finger position teaching (S1203) and a calibration process (S1205) between the camera and the finger. The other processes in steps S401, S402, and S404 are the same as those described in FIG. 4, and thus detailed description thereof is omitted here.

  FIGS. 15A to 15C show how the finger coordinate system F is defined in step S1203 in FIG. 14 (or contact measurement of the positioning reference portion in step S901 in FIG. 9). In the fifth embodiment, the XY plane of the finger coordinate system F is the palm 1103 (FIGS. 13A and 13B) of the hand 1101a. 13A and 13B, the origin of the finger coordinate system F is the root of the left finger 1102, and the X axis is the direction from the left finger 1102 to the right finger 1102.

  First, as shown in FIG. 15A, a hand 1101 a having a stereo camera 104 is placed on the surface plate 103 of the three-dimensional measuring machine 101. In this example, the stereo camera 104 itself has the same structure as that described in FIG. 3 of the first embodiment, for example, and is attached to a side portion of the hand 1101a via the camera mount 1104. Note that the monocular cameras (301 and 302 in the first embodiment) of the stereo camera 104 are not illustrated in detail in the present embodiment for the sake of simplification.

  Next, as shown in FIG. 15A, in order to determine the XY plane of the finger coordinate system F, three or more measurement probes (tip sphere 202) are brought into contact with the palm 1103 of the hand 1101a (201g to 201i). ). The probe control unit 108 in the coordinate measuring machine controller 107 receives the probe position at the moment of contact from the probe position measurement unit 111 and records it in the nonvolatile memory 109, for example. Next, as shown in FIG. 15B, at least three points of the measurement probe (the tip sphere 202) are brought into contact with the inner side surfaces of the left and right fingers 1102 (201j, 201k). At this time, the probe control unit 108 in the coordinate measuring machine controller 107 records the probe position at the moment of contact in the nonvolatile memory 109. Subsequently, as shown in FIG. 15C, three or more measurement probes (tip sphere 202) are brought into contact with the front side surfaces of the left and right fingers in the drawing (201l, 201m). At this time, the probe control unit 108 in the coordinate measuring machine controller 107 records the probe position at the moment of contact in the nonvolatile memory 109.

  From the above measurement results, the probe control unit 108 in the coordinate measuring machine controller 107 acquires coordinate values that define the plane of each surface of the finger 1102. Next, the probe control unit 108 in the coordinate measuring machine controller 107 obtains the inner side surfaces of the left and right fingers 1102 and 1102, and the intersections of the front side surface and the XY plane of the palm portion 1103. Then, as shown in FIG. 13B, the intersection of the left fingers is defined as the origin of the finger coordinate system F, and the direction from the left to the intersection of the right fingers is defined as the X direction. As described above, the definition of the finger coordinate system F is completed (step S1203). In the above description, the modifications such as right and left indicate a relationship with the display of the drawing, and are merely an example.

  The calculation process corresponding to the relative position and orientation of the finger coordinate system F (second coordinate system) and the camera coordinate system C (first coordinate system) in step S1205 in FIG. To (4)).

  As described above, in the present embodiment, (hand 1101a) is considered as an end effector of a robot as a functional component including the stereo camera 104 (three-dimensional visual measurement device). However, it goes without saying that the same processing can be applied to other robot end effectors such as welding torches and paint guns. In that case, the origin position of the first coordinate system defined by the measurement of the contact-type measurement probe 102 may be appropriately arranged at a specific reference portion of each end effector.

According to this embodiment, the finger coordinate system F (second coordinate system) determined from the reference part of the functional component including the stereo camera 104 (three-dimensional visual measurement device) and the camera coordinate system C (first Calibration data for performing coordinate conversion between coordinate systems) can be acquired. The storage destination of calibration data (coordinate transformation matrix ^M T _C and the correction matrix A), as in the above embodiment, and a non-volatile memory 116 in the vision controller 113, such as a housing of the non-volatile memory of the stereo camera 104 Conceivable. Alternatively, in the case of the present embodiment, a configuration in which a nonvolatile memory storing calibration data (coordinate transformation matrix ^M T _C and correction matrix A) is arranged inside the housing of the hand 1101a (end effector) is also conceivable.

The method of using calibration data when a functional component including a stereo camera 104 (three-dimensional visual measurement device) is attached to the arm of the robot apparatus is the same as that described in the first embodiment. When the control unit (or vision controller 113) of the robot apparatus obtains the coordinate value of the specific part of the object being photographed by using the image obtained by stereo imaging of the object such as the workpiece by the stereo camera 104, The coordinate value of the first coordinate system (C) is acquired. The control unit (or vision controller 113) of the robot apparatus uses the calibration data (coordinate transformation matrix ^M T _C and correction matrix A) to immediately identify the specific part of the object in the second coordinate system (M). Can be converted to the coordinate value.

The relative position and orientation of the hand 1101a (end effector) attached to the robot arm is known to the control unit (not shown) of the robot apparatus R. For this reason, the position / posture of the specific part of the target object measured three-dimensionally by the stereo camera 104 can be easily converted into, for example, a relative position / posture with respect to the robot arm. Moreover, according to this embodiment, the calibration data (coordinate transformation matrix ^M T _C and correction matrix A) can be acquired (calibrated) with extremely high accuracy using the three-dimensional measuring machine 101. Therefore, based on the three-dimensional visual measurement of the stereo camera 104, the robot apparatus (R) can accurately operate the object (work).

<Embodiment 6>
In the calibration method of the present invention, the first coordinate system (C) in which the three-dimensional measurement value of the three-dimensional visual measurement device (stereo camera 104) is expressed, and the reference of the three-dimensional visual measurement device (functional component provided). Calibration data for coordinate transformation between the second coordinate system (M) determined from the part is acquired.

  In each of the above embodiments, the configuration using the three-dimensional measuring machine 101 provided with the contact-type measurement probe 102 has been exemplified. Here, considering the configuration of each of the above embodiments, the measurement probe 102 photographed with the stereo camera 104 (three-dimensional visual measurement device) can be considered as a “mark” for calibration.

  Considering the measurement probe 102 of each of the above embodiments as a “mark” for calibration, it can be considered that the calibration control in each of the above embodiments includes the following steps.

  A mark in which a coordinate value can be measured by the three-dimensional visual measurement device at a measurement position whose positional relationship with the three-dimensional visual measurement device is known in advance within a range that can be photographed by the three-dimensional visual measurement device (stereo camera 104). Mark setting process to position.

  A three-dimensional visual measurement process in which the coordinate value of the mark in the first coordinate system (camera coordinate system C) is measured by the three-dimensional visual measurement device (stereo camera 104).

  A mark coordinate value calculation step of calculating the coordinate value of the mark in the second coordinate system (positioning coordinate system M) determined from the reference portion of the three-dimensional visual measurement device (stereo camera 104).

  A calibration data calculation step of calculating calibration data using the coordinate value of the mark in the first coordinate system visually measured and the coordinate value of the mark in the second coordinate system measured above; The calibration data calculated in this calibration data calculation step includes calibration data for coordinate conversion between the first coordinate system and the second coordinate system, and three-dimensional measurement coordinates measured by the three-dimensional visual measurement device. And a correction matrix including elements for correcting the scale of values.

  In that case, the three-dimensional measuring machine 101 of each of the above embodiments positions a mark whose coordinate value can be measured by the stereo camera 104 within a range that can be photographed by the stereo camera 104 (three-dimensional visual measurement device). Can be considered.

  In each of the embodiments described above, the positioning holes 304 and 304 of the stereo camera 104 (three-dimensional visual measurement device) are actually measured by the measurement probe 102 of the three-dimensional measuring machine 101. However, if there is a mark setting device that can position the same “mark” for calibration as the measurement probe 102 at a measurement position where the positional relationship with the stereo camera 104 (three-dimensional visual measurement device) is known in advance, this device can be used. It can be used in place of the three-dimensional measuring machine 101.

  Such a mark setting device 7000 can be configured as shown in FIG. 17, for example. In FIG. 17, the stereo camera 104 (three-dimensional visual measurement device) is attached to the positioning jig 105. The mounting to the positioning jig 105 uses the same engaging structure of the positioning pins 303 and 303 and the positioning holes 304 and 304 as described above. The stereo camera 104 is fixed on the base 705 of the mark setting device 7000 via the positioning jig 105.

  On the base 705 of the mark setting device 7000, a marker plate 702 can be arranged at an imaging distance (L) within a range that can be taken by the stereo camera 104. In addition, the mark setting device 7000 includes a drive unit 1072 that moves the marker plate 702 in the left-right direction in the figure while maintaining a posture in which the center of the marker plate 702 substantially coincides with the center of the two optical axes of the stereo camera 104, for example. . The driving unit 1072 is controlled by a mark placement control device 1071 corresponding to the coordinate measuring machine controller 107 (FIG. 1) of each of the above embodiments. The CPU level configuration of the mark placement control device 1071 may be the same as that of FIG.

  In each of the above embodiments, the three-dimensional measuring machine controller 107 (FIG. 1) positions the measurement probe 102 when the stereo probe 104 images the measurement probe 102. On the other hand, in this embodiment, the mark placement control device 1071 may perform positioning so as to determine the shooting distance of the marker plate 702 to the mark 7021 when shooting the mark 7021 of the marker plate 702 with the stereo camera 104. .

  The mark installation control device 1071 does not have a function of actually measuring the positioning holes 304 and 304 as in the above embodiments. However, it is assumed that the dimensional accuracy of the positioning jig 105 to the base 705 and the accuracy of controlling the shooting distance of the marker plate 702 to the mark 7021 by the driving unit 1072 are sufficiently maintained in advance. That is, the positions of the positioning holes 304 and 304 are known with a predetermined accuracy in advance with respect to the mark installation control device 1071. Accordingly, the positioning coordinate system M (second coordinate system) of the stereo camera 104 can be defined by the mark placement control device 1071 without performing probe actual measurement as in the above embodiment. Then, the coordinate value of the mark in the positioning coordinate system M (second coordinate system) determined from the reference portion (303) of the stereo camera 104 (the functional component including the stereo camera 104) can be calculated (mark coordinate value calculation step).

  On the marker plate 702, as shown on the right side of FIG. 17, for example, a large number of circular marks 7021, 7021,. The mark 7021 and the background have a color scheme that makes it easy to measure the position of the mark 7021 by image processing based on the shooting result of the stereo camera 104, such as white on a black background or black on a white background.

  The configuration of the present embodiment includes a first coordinate system (coordinate system C) in which the three-dimensional measurement values of the stereo camera 104 (three-dimensional visual measurement device) are represented, and a three-dimensional visual measurement device (functional component including the same). This corresponds to a calibration apparatus having a second coordinate system (coordinate system M) having an origin at the reference part. The calibration apparatus acquires calibration data for performing coordinate conversion between the first coordinate system (coordinate system C) and the second coordinate system (coordinate system M).

  As described above, the configuration of the present embodiment includes the mark setting device 7000 that positions the mark 7021 whose coordinate value can be measured by the stereo camera 104 within the range that can be captured by the stereo camera 104 (three-dimensional visual measurement device). .

  The mark placement control device 1071 can place the mark 7021 at a known measurement position with a predetermined accuracy, for example, a predetermined shooting distance L via the drive unit 1072 of the mark placement device 7000 (mark installation step). ).

  The mark installation control device 1071 causes the stereo camera 104 (three-dimensional visual measurement device) to measure the coordinate value of the mark in the camera coordinate system C (first coordinate system) (three-dimensional visual measurement step).

  Then, the mark installation control device 1071 calculates the coordinate value of the mark 7021 in the camera coordinate system M (second coordinate system) determined from the reference part (303 to 304) of the stereo camera 104 (the functional component including the stereo camera 104). (Mark coordinate value calculation step). Here, the dimensional accuracy of the positioning jig 105 to the base 705 and the control accuracy of the shooting distance of the marker plate 702 (mark 7021) by the drive unit 1072 are sufficiently maintained.

  For this reason, if the mark installation control device 1071 determines the shooting distance L of the marker plate 702, the mark 7021 in the camera coordinate system M (second coordinate system) is determined based on, for example, the control amount of the drive unit 1072 at that time. Coordinate values can be calculated. This camera coordinate system M (second coordinate system) may be considered to be predefined with a predetermined accuracy based on the mechanical accuracy specification of the mark setting device 7000 as described above. Therefore, according to the configuration of FIG. 17, for example, the process of actually measuring the positioning holes 304, 304 with the measurement probe 102 is not required, and the mark placement control device 1071 is in the second coordinate system (positioning coordinate system M). The coordinate value of the mark can be calculated (mark coordinate value calculation step).

  In the calibration processing of the present embodiment, the mark 7021 is set at a specific measurement position (shooting distance L) by the mark setting device 7000 (mark setting step). Then, the coordinate value of the mark 7021 in the first coordinate system (camera coordinate system C) is measured by the three-dimensional visual measurement device (stereo camera 104) (three-dimensional visual measurement step). On the other hand, the mark installation control device 1071 can calculate the coordinate value of the mark 7021 in the camera coordinate system M (second coordinate system) based on the control amount of the drive unit 1072 at the measurement position (shooting distance L).

  Then, the calibration data can be calculated using the visually measured coordinate value of the mark in the first coordinate system and the measured coordinate value of the mark in the second coordinate system (calibration data calculating step). . This calibration data includes calibration data for performing coordinate conversion between the first coordinate system and the second coordinate system, and elements for correcting the scale of the three-dimensional measurement coordinate value measured by the three-dimensional visual measurement device. Including a correction matrix.

  Calibration data for performing coordinate conversion between the first and second coordinate systems using the first coordinate system and the coordinate value of the mark in the second coordinate system, and the scale of the three-dimensional measurement coordinate value The calculation process for calculating the correction matrix including the elements for correcting is the same as in the first embodiment. Further, for example, the detailed configuration of the calibration control shown in Embodiments 2 to 5 with respect to the calibration data acquisition step can be similarly performed in this embodiment.

  In FIG. 8 (Embodiment 1), the measurement probe 102 (the mark 7021 of this embodiment) is positioned in the common visual field region OV of the monocular cameras 301 and 302 constituting the stereo camera 104 in the three-dimensional visual measurement process. Control for photographing with the stereo camera 104 is shown. Further, FIG. 11 shows control in which the common visual field area OV is divided into local areas 1101 and 1102, and a correction matrix (A) is calculated separately for each. Such calibration control is also possible in the configuration of FIG.

  For example, the portion on the right side of the shooting distance L in FIG. 17 shows a shooting distance range corresponding to the common visual field region OV in FIGS. Further, the common visual field region OV can be considered by being divided into a photographing distance range corresponding to the local regions 1101 and 1102 in FIG. Therefore, for example, by moving the marker plate 702 to the position of the vertical line of each boundary of the local areas 1101 and 1102 of the common visual field area OV and photographing the mark 7021 with the stereo camera 104, it will be described with reference to FIGS. The same three-dimensional visual measurement mode as that described above can be realized.

  In that case, the mark 7021 in the spatial range of the pyramid-shaped common visual field OV where the depths of field of the monocular cameras 301 and 302 overlap may be measured three-dimensionally. For this purpose, for example, according to the shooting distance of the marker plate 702, the mark 7021 used for the three-dimensional visual measurement is arranged in each of the regions 7021a, 7021b, and 7021c such as the center, the outside, and the outside of the marker plate 702. Just choose one. In this way, the same three-dimensional visual measurement mode as described in FIGS. 8 and 11 can be realized. For example, in the configuration in which the correction matrix (A) is calculated separately for each of the local regions 1101 and 1102, the scale correction suitable for the photographing distance can be performed by the correction matrix (A).

  DESCRIPTION OF SYMBOLS 101 ... Three-dimensional measuring machine, 102 ... Measuring probe, 104 ... Stereo camera, 105 ... Positioning jig, 107 ... Three-dimensional measuring machine controller, 113 ... Vision controller, 201 ... Stylus, 202 ... Tip ball, OV ... Common visual field area 7000 ... Mark installation device, 7021 ... Mark.

Claims (22)

Coordinate conversion is performed between the first coordinate system in which the three-dimensional measurement value of the three-dimensional visual measurement device is expressed and the second coordinate system determined from the reference part of the functional component including the three-dimensional visual measurement device. In a calibration method for obtaining calibration data for performing,
A mark in which a coordinate value can be measured by the three-dimensional visual measurement device at a measurement position where a positional relationship with the three-dimensional visual measurement device is known in advance within a range that can be photographed by the three-dimensional visual measurement device. Mark positioning process for positioning,
A three-dimensional visual measurement step in which the control device measures the coordinate value of the mark in the first coordinate system by the three-dimensional visual measurement device;
A mark coordinate value calculating step in which a control device calculates a coordinate value of the mark in a second coordinate system determined from a reference part of a functional component including the three-dimensional visual measurement device;
And a coordinate value of the mark in the first coordinate system visually measured in the three-dimensional visual measurement step, and a coordinate of the mark in the second coordinate system measured in the mark coordinate value measurement step. Using the value, the calibration data for coordinate conversion between the first coordinate system and the second coordinate system and the scale of the three-dimensional measurement coordinate value measured by the three-dimensional visual measurement device are corrected. A calibration method including a correction matrix including elements to be calculated, and a calibration data calculation step of calculating. 2. The calibration method according to claim 1, wherein in the calibration data acquisition step, the coordinate value ( ^C P _i ) of the mark in the first coordinate system (C) and the coordinate value in the second coordinate system (M). A calibration method for calculating ( ^M T _C ) as the calibration data and (A) as the correction matrix based on the coordinate value ( ^M P _i ) of the mark by the following equation.

3. The calibration method according to claim 1, wherein in the calibration data acquisition step, the correction matrix (A) is a scale for correcting the scale of the first coordinate system determined by the three-dimensional visual measurement device as shown in the following equation. functions _{m x, m y,} calibration method is a matrix containing the _{m z.}

4. The calibration method according to claim 1, wherein, in the calibration data acquisition step, (A) as the correction matrix is a first coordinate determined by the three-dimensional visual measurement device as follows: A calibration method which is a matrix containing a scale function m _z for correcting only the scale in the Z direction of the system.

  5. The calibration method according to claim 3, wherein any one of the scale functions is a constant. 5. The calibration method according to claim 3, wherein the three-dimensional visual measurement device is a stereo camera, and the scale function includes a distance r ₁ from a center of a photographing optical axis on a photographed image of the stereo camera to a measurement point, functions _m x which depends on _{r 2 (r 1, r 2} ), the calibration method is represented by _{m y (r 1, r 2} ), m z (r 1, r 2). 5. The calibration method according to claim 3, wherein the scale function is a function m _x (x, y, z) depending on a three-dimensional measurement coordinate value x, y, z measured by the three-dimensional visual measurement device. _{, m y (x, y,} z), m z (x, y, z) is a calibration method.   8. The calibration method according to claim 1, wherein in the calibration data acquisition step, the measurement range of the three-dimensional visual measurement device is divided into a plurality of local regions, and the correction matrix is divided for each of the local regions. A calibration method including a local correction data calculation step for calculating. 9. The calibration method according to claim 8, wherein, in the local correction data calculation step, after calculating the correction matrix (A), the measurement range is divided into N local regions, and the i th (1 ≦ i ≦ N) is obtained. A calibration method for calculating a correction matrix (B _i ) corresponding to a three-dimensional measurement coordinate value measured by the three-dimensional visual measurement device by the following equation for each local region.

The calibration method according to any one of claims 1 to 9,
Using a measuring probe of a CMM as the mark,
The control device uses the measurement probe to three-dimensionally measure the shape of the functional component or the shape of a jig that can be mechanically coupled to the functional component, and based on the three-dimensional measurement result, the second coordinates A probe measurement process that defines the system,
In the mark setting step, the control device is located at a measurement position where the positional relationship with the three-dimensional visual measurement device is known in advance within a range that can be photographed by the three-dimensional visual measurement device. Position the measuring probe as the mark that can measure the coordinate value by,
In the three-dimensional visual measurement step, the control device measures the coordinate value of the mark in the first coordinate system by the three-dimensional visual measurement device,
In the mark coordinate value calculation step, the control device determines the coordinate value of the mark in the second coordinate system defined in the probe measurement step, which is determined from a reference part of a functional component including the three-dimensional visual measurement device. Calibration method to calculate   11. The calibration method according to claim 10, wherein in the probe measurement step, the functional component is positioned as the shape of the functional component or the shape of a jig that can be mechanically coupled to the functional component by the measurement probe. A calibration method for three-dimensionally measuring a reference portion or a positioning reference portion of the jig.   12. The calibration method according to claim 11, wherein the functional component is an end effector of a robot equipped with the three-dimensional visual measurement device, and the shape of the functional component is determined by the measurement probe in the probe measurement step. A calibration method for three-dimensionally measuring a specific part of the end effector. Coordinates between a first coordinate system in which a three-dimensional measurement value of the three-dimensional visual measurement device is expressed and a second coordinate system having an origin at a reference part of a functional component including the three-dimensional visual measurement device In a calibration device that acquires calibration data for conversion,
A mark whose coordinate value can be measured by the three-dimensional visual measurement device is positioned at a measurement position whose positional relationship with the three-dimensional visual measurement device is known in advance within a range that can be photographed by the three-dimensional visual measurement device. A mark installation device;
A control device,
A mark installation step in which the control device installs the mark at the measurement position by the mark installation device;
A three-dimensional visual measurement step in which the control device measures the coordinate value of the mark in the first coordinate system by the three-dimensional visual measurement device;
A mark coordinate value calculating step in which a control device calculates a coordinate value of the mark in a second coordinate system determined from a reference part of a functional component including the three-dimensional visual measurement device;
And a coordinate value of the mark in the first coordinate system visually measured in the three-dimensional visual measurement step, and a coordinate of the mark in the second coordinate system measured in the mark coordinate value measurement step. Using the value, the calibration data for coordinate conversion between the first coordinate system and the second coordinate system and the scale of the three-dimensional measurement coordinate value measured by the three-dimensional visual measurement device are corrected. A correction matrix including elements to be corrected, a calibration data calculation step for calculating,
Perform calibration device.   A three-dimensional measuring machine used in the calibration method according to any one of claims 1 to 12, which transmits a measurement result of the probe measurement step to the control device.   The three-dimensional visual measurement apparatus which transmits the measurement result of the said three-dimensional visual measurement process used for the calibration method of any one of Claim 1 to 12 to the said control apparatus.   The calibration method according to any one of claims 1 to 12, wherein the calibration data and the correction matrix are stored in a storage device as calibration data of the functional component.   17. The calibration method according to claim 16, wherein the storage device includes a control unit that controls the three-dimensional visual measurement device, the three-dimensional visual measurement device, or the functional component including the three-dimensional visual measurement device. Calibration method using a storage device arranged in the housing of the computer.   A storage device storing calibration data acquired by the calibration method according to any one of claims 1 to 12 and the correction matrix is provided in a housing, and the functional component to which the three-dimensional visual measurement device is mounted is provided. Robot end effector. Coordinate conversion is performed between the first coordinate system in which the three-dimensional measurement value of the three-dimensional visual measurement device is expressed and the second coordinate system determined from the reference part of the functional component including the three-dimensional visual measurement device. In a calibration method for obtaining calibration data for performing,
The control device performs three-dimensional measurement of the shape of the functional component or the shape of a jig that can be mechanically coupled to the functional component using a measurement probe of a three-dimensional measuring machine, and based on the three-dimensional measurement result, A probe measurement step defining a second coordinate system;
At the three or more measurement points within the range that can be photographed by the three-dimensional visual measurement device, the control device uses the three-dimensional visual measurement device to determine the coordinate value of a specific part of the measurement probe in the first coordinate system. 3D visual measurement process for measuring 3D visual,
The control device includes: the second coordinate system defined in the probe measurement step; coordinate values in the second coordinate system of the three or more measurement points of the measurement probe in the three-dimensional visual measurement step; and The three-dimensional visual measurement device is provided on the basis of the coordinate value in the first coordinate system of the specific part of the measurement probe that is three-dimensionally measured at the three or more measurement points in the three-dimensional visual measurement step. The calibration data for performing coordinate conversion between the first coordinate system having the origin at the reference site of the functional component and the second coordinate system in which the three-dimensional measurement value of the three-dimensional visual measurement device is expressed. And a correction data acquisition step for acquiring a correction matrix including an element for correcting the scale of the three-dimensional measurement coordinate value measured by the three-dimensional visual measurement device,
Calibration method including: Coordinates between a first coordinate system in which a three-dimensional measurement value of the three-dimensional visual measurement device is expressed and a second coordinate system having an origin at a reference part of a functional component including the three-dimensional visual measurement device In a calibration device that acquires calibration data for conversion,
A three-dimensional measuring machine equipped with a measurement probe;
A control device, the control device comprising:
The shape of the functional component or the shape of a jig that can be mechanically coupled to the functional component is three-dimensionally measured using a measurement probe of a three-dimensional measuring machine, and the second coordinates are based on the three-dimensional measurement result. A probe measurement process that defines the system;
The coordinate value of the specific part of the measurement probe in the first coordinate system is three-dimensionally measured by the three-dimensional visual measurement device at three or more measurement points within the range that can be photographed by the three-dimensional visual measurement device. A three-dimensional visual measurement process,
The second coordinate system defined in the probe measurement step, the coordinate values in the second coordinate system of the three or more measurement points of the measurement probe in the three-dimensional visual measurement step, and the three-dimensional vision Based on the coordinate value in the first coordinate system of the specific part of the measurement probe measured three-dimensionally at the three or more measurement points in the measurement step, the reference part of the functional component including the three-dimensional visual measurement device The calibration data for coordinate conversion between the first coordinate system having an origin at the second coordinate system expressing the three-dimensional measurement value of the three-dimensional visual measurement device, and the three-dimensional A calibration data acquisition step for acquiring a correction matrix including an element for correcting the scale of the three-dimensional measurement coordinate value measured by the visual measurement device;
Perform calibration device.   The control program for making the said control apparatus perform each process of any one of Claim 1 to 12 or Claim 19.   A computer-readable recording medium storing the control program according to claim 21.

Images