JP2008014691A - Stereo image measuring method and instrument for executing the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that there is a limit to accuracy and reliability in cases where a model is used while taking account of a point group of the entire shape of a measuring object satellite as a point group model used for an ICP, by a method based on a combination of stereo processing and an ICP algorithm which is one method in 3D model matching among point groups. <P>SOLUTION: The stereo image measuring method includes: a step of finding the three-dimensional shape of a measuring object in the form of measured point groups by stereo-processing images in a plurality of cameras; a step of presupposing a visible part from known shape information on the measuring object, the relative position of a stereo camera, and its posture; a step of determining a model point group conformed to the spatial density of the point groups by using only shape information on the presupposed visible part; and a step of matching the measured point groups to the model point group by using the ICP algorithm to employ a position and posture corresponding to a model group whose evaluation function is the smallest as the result of measurement. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image measurement method for accurately measuring the position / orientation of an object to be measured, and an apparatus for implementing the same, by combining stereo vision by a plurality of cameras and an ICP (Iterative Closest Point) algorithm.

  As one of the elemental technologies necessary for the development of a spacecraft called `` Debris Remover '' that captures, collects, and dumps fault satellites floating in outer space, the movement of the capture target (relative position and attitude relative to the spacecraft) ) Is required to measure and estimate by processing camera images acquired by spacecraft. As a method for this, we have developed an image measurement device composed of a plurality of cameras, arithmetic processing devices, and storage devices that are known to have a positional orientation relationship by combining stereo vision and an ICP (Iterative Closest Point) algorithm. The ICP algorithm is a condition where the complete shape (model point cloud) of an object is known, and when the partial shape information of the object is obtained separately, the partial shape (measurement point cloud) and a known model point It was developed as a method for estimating the positional relationship (coordinate transformation) with the group. In that algorithm, the relative position relationship between the two point groups (measurement point group and model point group) is initially set using a coordinate transformation that is once estimated, and the correspondence between the points of both point groups is set to “the closest distance”. "Is constructed in accordance with the norm", and the difference between each corresponding point is aggregated to obtain the minimum appropriate coordinate transformation by repeated calculation. The coordinate conversion of the sample point group is performed based on a 3 × 3 rotation matrix and a 3 × 1 translation vector.

As an example to which this ICP algorithm is applied, Patent Document 1 proposes a robot image processing method for bin-picking objects randomly stacked. The present invention avoids the minimum value of the evaluation function that exists in the process of estimating the position and orientation of the target object, accurately estimates the three-dimensional position and orientation of the target object, and stably grasps objects that are randomly stacked. For this purpose, in particular, complicated pre-processing such as area division is omitted, and the processing speed and efficiency are improved. For this purpose, image denoising is performed as preprocessing, and a grippable area is searched. Based on this information, the initial position and orientation value of the model to which the ICP algorithm is applied is determined, and the position and orientation of the entire model is estimated. If the converged evaluation function is smaller than a certain threshold, the target object is checked for overlap with other objects, and the robot is moved to the estimated position and orientation for the object that is determined to be gripped and handled. Is.
In addition, the “information processing method and information processing apparatus” disclosed in Patent Document 2 uses a virtual model so that the position and orientation of a real model and a virtual model having the same shape and size can be accurately matched. The present invention is an information processing method for adjusting the position and orientation of a virtual object synthesized with a real object, and includes a virtual index set for the virtual object. Acquiring position information, informing the user of the position of the virtual index, acquiring position information of a real index on a real object corresponding to the virtual index based on position information of an instruction unit operated by the user, and The position / orientation information of the virtual object is set based on the position information of the index and the position information of the real index.
However, these inventions use the ICP algorithm, but do not use stereo image measurement points.
Japanese Patent Laid-Open No. 9-277184 “Image Processing Method” Published on October 28, 1997 JP, 2006-48639, A "Information processing method and information processing apparatus" Published on February 16, 2006 Futeru Terui, Heihachiro Uemura, Shinichiro Nishida “Terrstrial experiments for the motion estimation of a large space debris object using image data” Conference of Optics East 2005; 23-26 October 2005 Boston Marriott Copley Place

  The image measurement technique we have developed for grasping the position and orientation of the measurement object is basically a three-dimensional shape of the measurement object obtained by stereo processing using multiple camera images, in the form of a point cloud. A model point cloud determined from a measurement point cloud and a known shape information of a measurement target is matched using an ICP algorithm, that is, a stereo image measurement device according to the present invention includes a conventional stereo image measurement technique, a partial shape, A measurement method based on a combination with an ICP algorithm, which is a method for estimating a positional relationship with a known model point group, is presented. The present inventors disclosed Non-Patent Document 1 “motion estimation based on image information for large space debris”. This is a technique to measure the motion of non-cooperating objects that perform posture motion from images as one of the necessary techniques for missions where space robots capture large space debris such as broken satellites. Therefore, we proposed a method to estimate the posture motion of a target from a time-series image using a combination of area / best / stereo / vision and ICP algorithm, which is one of 3D model matching methods between point clouds. The result of having performed the analysis using the image obtained from the ground device is shown. However, when a model that takes into account the entire shape of the measurement target satellite is used as a model point group used for ICP in the analysis process, matching errors may increase, and sometimes the results are completely different from true values. There was a problem that reliability was limited. In addition, when the spatial density of the model point group is set to a simple equidistant interval, the accuracy in the three-dimensional model matching is low because the measurement point group obtained from stereo vision is not necessarily spatially equidistant. The problem of becoming.

An object of the present invention is to solve the above problems, that is, a method based on a combination of stereo processing and 3D model matching between point groups, which is an ICP algorithm. The object is to solve the problem that there is a limit to the accuracy and reliability when using a model that takes into consideration point clouds of all shapes.
In addition, when the density of the model point cloud is simply spatially spaced, the measurement point cloud obtained from stereo vision is not necessarily evenly spaced, so the limit in accuracy in 3D model matching is low. Is to solve the problem of becoming.

The stereo image measurement method of the present invention includes a step of obtaining a three-dimensional shape of a measurement target in the form of a measurement point group by stereo processing of a plurality of camera images, a known shape information of the measurement target, and a relative position of the stereo camera. A step of predicting a visible portion from the posture, a step of determining a model point cloud according to the spatial density of the measurement point cloud using only the shape information of the predicted visible portion, and the measurement point cloud and the model point cloud The ICP algorithm is used for matching, and the step of adopting the position / orientation corresponding to the model group having the smallest evaluation function as the measurement result is taken.
Further, in the stereo image measurement method of the present invention, the model point group matching the spatial density of the measurement point group virtually creates a three-dimensional model from the known shape information of the measurement target, and the virtual model and each image sensor The point of intersection with the line of sight that passes through the center of the camera is calculated as a point of the model point group.
The stereo image measuring apparatus according to the present invention includes a plurality of cameras, a means for arithmetically processing the captured images of the cameras, and a storage means, and the arithmetic means stereo-processes the plurality of camera images. And a function for predicting the visible part from the relative position and orientation of the stereo camera from the known shape information, and the space of the measurement point group using only the predicted shape information of the visible part. A model point cloud that matches the density, and the measurement point cloud and model point cloud are matched using the ICP algorithm, and the position / orientation corresponding to the model cloud with the smallest evaluation function is used as the measurement result. It was supposed to have the function to do.

The stereo image measurement method of the present invention predicts the visible part from the known shape information of the measurement target and the relative position / orientation of the stereo camera, and uses only the predicted shape information of the visible part, and the spatial density of the measurement point group Since the model point cloud matched to the model is determined and used for ICP, the measurement accuracy and reliability of the method based on the combination of stereo processing and the ICP algorithm is compared to the case of using a model that considers the point cloud of the entire shape of the measurement target satellite Will improve.
Further, in the stereo image measurement method of the present invention, the model point group matching the spatial density of the measurement point group virtually creates a three-dimensional model from the known shape information of the measurement target, and the virtual model and each image sensor Since the point of intersection with the line of sight that passes through the camera center is calculated as the point of the model point cloud, it is more practical than when the density of the conventional model point cloud is simply spatially spaced. Yes, accuracy in 3D model matching is improved.
The stereo image measuring apparatus of the present invention comprises a plurality of cameras, a means for calculating the captured image of the cameras, and a storage means, wherein the calculating means takes a plurality of camera images. A function for obtaining a measurement point cloud by performing stereo processing, a function for predicting a visible part from a relative position / posture of a stereo camera from known shape information, and the measurement point group using only the predicted shape information of the visible part A function to extract a model point cloud according to the spatial density of the image, and matching between the measurement point group and the model point group using an ICP algorithm, and the position / orientation corresponding to the model group having the smallest evaluation function Therefore, it is possible to implement the position / posture measurement method of the measurement object with high accuracy and reliability as described above.

  FIG. 1 is a flowchart for explaining the stereo image measurement method of the present invention. In step 1 in the figure, a measurement target is photographed by a plurality of cameras. In step 2, the three-dimensional shape of the measurement target is stereo-processed from a plurality of camera images and obtained in the form of measurement point groups, and stored in the storage means. Step 3 includes the known shape information of the measurement target and the stereo camera. A step of predicting a visible portion from the relative position / posture of step 4, step 4 is a step of determining a model point cloud according to the spatial density of the measurement point cloud using only the shape information of the predicted visible portion, and step 5 is the step of In this step, the measurement point group and the model point group are matched using the ICP algorithm, and the position / orientation corresponding to the model group having the smallest evaluation function is employed as the measurement result.

  The largest feature point of the present invention is not to use the point cloud of the entire shape of the measurement target satellite for the model point group used for ICP, but to predict the visible part (front surface) from the relative position and orientation of the measurement target and the stereo camera. However, the accuracy is improved by using the model point group of only the visible part in the ICP. If the process of estimating the relative position and orientation between the measurement target and multiple cameras cannot predict what orientation the measurement target will take, the visible part (front If a set of model point groups corresponding to a plurality of positions / postures having different surfaces is created and stored in a storage device, the work efficiency is improved. However, when the measurement point cloud is obtained as a time series according to the attitude movement of the satellite floating in the outer space to be measured, it is possible to predict the future position and attitude from the time series data of the measured attitude. Since the model point group corresponding to the position / orientation can be used for ICP, the set of model point groups described above is not necessarily required.

FIG. 2 schematically shows an example of a measurement three-dimensional point group obtained by stereo viewing when a measurement object is photographed using two cameras. Here, for the sake of simplicity, the shape of the measurement target is assumed to be a box shape. The measurement point group is a set of points representing the three-dimensional position of only the part visible from the two cameras, and one point corresponds to each pixel of the image sensor of the camera.
FIG. 3 schematically shows a method for obtaining a model point group on the visible portion (front surface) from a three-dimensional full shape model based on known shape / pattern information (CAD data) of a measurement target. A set of points determined by calculating the intersection of the line of sight passing through the center of the camera from each pixel of the image sensor and the entire shape model plane is defined as a model point group. By adopting such a method for determining the point group, the distribution between the point groups is not evenly spaced, and a model point group that matches the spatial density of the measurement point group in FIG. 2 can be obtained.

  FIG. 4 shows a basic configuration diagram of the stereo image measuring apparatus of the present invention. A plurality of cameras 1a, 1b,... Are connected to a personal computer PC provided with calculation means 2 and storage means 3, and image data is processed by calculation processing means 2 according to a program stored in storage means (HDD or the like) 3. . In addition to the program for executing the measurement method of the present invention, the storage means 3 stores a set of shape / pattern information (CAD data) to be measured, measurement point group information as a calculation result, and model point group. First, images of the measurement target photographed by a plurality of cameras are stereo-processed by the calculation means 2 according to the program described above, and a three-dimensional shape of the measurement target is obtained in the form of a measurement point group and stored in the storage means 3. Subsequently, the calculation means 2 predicts the visible portion according to the program from the shape / pattern information of the measurement object stored in the storage means 3 and the relative position / posture information of the stereo camera at this time. Further, the computing means 2 virtually creates a three-dimensional model of only the predicted visible portion based on the known shape information of the measurement target, and the line of sight passing through the camera center 5 from the outer surface of the virtual model and each pixel 4 i of the image sensor 4. The model point group ΣMi is determined in accordance with the spatial density of the measurement point group obtained by calculating the intersection point Mi with, and this model point group information is stored in the storage means. Then, the arithmetic means matches between the measurement point group and the model point group using an ICP algorithm, specifies the model group having the smallest evaluation function, and adopts the position / orientation corresponding to the model group as the measurement result. .

  The present invention was developed as one of the elemental technologies necessary for the development of a spacecraft called “debris remover” that performs the capture, recovery, and dumping operations of a failed satellite floating in space. Is not limited to the field of space, and can be widely used for measuring the relative position and orientation with an object whose shape data is known.

It is a flowchart explaining the stereo image measuring method of this invention. It is the figure which showed typically the example of the measurement three-dimensional point group obtained by the stereo vision at the time of imaging | photography the measurement object using two cameras. It is the figure which showed typically the method of calculating | requiring the model point group on a visible part from the three-dimensional all shape model based on the shape and pattern information of a known measurement object. It is a figure which shows the basic block diagram of the stereo image measuring device of this invention.

Explanation of symbols

1a, 1b ... Stereo camera PC Personal computer 2 Calculation means 3 Storage means 4 Image sensor 4i Each pixel 5 Optical center O Measurement object ΣS Measurement point group ΣM Model point group

Claims (3)

  A step of obtaining a three-dimensional shape of a measurement target in the form of a measurement point group by stereo processing of a plurality of camera images, and a step of predicting a visible portion from the known shape information of the measurement target and the relative position and orientation of the stereo camera Determining a model point cloud that matches the spatial density of the measurement point cloud using only the predicted shape information of the visible part, and matching between the measurement point cloud and the model point cloud using an ICP algorithm A stereo image position measuring method including the step of adopting a position / orientation corresponding to a model group having the smallest evaluation function as a measurement result.   To determine the model point cloud according to the spatial density of the measurement point cloud, a virtual 3D model is created from the known shape information of the measurement target, and the intersection point between the virtual model and the line of sight passing through the camera center is calculated. The stereo image position measuring method according to claim 1, wherein the points are model point groups.   The apparatus includes a plurality of cameras, a means for arithmetically processing captured images of the cameras, and a storage means, and the arithmetic means stereo-processes the plurality of camera images and obtains them in the form of measurement point groups. A function that predicts the visible part from the function, the known shape information and the relative position and orientation of the stereo camera, and a model point cloud that matches the spatial density of the measurement point cloud using only the predicted shape information of the visible part. A function for determining, and a function for matching the measurement point group and the model point group using an ICP algorithm and adopting a position / orientation corresponding to a model group having the smallest evaluation function as a measurement result, Is a stereo image position measurement device having a function of storing known shape information of a measurement target, measurement point group information, and model point group information.

Images