JP6066562B2 - Measuring device, measuring method and program - Google Patents

Description

  The present invention relates to a technique for measuring at least one of the position and orientation of an object.

  Along with the development of robot technology in recent years, robots are instead performing complicated tasks that have been performed by humans, such as assembly of industrial products. Such a robot performs assembly by gripping a part with an end effector such as a hand. In order for the robot to grip the part, it is necessary to measure the relative position and orientation between the part to be gripped and the robot (hand). As a method of measuring the position and orientation, measurement is performed by model fitting in which a three-dimensional shape model of an object is applied to features detected from two-dimensional image data captured by a camera or distance image data obtained from a distance sensor. Is common. Non-Patent Document 1 discloses a method of measuring position and orientation using distance image data. In the method disclosed in Non-Patent Document 1, an object is represented in advance by a three-dimensional shape model, and the position and orientation of the object are measured by applying the three-dimensional shape model to the three-dimensional point cloud data converted from the distance image data. To do. Specifically, the Iterative Closest Point (ICP) algorithm disclosed in Non-Patent Document 2 is used to search for the nearest polygon based on the approximate values of position and orientation, and the sum of the distances between the points and the polygon is minimum. The position and orientation are updated repeatedly.

D. A. Simon, M.C. Hebert, and T.W. Kanade, “Real-time 3-D position estimation using a high-speed range sensor,” Proc. 1994 IEEE International Conference on Robotics and Automation (ICRA '94), pp. 2235-2241, 1994. P. J. et al. Besl and N.M. D. McKay, “A method for registration of 3-D shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, 1992.

  However, the above-described conventional technique is an algorithm for calculating the position and orientation by minimizing local information such as the distance (residual) between the nearest three-dimensional point and the three-dimensional shape model surface. Therefore, for an object with a non-smooth shape, such as an object that contains high-frequency unevenness, even if the residual remains as a whole, the local residual of some unevenness is reduced. Therefore, there is a high possibility that the calculation will converge to an incorrect position and orientation.

  To this problem, [Kok-Lim Low and Anselmo Lastra, “Reliable and Rapidly-Converging ICP Algorithm Usage Multi-Resolution 3”, “The International Conferencing 3”, “The International Conferencing 3”. As disclosed in Japanese Patent Application Laid-Open No. 2005-218867, there is known a method for estimating the position and orientation by converting the shape of an object into a smooth shape. In this method, a B-Spline curved surface is applied to a target point group, the point group is smoothed, and the uneven shape is converted into a smooth surface. It solves the estimation problem.

  However, since this method is based on the presumption of position and orientation estimation using 3D point groups that have the same data amount and shape, the distance image data and the 3D shape model are different. It cannot be applied to position and orientation estimation using data. In general, a three-dimensional shape model includes information about the entire circumference of an object, whereas distance image data stores only three-dimensional information of an object surface that can be observed from one viewpoint. Therefore, the B-Spline curved surface information applied to the three-dimensional shape model is greatly different from the B-Spline information applied to the distance image data. When smoothing is performed based on B-Spline curved surfaces having different parameters, the resulting shapes are greatly different from each other, and it is not effective to perform model fitting using the smoothed results. Therefore, the conventional method cannot be applied to the problem of high-frequency unevenness in model fitting using dissimilar data such as distance image data and a three-dimensional shape model.

  Accordingly, an object of the present invention is to stably measure the position and orientation of an object even when the shape of the object includes high-frequency irregularities.

The measuring apparatus of the present invention is an input unit that inputs image data obtained by imaging an object to be measured at least one of position and orientation, and the image data input by the input unit , Projection means for projecting a model representing the object, smoothing means for smoothing the image data and a model projected on the image data by the projection means, and the image smoothed by the smoothing means Corresponding means for associating the coordinates on the data with the coordinates on the smoothed model projected onto the image data, and on the basis of the result of the association by the associating means Measuring means for measuring at least one of the position and orientation, and the smoothing means includes the position and orientation of the object by the measuring means. In degree with the elapse of at least one of the measurement process, characterized by smoothing the image data.

  According to the present invention, it is possible to stably measure the position and orientation of an object even when the shape of the object includes high-frequency irregularities.

It is a figure which shows the structure of the information processing apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the process of the information processing apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows the structure of the information processing apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the process of the information processing apparatus which concerns on the 2nd Embodiment of this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

  First, a first embodiment of the present invention will be described. In the first embodiment, it is assumed that the outline of the position and orientation of an object is known, and the position and orientation of the object are estimated using a three-dimensional shape model and distance image data. That is, in the first embodiment, the same data as the distance image data in which the distance from the viewpoint to the 3D shape model is stored by projecting the target 3D shape model onto the image plane of the distance image data. A three-dimensional shape model is converted into a form. Then, after both data are smoothed, the position and orientation of the measurement target object (hereinafter simply referred to as “object”) are estimated. As a result, the position and orientation of the object can be stably measured.

  FIG. 1 is a diagram showing a configuration of an information processing apparatus 1 according to the first embodiment of the present invention. As illustrated in FIG. 1, the information processing apparatus 1 includes a three-dimensional model storage unit 110, a distance image input unit 120, an approximate position and orientation input unit 130, a three-dimensional model projection unit 140, a smoothing unit 150, an association unit 160, and a position. An attitude calculation unit 170 is provided. The 3D model data storage unit 110 stores the 3D shape model 10. The three-dimensional data measuring device 20 is connected to the distance image input unit 120. The information processing apparatus 1 measures the position and orientation of the object imaged in the distance image data based on the three-dimensional shape model 10 representing the shape of the object stored in the three-dimensional model storage unit 110. In the present embodiment, the three-dimensional shape model 10 stored in the three-dimensional model storage unit 110 conforms to the shape of an object that is actually imaged, for example, design three-dimensional CAD data of an industrial part. It is preferable. The information processing apparatus 1 is a configuration that is an application example of a measurement apparatus.

  The three-dimensional data measuring device 20 measures the distance between a plurality of points in the scene including the object and the three-dimensional data measuring device 20, and outputs distance image data. The distance image data here is image data in which the distance from the three-dimensional data measuring device 20 is stored for each pixel. In the three-dimensional data measurement apparatus 20 of the present embodiment, an active distance sensor that captures the reflected light of the laser light irradiated on the object and measures the distance by triangulation is used. However, the distance sensor is not limited to this. That is, a time-of-flight distance sensor that uses the time of flight of light may be used, or the distance from the three-dimensional data measuring device 20 is calculated by triangulation from image data captured by a stereo camera. A passive distance sensor may be used. In addition, any distance sensor may be used as long as the distance image data can be output.

  The distance image data output from the three-dimensional data measuring device 20 is input to the information processing device 1 via the distance image input unit 120. Internal parameters such as the focal length, principal point position, and lens distortion parameters of the three-dimensional data measuring device 20 are obtained by referring to the specifications of the three-dimensional data measuring device 20. [R. Y. Tsai, "A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV ceras and ens. RA-3, no. 4, 1987. The internal parameters may be obtained by calibrating the three-dimensional data measuring apparatus 20 in advance by the method disclosed in FIG.

  The distance image input unit 120 inputs distance image data from the three-dimensional data measurement device 20. The three-dimensional shape model storage unit 110 stores the three-dimensional shape model 10 of the object. In the present embodiment, an object is represented by a three-dimensional shape model 10 composed of a three-dimensional surface. That is, the three-dimensional shape model 10 is defined by a set of points and a set of surfaces formed by connecting the points.

  The approximate position and orientation input unit 130 inputs approximate values of the position and orientation of the object with respect to the information processing apparatus 1. Here, the position and orientation of the object with respect to the information processing apparatus 1 mean the position and orientation of the object in the camera coordinate system of the three-dimensional data measurement apparatus 20 that captures the distance image data. However, any part of the information processing apparatus 1 may be used as a reference if the relative position and orientation relative to the camera coordinate system are known and do not change. In the present embodiment, it is assumed that the information processing apparatus 1 continuously performs measurement in the time axis direction, and uses the previous (previous time) measurement value as an approximate value of the position and orientation. However, the method for inputting the approximate values of the position and orientation is not limited to this. For example, the speed and angular velocity of an object may be estimated using a time series filter based on past position and orientation measurements, and the current position and orientation may be predicted from the speed and angular velocity estimated for the past position and orientation. . In addition, distance image data of an object captured in various postures is stored as a template, and approximate values of the position and posture of the object are estimated by matching the input distance image data with the template (template matching). May be. In addition, when the position and orientation of the object can be measured by another sensor, the output value of the sensor may be used as the approximate value of the position and orientation. As a sensor capable of measuring the position and orientation of an object, for example, a magnetic sensor that measures the position and orientation by detecting a magnetic field generated by a transmitter with a receiver attached to the object can be cited. Moreover, you may use the optical sensor which measures a position and attitude | position by image | photographing the marker arrange | positioned on the object with the camera fixed. Any sensor may be used as long as it can measure the position and orientation with six degrees of freedom. Further, when the approximate position or posture where the object is placed is known in advance, the value may be used as the approximate value.

  The model projection unit 140 uses the approximate position and orientation values input by the approximate position and orientation input unit 130 on the 3D shape model storage unit 110 on the image plane of the distance image data. Project to. As a result of this projection processing, distance image data (hereinafter referred to as model distance image data) in which the distance from the viewpoint to the surface of the three-dimensional shape model is stored is generated.

  The smoothing unit 150 performs a smoothing process on the distance image data input from the distance image input unit 130 and the model distance image data generated by the model projection unit 140. The associating unit 160 associates the distance image data subjected to the smoothing process in the smoothing unit 150 with the model distance image data. The position / orientation calculation unit 170 measures the position and orientation of the object based on the result associated with the association unit 160.

  FIG. 2 is a flowchart showing processing of the information processing apparatus 1 according to the first embodiment of the present invention. Hereinafter, the processing of the information processing apparatus 1 according to the first embodiment will be described with reference to FIG.

  In step S1010, the information processing apparatus 1 performs an initialization process. That is, the approximate position and orientation input unit 120 inputs approximate values of the position and orientation of the object as initial values. In the present embodiment, the approximate position and orientation values input as the initial values are sequentially updated based on the captured distance image data, thereby measuring the position and orientation of the object. Therefore, before starting the measurement of the position and orientation of the object, it is necessary to give approximate values of the position and orientation of the object as initial values in advance. In the present embodiment, as described above, the measurement value of the previous time (previous time) is used as the approximate value of the position and orientation.

  In step S1020, the distance image input unit 130 inputs distance image data for calculating the position and orientation of the object by model fitting from the three-dimensional data measurement apparatus 20. Hereinafter, the distance image data input in the distance image input unit 130 is referred to as measurement distance image data. The measurement distance image data stores the distance from the three-dimensional data measurement apparatus 20 to the target including the object for each pixel.

  In step S1030, the three-dimensional model projection unit 140 converts the three-dimensional shape model 10 so as to have a data amount and property equivalent to the measurement distance image data. Specifically, the three-dimensional model projection unit 140 projects the three-dimensional shape model 10 on the image plane of the measurement distance image data input in step S1020. As a result, model distance image data storing the distance from the viewpoint to the three-dimensional shape model 10 is generated in the same manner as the measured distance image data. In the present embodiment, the three-dimensional shape model 10 is obtained by using the approximate values of the position parameter, posture parameter, and deformation parameter of the object input in step S1010 and the internal parameters of the calibrated three-dimensional data measurement apparatus 20. Render. By the rendering, the distance from the viewpoint to the surface of the three-dimensional shape model 10 is stored in the depth buffer. The three-dimensional model projection unit 140 handles the distance stored in the depth buffer as the distance stored in the model distance image data. With the above processing, the three-dimensional shape model 10 is converted into a data format equivalent to the measurement distance image data.

  In step S1040, the smoothing unit 150 performs a smoothing process on the model distance image data generated in step S1030 and the measured distance image data input in step S1020 based on neighboring pixels on the image plane. In the present embodiment, as the smoothing process, smoothing by a Gaussian filter with a standard deviation σ is performed in the range of N × N pixels near the target pixel. The standard deviation σ of the Gaussian kernel is a parameter that determines how much the model distance image data and the measured distance image data are smoothed. The larger the σ, the smoother the shape of the object. As a result, the position and orientation estimation processing is stabilized, but the estimated position and orientation are ambiguous as the shape is smoothed. N indicates a range defining the neighborhood, and the value may be arbitrarily determined or may be obtained from the standard deviation σ of the Gaussian kernel. As long as the model distance image data and the measurement distance image data can be smoothed, the setting of N is not limited. The smoothing process is not limited to application of a Gaussian filter, and any process that smoothes image data, such as applying an averaging filter or a median filter, may be used, and is not particularly limited. In the present embodiment, it is assumed that the standard deviation σ is set to 1 and the range N in the vicinity is set to 3, and smoothing processing using a Gaussian filter is performed. By performing the smoothing process as described above on all the pixels on the model distance image data and the measurement distance image data, the model distance image data and the measurement distance image data whose shape is processed smoothly can be obtained. Hereinafter, model distance image data whose shape has been smoothly processed is referred to as smoothed model distance image data, and measurement distance image data whose shape has been smoothly processed is referred to as smoothed measurement processing image data.

  Next, the smoothing unit 150 calculates the coordinates of the three-dimensional point group from the smoothed model distance image data and the smoothed measurement distance image data. That is, the smoothing unit 150 multiplies each pixel of the smoothed measurement distance image data by a line-of-sight vector corresponding to the position of the pixel and a distance stored in the pixel, thereby obtaining a three-dimensional point corresponding to each pixel. Find the coordinates of the group. Further, the smoothing unit 150 performs the same process on the smoothed model distance image data, and obtains a three-dimensional point group corresponding to each pixel of the smoothed model distance image data. However, in the case of the smoothed model distance image data, an invalid distance is stored in a portion that is not an area corresponding to the three-dimensional shape model 10, and therefore the three-dimensional point group of this portion is not obtained. For the smoothed model distance image data, the three-dimensional normal of each pixel is also obtained. Specifically, the three-dimensional normal of the pixel of interest is obtained by calculating the outer product of the three-dimensional coordinates of three points of pixels adjacent to the pixel of interest.

  By performing the above processing on all pixels of the smoothed model distance image data and the smoothed measured distance image data, a smooth shape is obtained for each of the smoothed model distance image data and the smoothed measured distance image data. A three-dimensional point group converted into is obtained.

  In step S1050, the association unit 160 associates the smoothed measurement distance image data generated in step S1040 with the smoothed model distance image data. In the present embodiment, it is assumed that the relationship between the pixels on the smoothed model distance image data and the three-dimensional point group is known, and for all the pixels in the region corresponding to the three-dimensional shape model 10 on the smoothed model distance image data. Then, the 3D point of the smoothed measurement distance image data having the same 2D coordinates on the image plane is referred to. As a result, the three-dimensional points of the smoothed model distance image data and the smoothed measurement distance image data are associated with each other. However, the processing is not performed on all the pixels in the region corresponding to the three-dimensional shape model 10 on the smoothed model distance image data so as to be equally spaced on the image plane in the region, or You may process by thinning out so that it may become equal intervals on three-dimensional space. Note that the three-dimensional point on which the association process is performed is a trade-off between the data amount and the calculation amount. As long as the correspondence sufficient for the position and orientation calculation processing is obtained, the selection of the three-dimensional point to be processed is not limited in any case. With the above processing, the three-dimensional points are associated between the smoothed measurement distance image data and the smoothed model distance image data.

  In step S1060, the position / orientation calculation unit 170 calculates the position and orientation of the object using nonlinear optimization calculation based on the association of the three-dimensional points between the smoothed measurement distance image data and the smoothed model distance image data. To do. In this embodiment, the Gauss-Newton method is used as the nonlinear optimization method. Note that the nonlinear optimization method is not limited to the Gauss-Newton method. For example, the Levenberg-Marquardt method, which is more robust in calculation, may be used, or the steepest descent method, which is a simpler method, may be used. Further, other nonlinear optimization calculation methods such as a conjugate gradient method and an ICCG method may be used.

  In addition, the calculation of the position and orientation of the object is performed by calculating approximate values of the position and orientation of the object so that each three-dimensional point in the smoothed model distance image data applies to the corresponding three-dimensional point in the smoothed measurement distance image data. This is done by repeatedly updating. More specifically, the position / orientation calculation unit 170 calculates a three-dimensional space from a three-dimensional point in the smoothed measurement distance image data and a three-dimensional point and a three-dimensional normal in the smoothed model distance image data. The signed distance between the point and the plane is linearly approximated as a function of the position and orientation of the object.

  Next, the position / orientation calculation unit 170 assembles a linear equation that holds for the position and orientation when the signed distance is 0, and solves this linear equation as a simultaneous equation. As a result, the position / orientation calculation unit 170 obtains minute changes in the position and orientation of the object, and updates the approximate values of the position and orientation. By repeating the above processing, the final position and orientation are calculated. Note that the error minimization processing between the smoothed model distance image data and the smoothed measured distance image data is not related to the essence of the present invention, and thus description thereof is omitted. For details of the minimization process, see Non-Patent Document [Y. Chen and G. Medioni, “Object modeling by registration range images,” Image and Vision Computing, 10 (3), pp. 145-155, 1992. Please refer to].

  In step S <b> 1070, the position / orientation calculation unit 170 determines whether or not the end of the position and orientation calculation processing has been instructed. When the end of the position and orientation calculation process is instructed, the process ends. On the other hand, if the end of the position and orientation calculation processing is not instructed, the processing returns to step S1010, and the information processing apparatus 1 acquires new measurement distance image data and measures the position and orientation of the object again. .

  As described above, in the present embodiment, the three-dimensional shape model is converted into the same format as the measurement distance image data, and the smoothing process is executed on both the model distance image data and the measurement distance image data. . This makes it possible to perform smoothing while keeping the shapes of the two similar. In this embodiment, by converting the object shape to be smooth, the instability of the position and orientation measurement processing caused by the unevenness of the object shape is reduced, and the object has an uneven shape. It is possible to estimate a stable position and posture.

  In the first embodiment, the parameters of the smoothing process in step S1040 are fixed. However, the parameter of the smoothing process is not limited to this. For example, the degree of smoothing processing, that is, the parameter may be changed according to the progress of the position and orientation measurement processing, that is, the number of loops (the number of branches in step S1070). Specifically, when the number of loops is small, the parameter of the smoothing process is set so as to be greatly smoothed, and as the number of loops is increased, the parameter of the smoothing process is set so as to reduce the smoothing. Also good. Thus, at the beginning of the iterative calculation, a stable position and orientation measurement process is performed by largely smoothing the shape of the object, and in the second half of the iterative calculation, the position and the position are reduced by reducing the smoothing of the object shape. It is possible to improve the accuracy of the posture measurement process. The method for setting the parameters for the smoothing process is not limited to the above-described methods, and any method may be used as long as the parameters that enable normal smoothing processing are set.

  In the first embodiment, the three-dimensional shape model 10 is premised on a model constituted by a set of faces. However, the model format for representing an object is not limited to this. For example, a model constituted by a set of three-dimensional points may be used. In this case, in the conversion process of the three-dimensional shape model 10 in step S1030, without rendering the model, each three-dimensional point constituting the model is projected onto the image plane of the measurement distance image data, thereby Generate range image data. Specifically, the three-dimensional configuration of the model using the approximate values of the position parameter, posture parameter, and deformation parameter of the object input in step S1010 and the internal parameters of the calibrated three-dimensional data measurement apparatus 20 Project points onto the image plane. Then, the model distance image data is generated by storing the distance from the viewpoint to the point constituting the model in the pixel on the image plane corresponding to the three-dimensional point. In addition, the format which comprises the three-dimensional shape model 10 is not restricted to the method mentioned above, As long as it is a format which can be converted into the data format equivalent to measurement distance image data from the three-dimensional shape model 10, Any format may be used.

  Next, a second embodiment of the present invention will be described. In the first embodiment, the method of measuring the position and orientation of the object by converting the three-dimensional shape model into the distance image data format and executing the smoothing process on the entire model distance image data has been described. In contrast, in the second embodiment, the three-dimensional shape model is not converted into the distance image data format, but is smoothed for each three-dimensional point on the model surface defined on the three-dimensional shape model, A method for measuring the position and orientation of an object will be described.

  FIG. 3 is a diagram showing a configuration of the information processing apparatus 2 according to the second embodiment of the present invention. As illustrated in FIG. 3, the information processing apparatus 2 includes a 3D model storage unit 210, a distance image input unit 220, an approximate position / posture input unit 230, a 3D model projection unit 240, a smoothing unit 250, an association unit 260, and a position. An attitude calculation unit 270 is provided. The three-dimensional model data storage unit 210 stores the three-dimensional shape model 10. The three-dimensional data measuring device 20 is connected to the distance image input unit 220. The information processing apparatus 2 measures the position and orientation of the object imaged in the distance image data based on the three-dimensional shape model 10 representing the shape of the object stored in the three-dimensional model storage unit 210. In addition, it is preferable that the 3D shape model 10 in the second embodiment conforms to the shape of the object that is actually imaged, similarly to the 3D shape model 10 in the first embodiment.

  The three-dimensional shape model storage unit 210 stores the three-dimensional shape model 10 of the object whose position and orientation are to be measured. In the present embodiment, an object is represented by a three-dimensional shape model 10 composed of three-dimensional points having normal line information. That is, the three-dimensional shape model 10 is defined by a set of three-dimensional points and a set of normal directions of the points with respect to each of the three-dimensional points.

  The three-dimensional model projection unit 240 uses the approximate values of the position and orientation input from the approximate position and orientation input unit 130 for the three-dimensional shape model 10 stored in the three-dimensional shape model storage unit 210 to calculate the distance image data. Project onto the image plane. As a result of this projection processing, two-dimensional coordinates on the image plane of the three-dimensional point group in the three-dimensional shape model are calculated.

  The association unit 250 associates the three-dimensional point group in the three-dimensional shape model 10 with the distance image data. The smoothing unit 260 performs a smoothing process on the three-dimensional point group in the three-dimensional shape model 10 and the three-dimensional point group in the distance image data.

  The distance image input unit 220, the approximate position and orientation input unit 230, and the position and orientation calculation unit 270 are the same as the distance image input unit 120, the approximate position and orientation input unit 130, and the position and orientation calculation unit 170 in the first embodiment. Therefore, description of these will be omitted.

  FIG. 4 is a flowchart showing processing of the information processing apparatus 2 according to the second embodiment. Hereinafter, the processing of the information processing apparatus 2 according to the second embodiment will be described with reference to FIG.

  In step S2010, the information processing apparatus 2 performs an initialization process. Note that step S2010 is the same processing as step S1010 in the first embodiment, and thus detailed description thereof is omitted.

  In step S2020, the distance image input unit 220 inputs distance image data for calculating the position and orientation of the object by model fitting from the three-dimensional data measurement apparatus 20. Note that step S2020 is the same processing as step S1020 in the first embodiment, and thus detailed description thereof is omitted.

  In step S2030, the associating unit 250 associates the three-dimensional point group in the three-dimensional shape model 10 with the distance image data. Specifically, the associating unit 250 uses the three-dimensional point group in the three-dimensional shape model, the approximate value of the position and orientation of the object input in step S2010, and the calibrated three-dimensional data measuring apparatus 20 Using the internal parameters, the distance image data is projected onto the image plane. Thereby, the three-dimensional point group of the three-dimensional shape model 10 and the three-dimensional point group in the distance image data are associated with each other. The method for calculating the coordinates of the three-dimensional point group is the same as the process described in step S1040. When the association is completed for all the three-dimensional point group points in the three-dimensional shape model 10, the process proceeds to step S2040.

  In step S2040, the smoothing unit 260 applies the three-dimensional point group in the three-dimensional shape model 10 and the three-dimensional point group in the distance image data associated in step S2030 to neighboring pixels on the image plane. Based on the smoothing process. That is, the smoothing unit 260 calculates the coordinates of other three-dimensional points that are within N × N pixels on the image plane for each of the three-dimensional points of the three-dimensional shape model 10, and The smoothing process is performed by adopting the average of the coordinates as the coordinate value of the new three-dimensional point. This process is performed in the same manner for the three-dimensional point group of the three-dimensional shape model 10 and the three-dimensional point group in the distance image data. There is no particular restriction on the setting of the value of N that defines the neighborhood on the image plane. In the present embodiment, the average of the coordinates of neighboring three-dimensional points is employed as the smoothing processing of the coordinates of the three-dimensional points, but the present invention is not limited to this. For example, the weight in the three-dimensional space may be increased as the distance is closer to the coefficient. By applying the above processing to the three-dimensional point group in the three-dimensional shape model 10 and the three-dimensional point group in the distance image data, a smooth shape is obtained for each of the three-dimensional shape model 10 and the distance image data. A three-dimensional point group converted into is obtained.

  In step S2050, the position / orientation calculation unit 270 uses nonlinear optimization calculation based on the three-dimensional point group in the three-dimensional shape model subjected to the smoothing process in step S2040 and the three-dimensional point group of the distance image data. To calculate the position and orientation of the object. Since the optimization calculation process is the same process as step S1060 in the first embodiment, a detailed description thereof will be omitted.

  In step S2060, the position / orientation calculation unit 270 determines whether or not an instruction to end the position and orientation calculation processing has been input. If an instruction to end the position and orientation calculation process is input, the process ends. On the other hand, if the instruction to end the position and orientation calculation processing is not input, the processing returns to step S2010, and the information processing apparatus 2 acquires new distance image data and measures the position and orientation of the object again. To do.

  As described above, in the present embodiment, smoothing processing based on the neighborhood on the image plane is performed on the three-dimensional point group in the three-dimensional shape model and the three-dimensional point group in the distance image data to obtain a smooth shape. It has been converted. As a result, the instability of the position and orientation measurement processing caused by the unevenness of the shape of the object is reduced, and the stable position and orientation can be estimated for an object having unevenness in the shape of the object.

  In the first and second embodiments, it is assumed that the position / orientation calculation unit 170 and the position / orientation calculation unit 270 calculate both the position and orientation of the object. However, the present invention is not limited to the method of estimating both the position and orientation parameters. For example, there is no particular problem even if only one of the position and orientation is calculated. In this case, in steps S1060 and S2050, the approximate values input in step S1010 or S2010 are set for parameters that are not estimated (position parameters when position parameters are estimated and position parameters when position parameters are estimated). The processing may be repeated as a fixed value.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  1, 2: Information processing device, 10: Three-dimensional shape model, 20: Three-dimensional data measurement device, 110, 210: Three-dimensional model storage unit, 120, 220: Distance image input unit, 130, 230: Approximate position and orientation input , 140, 240: three-dimensional model projection unit, 150, 250: smoothing unit, 160, 260: association unit, 170, 270: position and orientation calculation unit

Claims (5)

Input means for inputting image data obtained by imaging an object to be measured at least one of position and orientation;
Projecting means for projecting a model representing the object on the image data input by the input means;
Smoothing means for smoothing the model projected on the image data by the image data and the projection means ;
Association means for associating coordinates on the image data smoothed by the smoothing means with coordinates on the model projected onto the image data and smoothed ;
Measuring means for measuring at least one of the position and orientation of the object based on the result of association by the association unit;
The measurement apparatus characterized in that the smoothing means smoothes the image data at a degree corresponding to the progress of at least one of the position and orientation of the object by the measurement means. The image data is image data including, for each pixel, a distance from an imaging device that captured the image data to the object, and the projecting unit projects the model by projecting the model onto the image data. The measurement apparatus according to claim 1, wherein image data including a distance from the apparatus to the model is generated for each pixel . The measuring apparatus according to claim 1, wherein the smoothing unit smoothes the image data and a model projected on the image data in a range in the vicinity of a target pixel. . A measurement method executed by a measurement device,
An input step of inputting image data obtained by imaging an object to be measured at least one of position and orientation;
A projecting step of projecting a model representing the object to the image data input in the input step;
A smoothing step of smoothing a model projected on the image data and the image data in the projecting step;
An associating step for associating the coordinates on the image data smoothed in the smoothing step with the coordinates on the model that are projected and smoothed on the image data;
A measurement step of measuring at least one of the position and orientation of the object based on the result of association in the association step;
Including
In the smoothing step, the image data is smoothed at a degree corresponding to the progress of the measurement process of at least one of the position and orientation of the object in the measurement step. An input step of inputting image data obtained by imaging an object to be measured at least one of position and orientation;
A projecting step of projecting a model representing the object to the image data input in the input step;
A smoothing step of smoothing a model projected on the image data and the image data in the projecting step;
An associating step for associating the coordinates on the image data smoothed in the smoothing step with the coordinates on the model that are projected and smoothed on the image data;
A measurement step of measuring at least one of the position and orientation of the object based on the result of association in the association step;
To the computer,
In the smoothing step, a program for smoothing the image data to a degree corresponding to the progress of at least one of the position and orientation of the object in the measurement step.

Images