JP6071522B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to an information processing apparatus that performs a model fitting process by associating a shape model of a target object with an observation point, and a control method thereof.

  The method of estimating the position and orientation of a shape model so as to reduce the difference between the shape model of the target object and the observed value in the image (distance image, etc.) of the target object is called model fitting. It is known as one of the techniques for estimating the position and orientation without contact. This technique is used when the position and orientation of a component (target object) placed at an unspecified position and orientation is measured by a vision sensor, such as bin picking of a component by a robot.

  Here, the shape model is a representation of the shape of the observed surface of the target object, for example, a combination of local planes such as a polygon model. When calculating the difference between the shape model and the observed value of the target object, each geometric shape that forms the shape corresponds to the edge of the geometric feature included in the observed image and the distance point of the distance image It is necessary to apply

  As a method of associating a shape model with an observation point, a method of associating the nearest observed geometric feature with each geometric shape is disclosed (for example, see Non-Patent Document 1). This method is effective for a part having no feature such as texture on the surface of the target object.

  The process of associating the geometric features is performed for each step of estimating the nonlinear parameter of the translation / rotation component of the position / orientation estimation. In general, since the amount of data of distance images and image features is enormous, a technique for improving the efficiency of this association processing is required.

  Regarding efficient association between the distance point of the distance image and the surface of the shape model, a method for storing the distance point in a spatially divided range and improving the search time is disclosed (for example, Non-patent document 2). As for the association between the edge point in the image and the shape model, a method of searching by limiting the search range of the image to one dimension in consideration of the direction of the geometric shape from the captured image is known (for example, And Patent Document 1).

Japanese Patent No. 4649559

Paul J. Besl, Neil D. McKay, “A Method for Registration of 3-D Shapes”. IEEE Transactions of pattern analysis and machine intelligence, vol.14, no.2, (1992). ZHENYOU ZHANG, “Iterative Point Matching for Registration of Free-Form Curves and Surfaces”. International Journal of Computer Vision, 13-2, pp.119-152, (1994).

  In the conventional model fitting process, the process of associating the shape model with the observation point is performed for each position and orientation estimation parameter update step, and the amount of processing is enormous. The technique described in Patent Document 1 described above has a problem in that the calculation load is large because the distance from each pixel of interest to the feature portion (edge portion) having the shortest distance is calculated.

  In view of the above-described problems, an object of the present invention is to provide an information processing apparatus and an information processing method capable of performing association between a shape model of a target object and an observation point at high speed in model fitting processing.

  As a means for achieving the above object, an information processing apparatus of the present invention comprises the following arrangement.

That is, information for associating a model point indicating a geometric feature in a model indicating the shape of the target object with an observation point indicating the geometric feature of the target object included in an image obtained by measuring the target object A processing device comprising:
Attaching an index value to the coordinate value of the observation point, a first holding means for holding a relationship between the index value and the coordinate value,
Second holding means for converting the coordinate value and holding an index map in which the index value is stored in a pixel corresponding to the converted coordinate value;
A model point coordinate conversion means for converting the coordinate values of the model point coordinate values definitive the coordinate system of the index map,
Obtaining means for obtaining an index value stored in a pixel of the index map corresponding to the coordinate value transformed by the model point coordinate transformation means;
The observation point corresponding to the model point is specified based on the index value acquired by the acquisition means.

  According to the present invention, in the model fitting process, the shape model of the target object and the observation point can be associated at high speed.

A block diagram showing a configuration of an information processing device in the first embodiment, A flowchart showing an index map generation process in the first embodiment, Conceptual diagram of index map generation in the first embodiment, A flowchart showing residual calculation processing of model points and observation points in the first embodiment, FIG. 6 is a block diagram showing a configuration of a position / orientation measurement apparatus according to a second embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention related to the scope of claims, and all combinations of features described in the present embodiments are essential for the solution means of the present invention. Is not limited.

<First Embodiment>
In the present invention, in order to associate an observation point with a model point, an index of a table storing observation point position information, which is a detected geometric feature, is held as a pixel, and the pixel value is extended to surrounding pixels. Create an index map to By using this index map, the correspondence between model points and observation points can be obtained at higher speed, that is, higher-speed model fitting is possible.

  More specifically, the present invention relates to an information processing apparatus that associates a model point in a model indicating the shape of a target object with observation points included in an image obtained by measuring the target object. To realize. First, an index value is assigned to the coordinate value of the observation point indicating the geometric feature of the target object, and the relationship between the index value and the coordinate value is held in the first holding means. Then, the coordinate value held by the first holding unit is converted, and the index map in which the index value is stored in the pixel indicated by the converted coordinate value is held in the second holding unit. Then, the coordinate value of the model point indicating the geometric feature in the model is converted into the coordinate system of the index map, and the index value stored in the pixel of the index map corresponding to the converted coordinate value is acquired. Based on the index value acquired in this way, an observation point corresponding to the model point is specified.

Apparatus Configuration FIG. 1 shows a block configuration example of the information processing apparatus in the present embodiment. This embodiment
In the model fitting method that estimates the position and orientation of the model so that the difference (residual) between the shape model of the target object (hereinafter referred to as the model) and the observation information of the target object is reduced, the difference between the observation information and the model is calculated. This is related to the process of associating. Specifically, information on observation points, model points, and conversion parameters that convert the model points to match the observation coordinate system are input, and the associated observation points and model points are stored in the observation coordinate system. The difference is output.

  Note that the observation points in the present embodiment are geometric feature points included in a distance image or a grayscale image, and the geometric features are distance points for a distance image and edges or the like for a grayscale image. In the present embodiment, an example using a grayscale image will be described, but the present invention is not limited to this, and can be applied to any two-dimensional image such as a color image. Further, the observation point may be a feature point representing a local density distribution of the texture. That is, any geometric feature may be used as long as the geometric feature can correspond to a point detected by actually observing the measurement target object.

  The model point is a point that can correspond to a geometric feature actually observed from the shape of the measurement target object. The model point for the distance image is a point on the shape surface. When the edge of the grayscale image is a geometric feature, the position of the edge where the density gradient observed from the model shape changes can be used as the model point. Any combination of geometric features may be used as long as the geometric features of the observation points can be associated with the geometric features of the model points.

  Hereinafter, each configuration of the information processing unit 100 in the present embodiment will be described. The observation point input unit 110 inputs a coordinate value of a geometric feature included in a distance image or a grayscale image that is a target for calculating a residual with the model as an observation point. Specifically, a distance point is input for a distance image, and a coordinate value indicating the position of an edge point is input for a grayscale image.

  The observation point table holding unit 140 stores the coordinate values input as observation points by the observation point input unit 110 by assigning indexes so as not to overlap, as a table.

  The observation point coordinate conversion unit 150 converts the coordinates of the observation point into an index map coordinate system. When the observation point has a three-dimensional coordinate value, a calculation (projection calculation) is performed to project the observation point onto a two-dimensional (observation screen) coordinate value. In this projection calculation, parameters are prepared so that the projected two-dimensional observation screen matches the size of the index map image described later, that is, conversion into the index map coordinate system is performed. To do. When the observation point is a two-dimensional coordinate value, translation / expansion to the index map coordinate system is performed.

  Based on the coordinate values converted by the observation point coordinate conversion unit 150, the index map reference area processing unit 160 writes the index values held in the observation point table holding unit 140 into the index map, and writes the index values to the surrounding areas. Propagate to. The index map holding unit 170 holds the index map processed by the index map reference region processing unit 160 so that it can be referred to as an image. For example, it is stored as an index map image composed of an image format array that stores values indicating indexes in the observation point table. The index value may be a direct array number, but a conversion value of a predetermined reference value such as a hash value may be used.

  Next, a configuration for comparing with the coordinate value of the geometric feature on the model surface of the target object to be measured will be described.

  The model point coordinate parameter input unit 120 uses, as model point coordinate parameters, translation / rotation parameters for geometric transformation of model points expressed in the model coordinate system to the same observation coordinate system (view coordinate system) as the observation points. input. In general, the model point coordinate parameter is the value of the model view transformation matrix that performs transformation from the model coordinate system to the view coordinate system, and corresponds to the observation points such as the camera position and orientation, focal length, and principal point position. It is desirable to be a value. The model point coordinate parameter value is assumed to be an estimated value of the position / orientation parameter, but if only the residual amount is to be calculated, it may be the parameter value of the position / orientation to be compared. Absent.

  The model point input unit 130 inputs coordinates representing the position on the model surface in the model coordinate system corresponding to the geometric feature in the measurement target object as model points. Here, as a model point, when a distance image is an observation target, a visible point on a surface polygon in a shape model of a measurement target object is used. As long as it can be used as shape information, the model point may be expressed as an expression other than a polygon, for example, an expression by an analytical curved surface and a boundary, or a volume expression by voxel. In other words, the model point may be expressed in any form as long as it is a shape expression that provides information on the observed object surface.

  Further, when an edge in the image is an observation target, a point on the polygon side of the structurally visible roof edge or jump edge of the measurement target object is used as a model point. Here, what is associated with the geometric feature that can be detected from the observation information of the measurement target object is the model point information, and any geometric feature may be used as long as at least position coordinates in the model coordinate system are included. For example, a geometric feature such as a character printed on a sticker attached to the object surface may be used as the model point information.

  The model point coordinate conversion unit 180 first multiplies the model view conversion matrix input by the model point coordinate parameter input unit 120 by the coordinate value of the model point input by the model point input unit 130, thereby obtaining a model coordinate system. Convert the model point to the observation coordinate system. The coordinate values of the model points in the observation coordinate system (hereinafter referred to as model view coordinate values) converted here are output to the observation point / model point residual calculation unit 200 and used for residual calculation. The model point coordinate conversion unit 180 further performs projective conversion from the model view coordinate value to the index map coordinate value. This coordinate transformation requires a projective transformation matrix. As the value of the projective transformation matrix, an image parameter held by the index map holding unit 170 may be used, or may be input from the model point coordinate parameter input unit 120.

  The index map reference unit 190 refers to the value of the index map registered in the coordinate value of the index map coordinate system converted by the model point coordinate conversion unit 180, and acquires the value as an index value corresponding to the observation point. . Thereby, the observation point registered in the observation point table is substantially associated with the model point. Since the index map can be referred to as an image, if the index map is held as a memory, an offset of the coordinate system may be calculated to refer to the address.

  The observation point / model point residual calculation unit 200 outputs a residual by calculating a difference between the associated observation point and the model point. First, based on the index value obtained from the index map reference unit 190, the coordinate value of the observation point corresponding to the model point currently being processed is referred to by referring to the data having the same index value in the observation point table holding unit 140 Get. Then, from the obtained coordinate value of the observation point and the coordinate value (model view coordinate value) of the model point converted into the observation coordinate system by the model point coordinate conversion unit 180, the distance between the two points is calculated as a residual. To do. As a residual calculation method, for example, when the observation target is a distance image, the Euclidean distance of a three-dimensional point may be calculated. By approximating the periphery of the observation point in the distance image as a local plane, the normal of that plane can be calculated. Alternatively, the normal of the model surface can also be obtained. If the inner product of the vector between two points of the associated geometric feature and the normalized one of the normals is calculated, the residual according to the normal direction of the surface can be calculated. In addition, when using an edge as an observation point, a two-dimensional distance in the observation coordinate system may be calculated. Further, in the case of an edge, a gradient can be detected from an image by using a direction detection filter such as a Sobel filter. By calculating the inner product of the vector of the model position and the edge position with the normalized vector of the gradient direction, a residual considering the gradient direction can be calculated. The residual calculation formula can be set by the user.

  The observation point / model point residual output unit 210 obtains the residual calculation value, the model point used there, the coordinates of the converted model point, the observation point table, as a processing result by the observation point / model point residual calculation unit 200. The coordinate value of, etc. is output. These are used as Jacobian matrix values when calculating the position and orientation estimation parameters in model fitting, and the position and orientation parameters are estimated so that the residual becomes small.

  The configuration of the information processing apparatus shown in FIG. 1 described above can be executed as a program on a general program operation platform such as a CPU, a memory, a bus connecting them, and an input / output device. Therefore, the present embodiment can be operated as a part of the position / orientation estimation processing program. Further, the apparatus may function as a hardware configuration in which information of observation points and model points is input via an input / output interface, and their correspondence and residual are output. Alternatively, data may be transmitted to the interface via a network, executed by a device at another location, and the result received and used. It is useful that the program can be copied from a medium, downloaded from a network, and used as a part of an application because it can be operated on various devices.

Index Map Generation Processing Hereinafter, the flow of index map generation processing performed in the configuration from the observation point input unit 110 to the index map holding unit 170 will be described using the flowchart of FIG. 2 and the conceptual diagram of FIG.

  First, in S100, the index map secured in the index map holding unit 170 at the start of processing is initialized, and then in S110, the observation point table secured in the observation point table holding unit 140 is initialized.

  Next, in S120, information regarding the observation point is input from the observation point input unit 110. When the observation target is a distance image, the observation point information is a three-dimensional value of X, Y, and Z. If the observation point is an edge included in the image, the observation point information is a two-dimensional value of X and Y.

  Next, in S130, an index for referring to the observation point table is assigned to the coordinate value of the observation point input in S120, and then the coordinate value is additionally stored in the observation point table holding unit 140. FIG. 3 shows an example in which the coordinates (X, Y, Z) of the distance image obtained in S120 are stored in the index value “25” in the observation point table. When the observation points are expressed using a memory array, a memory structure indicating the observation coordinates and a register holding the index are used to store the size of the memory structure in a series of memory spaces. Just slide and write the value.

  In S140, the coordinate value of the observation point obtained in S130 is converted into the coordinate system of the index map. When the edge of the image is used as the observation point, the observation point is a captured image, and the coordinates of the observation point can be used as they are by setting the index map to an image of the same size. On the other hand, when using a distance image as an observation point, it is necessary to perform projective conversion from the three-dimensional information of the observation point to the two-dimensional information. Therefore, conversion coefficients and offset values for the index map coordinate system can be set in advance, and conversion can be performed based on the values. For example, if the center of the index map image is (cx, cy) and the conversion coefficient is f, an arbitrary point (X, Y, Z) in the distance image can be represented by the following formula (1), (2). Converted to coordinate system (u, v). Since the coordinate value is referred to as an integer value, it is assumed that the expressions (1) and (2) perform a floor () operation that rounds off the decimal point.

u = floor (f (X / Z) + cx) (1)
v = floor (f (Y / Z) + cy) (2)
Next, in S150, in the index map, the index value assigned to the observation point coordinates in S130 is written as the pixel value indicated by the coordinate values converted in S140. Therefore, the information amount of each pixel of the index map requires a number of bits that can store the upper limit of the observation point table array. As long as the image size is generally used, each pixel only needs to have an unsigned 16-bit size. The number of bits is switched according to the image size and resolution to be processed.

  Here, in FIG. 3, the coordinates (X, Y, Z) of the observation point are converted into the coordinates (u, v) of the index map by the above formulas (1) and (2) in S140. In S150, the index value “25” in the observation point table corresponding to the observation point coordinates (X, Y, Z) is written in the coordinates (u, v) of the index map.

  In S160, it is checked whether or not the coordinate input of the observation point has been completed. If it has not been completed yet, the process returns to S120 and the above processing is repeated. If it has been completed, the process proceeds to S170.

  In S170, reference area processing for the index map is performed. That is, in the index map, the reference area is set so that the index value written in S150 is extended to surrounding pixels. Specifically, when the index map value is written in the target pixel, the process of writing the index value in the surrounding pixels may be repeated. As this image processing, general dilation processing (Dilate) can be applied. Further, by applying Voronoi diagram generation processing, it is also possible to set an index area for peripheral pixels. Here, since the Voronoi diagram is created so as to hold the value of the nearest pixel at the target pixel position, it can be used as an index map of this embodiment. As the reference image processing, any method may be used as long as the region can be expanded so that the index value written in S150 can be propagated to adjacent pixels whose values are not set. Absent. Here, it is only necessary to propagate the numerical value of the index to surrounding pixels, and it is not necessary to calculate the distance or gradient from the original reference point unlike the distance map described in Patent Document 1 above. A map can be created at high speed.

  In S180, the index map holding unit 170 sets and holds the index map image, which is the processing result in S170, so that it can be referred to.

  Here, according to FIG. 3, by performing the reference area processing in S170 on the index map in which the index values “24” and “25” are written, for example, by the process up to S160, the area of each index value is changed. Expanded. Then, the finally generated index map is held in S180.

  Here, in the conventional model fitting process, a process for partially searching the periphery of the pixel of interest to be referred to is performed, so a processing time by a plurality of random accesses to the memory is required, and the processing time is increased. There was a problem. On the other hand, in the present embodiment, as shown in FIG. 3, for the target pixel in the index map, the same index value as that of the target pixel is held for the surrounding pixels. Accordingly, it is possible to perform the nearest neighbor association calculation only by referring to the target pixel, that is, the number of accesses to the memory is reduced, and the processing time can be shortened.

  For convenience of explanation, the steps shown in the flowchart of FIG. 2 are described as sequential processing. However, the processing for registering input values in S120 to S160 and the reference region processing in S170 are performed in parallel for each data. Thus, the processing time can be further shortened.

  Although an example in which the index map is configured by an image format array is shown here, the information amount of each pixel element may be reduced by holding the index map only in the region of interest.

Residual calculation processing The following is the processing performed in the configuration from the model point coordinate parameter input unit 120 to the observation point / model point residual calculation unit 200 to calculate the residual by associating the model point with the observation point. This will be described in detail with reference to the flowchart of FIG.

  First, in S200, the model point coordinate parameter input unit 120 receives a model view conversion matrix for converting a model point to the model view coordinate system as a conversion parameter to the index map coordinate system. In S210, the model point input unit 130 inputs the coordinate value (X, Y, Z) of the model coordinate system as a model surface point (model point) to be matched.

  In S220, the model point coordinate conversion unit 180 converts the coordinate value (X, Y, Z) of the model point into the index map coordinate system. If the observation point is a point of the distance image, first, as shown in the following equation (3), the coordinate value (X, Y, Z) of the model point is converted into the model view coordinate value using the model view transformation matrix. Convert to (X ', Y', Z '). In Equation (3), R is a 3 × 3 matrix for converting the rotation component, t is a 3 × 1 matrix for converting the translation component, and is the conversion parameter input in S200.

  Then, by using the model view coordinate values (X ′, Y ′, Z ′) converted into the observation coordinate system by the expression (3) using the parameters of the above expressions (1) and (2), the following (4 ) And (5) are converted into coordinates (u ′, v ′) in the index map coordinate system.

u '= floor (f (X' / Z ') + cx) (4)
v '= floor (f (Y' / Z ') + cy) (5)
Even when the observation point indicates an edge, the model point is assumed to be held as a three-dimensional point on the surface of the target object, and therefore, the above equations (3) to (5) can be applied.

  In step S230, the index map reference unit 190 refers to the pixel value at the coordinates (u ′, v ′) in the index map held in the index map holding unit 170. Here, since the index value in each observation point table held in the observation point table holding unit 140 is written in each pixel of the index map, that is, in S230, the index value of the observation point table is acquired. As a result, the model points input in S210 are substantially associated with the observation points registered in the observation point table.

  In step S240, the observation point / model point residual calculation unit 200 identifies a corresponding observation point from the index value in the observation point table acquired in step S230. If an empty index value is set for a pixel in the index map, the processing is performed assuming that there is no corresponding observation point.

  In S250, the observation point / model point residual calculation unit 200 calculates a residual between the associated observation point and model point. When the observation point is a three-dimensional coordinate, the model view coordinate value (X ', Y', Z ') of the observation coordinate system calculated by the above equation (3) and the observation point set in the observation point table The Euclidean distance with respect to the coordinate value (X, Y, Z) may be calculated, or the difference between each component may be calculated. If the observation point is a two-dimensional coordinate, the value of (u ', v') calculated by the above equations (4) and (5) and the Euclidean distance of (X, Y) in the observation point table May be calculated, or the difference between each component may be calculated. Here, it is assumed that the residual calculation method can be switched by setting. Therefore, the residual calculation may be performed by another processing unit, and the associated coordinate value or index value may be output to the other processing unit. In that case, in this embodiment, the setting for not performing the residual calculation may be used.

  In the index map of this embodiment, only the correspondence between each pixel and the observation point table is held. Therefore, since the residual calculation indicating the distance between the associated model point and the observation point is performed each time, the calculation accuracy of the discretization error due to pixel reference does not deteriorate. Therefore, it is possible to calculate the residual with higher accuracy than the method of referring to the distance information calculated in advance like the distance map described in Patent Document 1. Also, since the index map generation process is simple, it is superior in that it requires less data storage capacity and processing time than the process of generating a data structure such as a kd tree. Yes.

  As described above, according to the present embodiment, high-speed model fitting processing can be performed by associating observation points with model points using an index map. In addition, since the distance point included in the distance image, the edge included in the image, or the like can be used as the observation point indicating the geometric feature, the application range is wide.

<Second Embodiment>
Hereinafter, a second embodiment according to the present invention will be described. In the second embodiment, a position / orientation estimation apparatus including the model fitting function shown in the first embodiment described above is shown.

  FIG. 5 shows a block configuration example of the position / orientation estimation apparatus according to the second embodiment. As shown in the figure, the position / orientation estimation apparatus measures the position / orientation parameters of the measurement target object 10 using the projector 300 that projects a pattern and the camera 310 that captures a reflection image of the measurement target object 10. It is characterized by that.

  The projector 300 has a function of projecting a pattern, and generally includes a light source and a screen that forms the pattern, and can illuminate a video pattern from a computer. The camera 310 is arranged to photograph the projected pattern, and the photographing timing is controlled by the target photographing unit 330 so that photographing (synchronous photographing) is performed in accordance with the projection timing of the projector 300. Here, as the projection pattern in the projector 300, a pattern that can correspond to the position of the captured image and the projected image of the projector, such as a spatial encoding method using a Gray code or a phase shift method, is assumed. The distance can be calculated by the triangulation method from the arrangement relationship associated with this pattern.

  The distance image generation unit 340 calculates the distance value of each pixel from the correspondence between the projection coordinates of the projector 300 and the coordinates of the captured image, using the image on which the pattern is projected from the projector 300. Is generated. A distance point in this distance image is input to the information processing apparatus 101 described later (first observation point input) as an observation point (first observation point) in the first embodiment. Note that the calculation of the distance value requires camera parameters in both the projector 300 and the camera 310, but these parameters are assumed to have been set after being calibrated in advance.

  The image edge detection unit 350 performs image processing for detecting geometric features such as a roof edge and a jump edge from a captured image of the measurement target object 10 in a state where the projector 300 does not project a pattern. For edge detection, an image detection filter such as a Sobel filter or a Canny filter may be used. That is, an edge having an intensity equal to or higher than the threshold set as the filter is detected, and the coordinate value of the edge is input to the information processing apparatus 102 described later as an observation point (second observation point) in the first embodiment ( Second observation point input).

  The parameter initial value 360 is obtained by setting an array of coordinate values of sample points set from the shape model of the measurement target object 10 as a parameter initial value, and is used as an initial position and orientation value in the position and orientation parameter calculation. . As sample points, a computer graphic rendering function is used to determine a visible surface or edge at a preset position and orientation, and a value stored therein may be used.

  The information processing apparatus 101 has the same configuration as the information processing unit 100 in the first embodiment described above, inputs the distance image generated by the distance image generation unit 340, and sets the distance point as an observation point (first observation point). The matching residual calculation is performed as a point). The model point (distance) 370 is a three-dimensional coordinate array (first model point) of model points observed on the surface of the model shape corresponding to the distance image.

  The information processing apparatus 102 also has the same configuration as the information processing unit 100 in the first embodiment, and performs an association residual calculation using the edge point detected by the image edge detection unit 350 as an observation point (second observation point). . The model point (edge) 380 is an array of coordinate values (second array) representing edge points observed as brightness gradient change points such as roof edges and jump edges in the model shape in a three-dimensional coordinate system of the model coordinate system. Model point).

  The parameter updating unit 390 obtains a Jacobian matrix for estimating the position and orientation parameters by using the correlation between observation points and model points by the information processing apparatuses 101 and 102 and the result of residual calculation, and uses this as a residual matrix. Substitute into simultaneous equations to calculate least squares method. The value calculated by this is used as the updated value of the position / orientation parameter. That is, the position and orientation parameters are estimated so that the residual between the observation point and the model point becomes small.

  Here, since both the edge point and the distance point are used as observation points, the first parameter according to the information processing device 101 and the second parameter according to the information processing device 102 are estimated, The position and orientation parameters can be estimated from the first and second parameters. When updating the parameters, it is sufficient to use one of the estimated parameters by each having a better evaluation, that is, the one having a smaller residual. In addition, since the first observation point and the second observation point are observing the same measurement target object, the error between each observation point and the model is estimated with maximum likelihood to estimate the position and orientation parameters. Thus, better parameter estimation is possible. (Tateno et al., “Highly accurate and stable model fitting method that best integrates distance and grayscale images for bin picking”, IEICE Transactions D, Vol. J94-D, No. 8, pp. 1410-1422, 2011). The Jacobian matrix may be calculated using a formula that is well known by research such as Visual Servo. In addition, calculation by the Gauss-Newton method may be used for parameter calculation. That is, the parameter update unit 390 only needs to be able to update the position and orientation parameters in accordance with the association between the observation points and the model points and the residual calculation result, and does not specify the update method.

  The parameter update unit 390 determines the position / orientation estimation parameter value estimation result based on a convergence determination condition such as when the update rate satisfies a predetermined condition or when a predetermined number of repetitions are performed, and the estimation result is Output from the estimated parameter output unit 400.

  In the second embodiment, the information processing apparatuses 101 and 102 have shown an example in which the residual calculation based on each of the distance point and the edge point is performed. However, these are integrated and both residual calculations are performed in one apparatus. You may comprise as follows.

  As described above, according to the second embodiment, by estimating the position and orientation parameters used in the information processing apparatus that performs the model fitting described in the first embodiment, the processing load on the information processing apparatus is reduced. It can be reduced and the processing speed can be improved.

<Third embodiment>
The third embodiment according to the present invention will be described below. Since the position and orientation measurement realized by the first and second embodiments described above enables high-speed processing, it can be used when the tact time in a work system combining a robot and a vision becomes a problem. . As a specific configuration of the third embodiment, a robot and a robot controller for controlling the position and orientation of the robot may be added to the configuration of FIG. 5 shown in the second embodiment. As a result, the position and orientation parameters of the measurement target object 10 output from the estimation parameter output unit 400 are input to the robot controller and converted to the robot coordinate system, so that the robot's hand is directed toward the measurement target object 10. Is possible.

  As described above, according to the third embodiment, the process of associating observation points with model points is performed at high speed in the position and orientation measurement processing according to the first and second embodiments, so the processing time from shooting to estimation Is shortened. Therefore, it is possible to shorten the tact time of the entire work in the robot system.

  As can be seen from the third embodiment, the present invention can speed up a process that needs to associate observation points with model points using a large amount of observation values in order to improve accuracy. . Therefore, for example, in the automatic supply of parts and the assembly work in the assembly of industrial products, the position and orientation measurement of parts can be processed at high speed.

<Other embodiments>
The present invention provides a system or apparatus with a storage medium in which a program code of software that realizes the functions of the above-described embodiments (for example, processing shown by a flowchart in which processing of each unit described above is associated with each step) is recorded. This can also be realized. In this case, the function of the above-described embodiment is realized by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium so that the computer can read it.

Claims (13)

Information processing apparatus for associating a model point indicating a geometric feature in a model indicating a shape of a target object with an observation point indicating a geometric feature of the target object included in an image obtained by measuring the target object Because
Attaching an index value to the coordinate value of the observation point, a first holding means for holding a relationship between the index value and the coordinate value,
Second holding means for converting the coordinate value and holding an index map in which the index value is stored in a pixel corresponding to the converted coordinate value;
A model point coordinate conversion means for converting the coordinate values of the model point coordinate values definitive the coordinate system of the index map,
Obtaining means for obtaining an index value stored in a pixel of the index map corresponding to the coordinate value transformed by the model point coordinate transformation means;
An information processing apparatus, wherein an observation point corresponding to the model point is specified based on an index value acquired by the acquisition unit. Further, the coordinate values of the observation point, and observation point coordinate transforming means for converting the coordinate values definitive the coordinate system of the index map,
Write means for writing the index value assigned to the observation point corresponding to the coordinate value for the pixel corresponding to the coordinate value converted by the observation point coordinate conversion means ;
In the index map, anda expansion means for expanding the index value for the pixels around the pixel corresponding to the index value,
2. The information processing apparatus according to claim 1, wherein the second holding unit holds the index map in which an index value is extended by the extension unit.   3. The information processing apparatus according to claim 2, wherein the expansion unit propagates the index value to adjacent pixels whose values are not set. Furthermore, any one of claims 1 to 3, characterized in that it has a coordinate parameter input means for inputting coordinate parameters for transforming the coordinate values of the model point coordinate values definitive the coordinate system of the observation point The information processing apparatus described in 1. Further, by using the coordinate values of the observation point is obtained based on the index value acquired by the acquisition unit, and a coordinate value converted into the coordinate system of the observation point in the model point coordinate transforming means, said 5. The information processing apparatus according to claim 4, further comprising difference calculation means for calculating a difference between the observation point and the model point.   6. The information processing apparatus according to claim 5, further comprising an estimation unit configured to estimate a position / orientation parameter indicating a position / orientation of the target object based on the difference. A first observation point input means for inputting, as a first observation point, a distance point indicating a geometric feature of the target object in a distance image including the image of the target object;
A second observation point input means for inputting, as a second observation point, an edge point indicating a geometric feature of the target object in a two-dimensional image including the image of the target object;
7. The information processing apparatus according to claim 6, wherein the estimation unit estimates the position / orientation parameter based on the first and second observation points.   8. The information processing apparatus according to claim 6, further comprising position and orientation control means for controlling the position and orientation of the target object based on the position and orientation parameters estimated by the estimation means. Furthermore, it has an imaging means for photographing the target object,
9. The information processing apparatus according to claim 1, wherein a coordinate value of the observation point is acquired from an image of the target object imaged by the imaging unit.   10. The information processing apparatus according to claim 1, wherein the observation point is a distance point indicating a geometric feature of the target object in a distance image including the image of the target object.   10. The information processing apparatus according to claim 1, wherein the observation point is an edge point that indicates a geometric feature of the target object in a two-dimensional image including the image of the target object. In an information processing apparatus having first and second holding means, model point coordinate conversion means, and acquisition means, a model point indicating a geometric feature in a model indicating the shape of a target object, and measurement of the target object An information processing method for associating with an observation point indicating a geometric feature of the target object included in a captured image,
It said first holding means, denoted by the index value to the coordinate value of the observation point, and holds the relationship between the index value and the coordinate value,
The second holding means converts the coordinate value, and holds an index map in which an index value attached to the converted coordinate value is written;
The model point coordinate converting means converts the coordinate values of the model point coordinate values definitive the coordinate system of the index map,
The acquisition unit acquires an index value stored in a pixel of the index map corresponding to the converted coordinate value;
An information processing method, wherein an observation point corresponding to the model point is specified based on the acquired index value. A computer apparatus for associating a model point showing a geometric feature in a model showing a shape of a target object with an observation point showing a geometric feature of the target object included in an image obtained by measuring the target object Because
First holding means for attaching an index value to the coordinate value of the observation point, and holding the relationship between the index value and the coordinate value;
Second holding means for converting the coordinate value and holding an index map in which the index value is stored in a pixel corresponding to the converted coordinate value;
The computer device comprising:
Model point coordinate conversion means for converting the coordinate value of the model point into a coordinate value in a coordinate system of the index map;
A computer program for functioning as an acquisition unit that acquires an index value stored in a pixel of the index map corresponding to a coordinate value converted by the model point coordinate conversion unit,
A computer program characterized in that an observation point corresponding to the model point is specified based on an index value acquired by the acquisition means.

Images