JP2021043075A - Information processing device, control method therefor, and program - Google Patents

Abstract

To enable the position-attitude of a measurement object to be estimated with high accuracy while reducing as much as possible the number of measurement and movement times when measuring from a plurality of viewpoints.SOLUTION: An information processing device of the present invention comprises: a model information holding unit for holding a model feature group that represents the shape of a measurement object; a measurement feature acquisition unit for acquiring a measurement feature measured from a first measurement viewpoint; a position-attitude estimation unit for estimating the position-attitude of the measurement object from the model feature group and the measurement feature group of the measurement object acquired by the measurement feature acquisition unit; a measurement feature evaluation unit for evaluating the measurement feature group of the measurement object acquired by the measurement feature acquisition unit, on the basis of the position-attitude value estimated by the position-attitude estimation unit, the model feature group, and the measurement feature group; and a measurement viewpoint determination unit for determining a second measurement viewpoint different from the first measurement viewpoint, on the basis of the result of evaluation made by the measurement feature evaluation unit.SELECTED DRAWING: Figure 4

Description

The present invention relates to a technique for estimating the position and orientation of an object.

In recent years, with the development of robot technology, robots are taking over the complicated tasks that humans have traditionally performed, such as assembling industrial products. In such a robot, parts are picked and assembled by an end effector such as a hand. In order to assemble a part by a robot, it is naturally necessary to measure the relative position and posture between the gripping source part and the assembly destination part.

As in Patent Document 1, a method of combining data features measured from adjacent viewpoints between frames obtained by imaging is disclosed. By using this method to acquire data features from a plurality of viewpoints while moving the robot, it is possible to specify the position and orientation of the parts that could not be specified only by the measurement features obtained from one viewpoint.

Patent No. 5285487

However, in order to measure the target object by changing the viewpoint many times using a robot, it is necessary to repeat the movement and measurement of the measuring device or the target object many times. Therefore, a lot of time and labor are required for measurement.

The present invention has been made in view of the above problems, and provides a technique for measuring the position and posture of an object by moving the viewpoint and reducing the number of measurements at the viewpoint position.

In order to solve this problem, for example, the information processing apparatus of the present invention has the following configuration. That is,
A model information holding means for holding a model feature group representing the shape of a measurement object, a measuring feature acquiring means for acquiring measurement features measured from the first measurement viewpoint, and a measurement feature acquiring means.
A position / orientation estimating means for estimating the position / orientation of the measurement object from the model feature group and the measurement feature group of the measurement object acquired by the measurement feature acquisition means.
A measurement feature evaluation means that evaluates a measurement feature group of a measurement object acquired by the measurement feature acquisition means based on the position / attitude value estimated by the position / orientation estimation means, the model feature group, and the measurement feature group.
A measurement viewpoint determining means for determining a second measurement viewpoint different from the first measurement viewpoint is provided based on the result of evaluation by the measurement feature evaluation means.

According to the present invention, it is possible to estimate the position and orientation of a measurement object with high accuracy while reducing the number of measurements and movements when measuring from a plurality of viewpoints as much as possible.

The figure which shows the structure of the system in 1st Embodiment. The figure which shows the structure of the measuring apparatus in 1st Embodiment. The figure for demonstrating the projection pattern and the imaging process in 1st Embodiment. The figure which shows the structure of the information processing apparatus in 1st Embodiment. The figure which illustrates the 3D shape model of the measurement object in embodiment. The flowchart which shows the whole processing in 1st Embodiment. The figure for demonstrating the determination of the measurement viewpoint in 1st Embodiment. The figure for demonstrating the moving viewpoint determination in 2nd Embodiment. The flowchart which shows the whole processing in 2nd Embodiment. The figure which shows the structure of the information processing apparatus in 3rd Embodiment. The flowchart which shows the whole processing in 3rd Embodiment. The figure for demonstrating the integrated processing of the measurement feature in 3rd Embodiment. The flowchart which shows the calculation process of the evaluation value of the measurement feature in 3rd Embodiment. The flowchart which shows the moving viewpoint determination processing in 3rd Embodiment. The figure for demonstrating the determination of the moving viewpoint in 3rd Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are designated by the same reference numbers, and duplicate explanations are omitted.

[First Embodiment]
In the first embodiment, a method of determining a viewpoint for acquiring more measurement feature groups will be described. By doing so, if a large number of measurement feature groups are acquired, the position and orientation can be estimated stably, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible.

FIG. 1 shows a configuration example of the system according to the present embodiment. The information processing device 1 is communicably connected to the measuring device 110 and the mobile device 120.

As shown in FIG. 2, the measuring device 110 includes a photographing device 111 and a projection device 112. The broken line in the figure indicates the imaging range and the projection range.

The projection device 112 projects the illumination pattern onto the measurement object 130, and the photographing device 111 photographs the measurement object 130 on which the illumination pattern is projected. From the image obtained by this imaging, a three-dimensional point cloud on the surface of the measurement object 130 can be acquired. The format of the captured image is not particularly limited as long as the target distance can be acquired, such as gray scale and color. The focal length, principal point position, lens distortion parameter, and other internal parameters of the photographing device 111 and the projecting device 112, the relative position and orientation between the cameras, and the relative position and orientation between the cameras and the projector are, for example, [RY Tsai, "A versatile". camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, pre-calibrated by the method shown in "IEEE Journal of Robotics and Automation, vol.RA-3, no.4, 1987." I'll leave it to you. The relative position and orientation between the imaging device 110 and the projection device 120 referred to here is a parameter with 6 degrees of freedom represented by the rotational components (α, β, γ) and translational components (x, y, z) indicated by Euler angles. Is. The pattern projected by the projection device 120 is the multi-line pattern shown in FIG. 3A in the present embodiment, and the image reflected from the measurement object 130 (FIG. 3B) is photographed by the photographing device 110. To do. To acquire the 3D point cloud, for example, the line pattern is projected by decoding the pattern from the image taken by the image pickup apparatus 111 by the method described in [Iguchi, Sato, 3D image measurement, Shokodo, 1990]. The position can be specified and obtained from the relative position / orientation between the image pickup device 111 and the projection device 112 calibrated earlier by a method by triangulation.

The moving device 120 is a device that can move in six axes, such as a robot arm, and is equipped with a measuring device 110 as shown in FIG. That is, the measuring device 110 measures the measurement object 130 from the viewpoint on the moving device 120 side. The moving device 120 receives the viewpoint to be measured from the information processing device 1, and controls and moves the robot arm so that the measuring device 110 can measure from the viewpoint to be measured next. The transformation matrix from the coordinate system of the measuring device 110 to the coordinate system of the moving device 120 is, for example, [F. Dornaika and R. Horaud. "Simultaneous robot-world and hand-eye calibration. Robotics and Automation" IEEE Transactions on, 14 (4): 617-622, 1998.] may be calibrated in advance and calculated.

Next, the configuration of the information processing device 1 will be described with reference to FIG. The information processing device 1 is composed of a model information holding unit 100, a measurement feature input unit 101, a position / orientation estimation unit 102, a measurement feature evaluation unit 103, a measurement viewpoint determination unit 104, and a control unit 105. The control unit 105 controls the entire device, and includes a CPU and a memory for storing a program executed by the CPU.

The model information holding unit 100 holds model information representing the shape of the measurement object 130. In the present embodiment, the model feature of the shape model is local three-dimensional plane information (hereinafter, local) on the surface of the object composed of the three-dimensional position and the three-dimensional normal direction as shown in FIG. 5 (b). It shall be composed of (referred to as surface features). The shape information to be input as a model may be represented by a simple set of three-dimensional points, shape information of a polygon composed of three three-dimensional points, or the like.

The measurement feature acquisition unit 101 acquires information on the measurement feature group of the measurement object 130 from the measurement information acquired by the measurement device 110. The measurement feature group in this embodiment is a three-dimensional point cloud.

The position / orientation estimation unit 102 estimates the position / orientation of the measurement object 130 using the shape model of the measurement object 130 held by the model information holding unit 100 and the measurement feature group input by the measurement feature acquisition unit 101. To do. In this embodiment, local surface features are used as model features of the shape model, and 3D point clouds are used as measurement features [S.Rusinkiewicz, M.Levoy, "Efficient Variants of the ICP Algorithm", In 3-D Digital. Imaging and Modeling, 2001, Proceedings.Third International Conference on (pp.145-152). IEEE] estimates the position and orientation of the object to be measured 130 by ICP (Iterative Closest Point). In order to estimate the position and orientation, it is necessary to search for geometrically similar three-dimensional points for each point of the local surface feature, and these are called corresponding points. Alignment is performed using the corresponding points of the three-dimensional points and the model features. In order to find the corresponding point between the 3D point and the model feature, a pair with a distance difference below the threshold is searched from the pair closest to the local surface feature and the 3D point in the 3D coordinate system.

The measurement feature evaluation unit 103 includes a shape model of the measurement object 130 held by the model information holding unit 100, a measurement feature acquired by the measurement feature acquisition unit 101, and a measurement object 130 estimated by the position / orientation estimation unit 102. The evaluation value of the measurement feature group is calculated based on the position and orientation of. The method of calculating the evaluation value will be described later.

The measurement viewpoint determination unit 104 determines the viewpoint for measuring the measurement object 130 next, based on the evaluation result of the measurement feature evaluation unit 103. The moving device 120 moves the measuring device 110 to the determined viewpoint. The viewpoint determination method will be described later.

The measurement and viewpoint determination processing of the measuring device 110 with respect to the measurement object 130, which is executed by the information processing device 1, will be described with reference to the flowchart of FIG.

In S1000, the control unit 105 performs an initialization process. For initialization, the imaging parameters such as the exposure time and gain of the measuring device 110 are read, the calibration parameters are read, the conversion matrix from the coordinate system of the measuring device 110 to the coordinate system of the moving device 120 is read, and the model information holding unit 100 It includes reading the model shape information of the measurement object 130 held in. The control unit 105 returns to the initial position of the moving device 120, and further, as the moving device 120 moves to the initial position, the position / posture value of the measuring device 110 is also set to the initial state.

In S1100, the measurement feature acquisition unit 101 acquires a three-dimensional point cloud of the measurement object 130 from the image obtained by the measurement device 110 in the current position and orientation of the moving device 120.

In S1200, the position / orientation estimation unit 102 aligns the local surface feature held by the model information holding unit 100 with the three-dimensional point cloud acquired by the measurement feature acquisition unit 101 by using the ICP method.

In S1300, the measurement feature evaluation unit 103 includes a shape model of the measurement object 130 held by the model information holding unit 100, a three-dimensional point cloud acquired by the measurement feature acquisition unit 101, and a position / orientation estimation unit 102. Based on the estimated position and orientation of the measurement object 130, the pair of the local surface feature and the nearest vicinity of the three-dimensional point is determined, and the number of the three-dimensional point cloud in which the distance difference between the pairs is equal to or less than the threshold value is used as the evaluation value V. calculate.

In S1400, the control unit 105 processes the process to S1500 if there is a viewpoint in which the number of local surface features that can be observed from a viewpoint different from the current viewpoint is larger than the evaluation value V calculated in S1300, otherwise. End the process.

In S1500, the measurement viewpoint determination unit 104 determines the position of the viewpoint having the largest number of local feature groups on the model surface when observed from another viewpoint in S1400.

In S1600, the control unit 105 sends a control value to the moving device 120 so as to have the viewpoint determined in S1500, and moves the viewpoint. Then, when this movement is completed, the control unit 105 returns the process to S1100.

The illumination pattern projected by the projection device 112 in the present embodiment is a solid multi-line pattern, but is not limited thereto. For example, a pattern in which the brightness or color of the pattern is changed on the line, a broken line pattern, or a pattern in which the width of the line is changed may be used. The line pattern may be evenly spaced or unequally spaced. In addition to the line pattern, a random dot pattern or a grid pattern may be used. In any case, any pattern can be used as long as the illumination pattern projected on the object to be measured on the captured image and the background portion of the pattern can be discriminated.

In the present embodiment, the method of searching for the corresponding point between the model feature and the measurement feature in the position / orientation estimation unit 102 is to select the nearest pair in the three-dimensional coordinate system, but the method is not limited to this. For example, the model features and the three-dimensional points may be projected onto an image showing the image of the multi-line pattern input by the measuring device 110, and the nearest pair may be searched and obtained in the two-dimensional coordinate system.

When selecting a viewpoint, the measurement viewpoint determination unit 104 may limit the selection of the viewpoint from a hemisphere that does not change the distance D between the measurement object 130 and the measurement device 110 as shown in FIG. ..

In the present embodiment, the control value is sent to the moving device 120 so that the viewpoint is determined in S1600, the viewpoint is moved, and then the measurement is performed again in S1100. However, the measurement is not limited to this, and the measurement is performed during the movement. You may go.

The above is the method of determining from the viewpoint of acquiring more measurement feature groups in the present embodiment. By doing so, if a large number of measurement feature groups are acquired, the position and orientation can be estimated stably, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible.

[Modification 1]
In the above embodiment, a method of using a three-dimensional point cloud as the measurement feature group and determining the viewpoint for acquiring more measurement feature groups has been described. However, the measurement feature group used for position / orientation estimation is not limited to the three-dimensional point cloud, and the image feature group on the grayscale image may be used. In this modification 1, a method of using an image feature group as a measurement feature group and determining a viewpoint at which a larger number of image feature groups can be acquired will be described.

Since the configuration of the information processing device in the first modification is the same as that of the first embodiment, the description thereof will be omitted. The measuring device 110 in the first modification captures a shade image of the measurement object 130. At this time, the projection device 112 does not project the illumination pattern, and the photographing device 111 images the measurement object 130.

In the model information holding unit 100 in the first modification, the model feature of the shape model is a local three-dimensional on the object contour composed of the three-dimensional position and the three-dimensional line segment direction as shown in FIG. 5 (a). It is composed of line segment information (hereinafter referred to as local line feature). The shape information input as a model may be represented by a set of three-dimensional lines representing ridgelines.

The measurement feature acquisition unit 101 in the first modification acquires an image feature group from a shade image acquired from the measurement device 110. As the image features acquired here, the edge features described in [Canny. J, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8: 679-714, 1986] shall be used. It should be noted that the method is not limited to this, as any method may be used to detect the contour of the measurement object 130 in the image.

The position / orientation estimation unit 102 in the first modification is described in [T. Drummond and R. Cipolla, "Real-time visual tracking of complex structures", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, no.7, pp. 932-946, 2002.], the position of the measurement object 130 is selected by selecting the pair closest to the local line feature and the image feature, and using only the pair whose distance difference is less than or equal to the threshold value. Estimate the posture.

The measurement feature evaluation unit 103 in the first modification is estimated by the shape model of the measurement object 130 held by the model information holding unit 100, the image feature group acquired by the measurement feature acquisition unit 101, and the position / orientation estimation unit 102. Based on the position and orientation of the measurement object 130, the number of image feature groups whose distance difference is less than or equal to the threshold value when the nearest pair of local line feature and image feature is selected from the current viewpoint is used as the evaluation value V. calculate. This process is performed in S1300.

The viewpoint determined by the measurement viewpoint determination unit 104 in the first modification is determined to be a viewpoint in which the number of local line features that can be observed from a viewpoint different from the current viewpoint is larger than the evaluation value V calculated by the measurement feature evaluation unit 103. ..

In the first modification, the image is taken by the photographing device 111 without projecting the illumination pattern from the projection device 112, but the present invention is not limited to this. For example, another photographing device is installed in the measuring device 110, and an optical filter that does not transmit the illumination pattern is installed in front of the sensor surface to acquire a grayscale image. Acquire an image of the measurement object 130 on which the illumination pattern is not projected, such as by adding a sensor with an optical filter that does not transmit the pattern to the measuring device 110 or by splitting it with a prism or the like to acquire a shading image. Any method that can be used will do. Further, the image on which the illumination pattern is projected and the shade image may be acquired at the same time using these.

The position / orientation estimation unit 102 in the first modification aligns the local line feature held by the model information holding unit 100 with the image feature group, and obtains the position / orientation of the measurement object 130, but this is limited to this. It's not a thing. For example, [Tateno, Kotake, Uchiyama, "High-precision and highly stable model fitting method that most likely integrates distance and shading images for bin picking of actual parts", Image Recognition and Understanding Symposium-MIRU2010, OS5-1 (2010). 7], the image feature group and the three-dimensional point cloud of the grayscale image are acquired by the measuring device 110, and the local line feature and the local surface feature of the three-dimensional model representing the shape of the measurement object 130 are used. The position and orientation of the measurement object 130 may be obtained. In any case, any method can be used as long as the position and orientation of the measurement object 130 can be estimated.

In this case, the evaluation value V is the number of image feature groups whose distance difference from the local line feature when the position and orientation are estimated from the current viewpoint is less than the threshold value, and the three-dimensional point cloud whose distance difference from the local surface feature is less than the threshold value. Let it be the sum of the numbers. In the viewpoint determination method, a viewpoint larger than the evaluation value V is determined by using the sum of the number of local line features and local surface features that can be observed from a viewpoint different from the present one. Further, instead of using a simple sum for determining the evaluation value V and the viewpoint, weighting may be applied to determine the number of image feature groups or the number of three-dimensional point clouds with priority.

The above is the method of determining the viewpoint in which more model features can be acquired by using the image feature group as the measurement feature group in the modification example 1. By doing so, it becomes possible to measure from a viewpoint that can acquire more image feature groups, so that it is possible to estimate the position and orientation with high accuracy while reducing the number of measurements and movements when measuring from multiple viewpoints as much as possible. Become.

[Second Embodiment]
In the first embodiment and the first modification thereof, a method of changing from the currently measured viewpoint to a viewpoint in which more measurement feature groups can be acquired than the currently measured viewpoint has been described. However, depending on the angle of view of the imaging device and the projection device and the relative positional relationship between the measurement target, the distance between the illumination patterns reflected on the measurement target may be projected coarsely, and the distance may not be calculated even for minute shaped parts. .. For example, as shown in FIG. 8A, a line pattern is projected as an illumination pattern in the region C which is a fine shape portion of the measurement object 130, but the illumination pattern is too short to be identified and a three-dimensional point cannot be acquired. .. Therefore, since there is no three-dimensional point in the position / orientation estimation unit 102, the model feature of this portion is not used for the position / orientation. Therefore, the shape of the unmeasured region C on the image is calculated from the model features, and the viewpoint is changed according to the shape as shown in FIG. 8B to change the direction in which the illumination pattern is observed for a sufficiently long time. Then, the method of measuring again will be described. By doing so, it becomes possible to know the position where more measurement features can be obtained, and it becomes possible to estimate the position and orientation with high accuracy while reducing the number of measurements and movements when measuring from multiple viewpoints as much as possible. ..

The configuration of the information processing apparatus in the second embodiment is the same as that in the first embodiment and the first modification thereof. Therefore, the difference between the first embodiment and the modified example 1 will be described below.

The configuration of the measuring device 110 in the second embodiment is the same as that in the first embodiment. The measurement feature evaluation unit 103 includes a shape model of the measurement object 130 held by the model information holding unit 100, a three-dimensional point cloud acquired by the measurement feature acquisition unit 101, and a measurement target estimated by the position / orientation estimation unit 102. Based on the position and orientation of the object 130, the unmeasured region C is evaluated (details will be described later).

In the measurement viewpoint determination unit 104 in the present embodiment, the measurement feature evaluation unit 103 determines the viewpoint based on the evaluation result (details will be described later).

The processing of measurement and viewpoint determination in the present embodiment will be described with reference to the flowchart of FIG.

In S2000, the control unit 105 performs the initialization process. In this initialization process, the imaging parameters such as the exposure time and gain of the measuring device 110 are read, the calibration parameters are read, the conversion matrix from the coordinate system of the measuring device 110 to the coordinate system of the moving device 120 is read, and the model information. The reading of the model shape information of the measurement object 130 held in the holding unit 100 is included. Further, the control unit 105 returns the moving device 120 to the initial position, and sets the viewpoint to the position / posture value of the measuring device in the coordinate system of the moving device when the measuring device 110 measures at the initial position.

In S2100, the measurement feature acquisition unit 101 acquires a three-dimensional point cloud of the measurement object 130 from the measurement device 110 in the current position and orientation of the moving device 120.

In S2200, the position / orientation estimation unit 102 aligns the local surface feature held by the model information holding unit 100 with the three-dimensional point cloud acquired by the measurement feature acquisition unit 101. As the alignment method, the same method as in the first embodiment shall be used.

In S2300, the measurement feature evaluation unit 103 includes the shape model of the measurement object 130 held by the model information holding unit 100, the three-dimensional point cloud input by the measurement feature input unit 101, and the position / orientation estimation unit 102. Based on the calculated position and orientation of the measurement object 130, the directionality of the distribution of the unmeasured region C is detected. First, as the unmeasured region C, when the position and orientation are estimated, the local surface feature group in which the measured three-dimensional point cloud does not exist within a certain threshold range in the three-dimensional space is detected. Next, from the two-dimensional distribution when the local surface feature group of region C is projected onto the captured image, the absolute difference between the eigenvalue σ1 of the first principal component and the eigenvalue σ2 of the second principal component calculated by principal component analysis is absolute. The value is calculated as the evaluation value of the unmeasured area.

In S2400, the control unit 105 ends the process if the unmeasured area evaluation value calculated in S2300 is smaller than the threshold value. If not, the process proceeds to step S2500.

In S2500, the measurement viewpoint determination unit 104 determines the viewpoint in which the eigenvector of the first principal component calculated by the principal component analysis calculated in S2300 and the direction of the line pattern in the vicinity when projected on the captured image match. .. At this time, the position of the measuring device 110 is fixed, only rotation is performed around the line-of-sight vector from the camera origin of the measuring device 110 to the center of the image, and the direction of the line pattern and the eigenvector of the first principal component of the distribution are parallel to each other. Choose a perspective that will be. For example, the measurement viewpoint determination unit 104 rotates the measuring device by 90 ° from the state of FIG. 8A so that the direction of the distribution of the region C and the direction of the line pattern are parallel to each other as shown in FIG. 8B. Determine the viewpoint so that it will be done. Then, the measurement viewpoint determination unit 104 sends a control value to the moving device 120 so that the viewpoint is determined in S2500, and moves the viewpoint. When this movement is completed, the control unit 105 returns the process to S1100.

The measurement feature evaluation unit 103 automatically detects and evaluates a local surface feature group in which the three-dimensional point cloud measured within a certain threshold range in the three-dimensional space does not exist when the position and orientation are estimated as the unmeasured region C. It was, but it is not limited to that. For example, instead of automatically detecting the area C, the unmeasured area evaluation value may be calculated by selecting the area that the user wants to measure. This is because it is only necessary to be able to detect an area where the distance has not been measured.

As described above, in the present embodiment, since the position where many measurement features can be obtained can be known by changing the viewpoint of the measuring device 110 according to the shape and measuring, the number of measurements and movements when measuring from a plurality of viewpoints can be determined. It will be possible to estimate the position and orientation with high accuracy while reducing it as much as possible.

[Modification 2]
The illumination pattern projected by the measuring device 110 used in the second embodiment is a multi-line pattern, and the viewpoint is changed according to the directionality of the illumination pattern, but the present invention is not limited to this. Therefore, in the second modification, a method of projecting a random dot pattern as an illumination pattern and measuring the shape by matching the patterns will be described, and a method of measuring by changing the viewpoint according to the shape will be described.

The configuration of the information processing device in the second modification is the same as that of the first and second embodiments and the first modification. Hereinafter, differences from the first and second embodiments and the first modification will be described.

The measuring device 110 in the second modification projects a random dot pattern to acquire a three-dimensional point cloud of the measurement target 130. As the measurement method, the method described in [M. Sjodahl and P. Synnergren, “Measurement of shape by using projected random patterns and temporal digital speckle photography”, Applied Optics, vol.38, No.23, 1999] is used.

The measurement feature evaluation unit 103 in the second modification is a group of three-dimensional points measured within a certain threshold range in the three-dimensional space among the local surface features of the measurement object 130 when viewed from the viewpoint currently being observed. The dispersion value n1 of the non-existent local surface feature group distribution is calculated. Next, the measurement feature evaluation unit 103 calculates the ratio M = n1 / n2 with the window size n2 for matching the random dot pattern, and sets this as the unmeasured area evaluation value. Further, instead of automatically detecting the area C, the evaluation value of the unmeasured area may be calculated by selecting the area to be measured by the user, and another method is adopted as long as the unmeasured area can be detected. May be good.

The measurement viewpoint determination unit 104 in the second modification determines a viewpoint that brings the measurement viewpoint closer to the measurement object 130. Specifically, first, the average value Z of the depth values when viewed from the current viewpoint of the local surface feature of the unmeasured region C detected by the measurement feature evaluation unit 103 is calculated. Next, using the unmeasured area evaluation value M calculated by the measurement feature evaluation unit 103, a viewpoint is selected so that the depth value Z'after the viewpoint change becomes Z / M. At this time, the viewpoint changing direction with respect to the measuring device 110 is limited only to the line-of-sight direction of the measuring device 110.

As described above, according to the present modification 2, it becomes possible to know the position where more measurement features can be obtained, which is a method in which the measuring device 110 can change the viewpoint according to the shape and measure. Therefore, when measuring from a plurality of viewpoints. It will be possible to estimate the position and orientation with high accuracy while reducing the number of measurements and movements as much as possible.

[Third Embodiment]
In the first and second embodiments and the first and second modifications, a method of changing from the currently measured viewpoint to a viewpoint in which more measurement feature groups can be obtained than the currently measured viewpoint has been described. However, if the distribution of the measurement feature group is biased, the robustness in the position / orientation estimation may decrease and it may not be possible to measure with high accuracy. Therefore, in the third embodiment, the spatial distribution of the three-dimensional point cloud is measured, and if the distribution is biased, the viewpoint of the measuring device is changed in a direction that makes the distribution uniform, and the three-dimensional point cloud is integrated. The method of estimating the position and orientation will be described. By doing so, it becomes possible to estimate the position and orientation with high accuracy while reducing the number of measurements and movements when measuring from a plurality of viewpoints as much as possible.

The measuring device 110 in the third embodiment is the same as the first embodiment and acquires a three-dimensional point cloud.

FIG. 10 is a block configuration diagram of the information processing apparatus 2 according to the third embodiment. It should be understood that the information processing device 2 replaces the information processing device 1 in FIG. The information processing device 2 in the third embodiment includes a model information holding unit 100, a measurement feature acquisition unit 101, a position / orientation estimation unit 102, a measurement feature evaluation unit 103, a measurement viewpoint determination unit 104, a measurement feature integration unit 205, and measurement. It is composed of a feature holding unit 206 and a control unit 105.

The measurement feature acquisition unit 101 in the third embodiment inputs the measurement feature acquired by the measurement device 110. In the third embodiment, the measurement feature group is a three-dimensional point cloud of the measurement object 130.

The position / orientation estimation unit 102 in the third embodiment includes a shape model of the measurement object 130 held by the model information holding unit 100, a measurement feature group acquired by the measurement feature input unit 101, and a measurement feature holding unit. The position and orientation of the measurement object 130 are estimated using the measurement feature group held by 205. The method of measuring the position and posture is the same as that of the first embodiment.

The measurement feature evaluation unit 103 includes a shape model of the measurement object 130 held by the model information holding unit 100, a measurement feature group acquired by the measurement feature input unit 101, and a measurement feature held by the measurement feature holding unit 205. The evaluation value of the measurement feature is calculated based on the position and orientation of the group and the measurement object 130 calculated by the position and orientation estimation unit 102. The method of calculating the evaluation value by the measurement feature evaluation unit 103 will be described later.

The measurement viewpoint determination unit 104 determines the viewpoint for measuring the measurement object 130 next based on the evaluation result of the measurement feature evaluation unit 103, and controls the moving device 120 so as to be the determined viewpoint. The moving device 120 moves to the determined viewpoint by this control. The viewpoint determination method will also be described later.

The measurement feature integration unit 205 adds the measured coordinate values of the measurement feature group to the measurement feature holding unit 206. When adding, the viewpoint is converted to the coordinate system of the viewpoint determined in advance by the measurement viewpoint determination unit 104. Therefore, the relative position and orientation of the current viewpoint with respect to the coordinate system of the determined viewpoint is obtained. The method is particularly limited as long as it is a method that can acquire the relative position and orientation, such as acquiring the output value of the magnetic sensor, acquiring the joint position information of the robot, and using the position and orientation value estimated by the position and orientation estimation unit 102. Absent.

The measurement feature holding unit 206 holds the measurement feature group added from the measurement feature integration unit 205.

Next, the measurement and the viewpoint determination / integration process in the third embodiment will be described with reference to the flowchart of FIG.

In S3000, the control unit 105 initializes the information processing device 2. In this initialization process, the imaging parameters such as the exposure time and gain of the measuring device 110 are read, the calibration parameters are read, the conversion matrix from the coordinate system of the measuring device 110 to the coordinate system of the moving device 120 is read, and the model information. The reading of the model shape information of the measurement object 130 held in the holding unit 100 is included. Further, the control unit 105 returns the moving device 120 to the initial position, and sets the viewpoint to the position / posture value of the measuring device in the coordinate system of the moving device when the measuring device 110 measures at the initial position.

In S3100, the measurement feature acquisition unit 101 acquires a three-dimensional point cloud of the measurement object 130 from the image captured by the measurement device 110 in the current position and orientation of the moving device 120.

In S3200, the position / orientation estimation unit 102 includes the local surface features held by the model information holding unit 100, the three-dimensional point cloud acquired by the measurement feature input unit 101, and the measurement feature holding unit 206 by the ICP method. Align the held 3D point cloud.

In S3300, the measurement feature evaluation unit 103 includes the shape model of the measurement object 130 held by the model information holding unit 100, the three-dimensional point cloud acquired by the measurement feature input unit 101, and the measurement feature holding unit 102. Based on the three-dimensional point cloud held and the position / orientation of the measurement object 130 calculated by the position / orientation estimation unit 102, the evaluation value V of the measurement feature group indicating the magnitude of the distribution bias is calculated (for details, see See below).

In S3400, the control unit 105 determines whether or not the evaluation value V indicating the magnitude of the bias of the distribution calculated in S3300 is equal to or less than a certain threshold value, and if it is determined to be equal to or less than the threshold value, the process ends. If not, the process proceeds to S3500.

The processing proceeds in S3500 when the evaluation value V exceeds a certain threshold value. Therefore, the measurement viewpoint determination unit 104 determines the viewpoint in order to move to a viewpoint different from the current one for measurement (details will be described later).

In S3600, the measurement feature integration unit 205 adds the three-dimensional point cloud from the current viewpoint to the measurement feature holding unit 206. The added three-dimensional point cloud and the three-dimensional point cloud held in the measurement feature holding unit 206 are coordinate-converted to the coordinate system serving as the viewpoint reference determined in S3500.

In S3700, the measurement viewpoint determination unit 104 sends a control value to the moving device 120 so that the viewpoint is determined in S3500, and moves the viewpoint. The control unit 105 returns the process to S1100 in response to the completion of this movement.

As for the three-dimensional point group of the measurement object 130 before the viewpoint movement held by the measurement feature holding unit 206, the three-dimensional points on the line pattern as shown in FIG. 12A are measured, and the measurement target after the viewpoint movement. As for the three-dimensional point cloud of the object 130, the three-dimensional points on the line pattern as shown in FIG. 12B are measured. In S3600 after the viewpoint is moved, the measurement feature integration unit 205 is the same as FIG. 12 (a) by performing the inverse conversion using the relative position / orientation of the parts between FIGS. 12 (a) and 12 (b). It is converted so that it becomes a coordinate system, and integrated as shown in FIG. 12 (c).

Next, the details of the processing of the measurement feature evaluation unit 103 in S3300 will be described with reference to the flowchart of FIG.

In S3301, the measurement feature evaluation unit 103 acquires the three-dimensional point cloud used for the alignment of the shape model in S3200.

In S3302, the measurement feature evaluation unit 103 projects the three-dimensional coordinate values (Xn, Yn, Zn) of the point cloud data obtained in S3301 onto the grayscale image, and the two-dimensional coordinate values (xn, yn, yn). ) Is calculated. In the third embodiment, the two-dimensional projection destination of the point cloud data is performed by perspective projection conversion from the internal parameters of the photographing apparatus 111 described in [Iguchi, Sato, three-dimensional image measurement, Shokodo, 1990]. The coordinate values (xn, yn) of are calculated.

In S3303, the measurement feature evaluation unit 103 calculates the bias of the point cloud distribution from the two-dimensional coordinate values of the point cloud data obtained in S3302. In this embodiment, the eigenvalues (variance values) of the first principal component σ1 and the second principal component σ2 of the two-dimensional coordinates of the point cloud data are calculated by the principal component analysis, which is a known technique, and the magnitude of the distribution bias is determined. It is calculated as the absolute value of the difference | σ2-σ1 |. Output as evaluation value V.

The above is the details of the processing of the measurement feature evaluation unit 103 in S3300.

Next, the detailed processing of the measurement viewpoint determination unit 104 in S3500 will be described with reference to the flowchart of FIG.

In S3501, the measurement viewpoint determination unit 104 determines the moving direction to the next viewpoint from the bias of the distribution of the point cloud data calculated in S3303. In the third embodiment, the moving direction is the eigenvector (u, v) of the second principal component of the point cloud data distribution when projected onto the grayscale image calculated in S3303 as shown in FIG. It is assumed that (u, v) is a unit vector.

In S3502, the measurement viewpoint determination unit 104 determines the start point in the moving direction. In the third embodiment, the starting point is the position of the center of gravity (xs, ys) of the point cloud data distribution when projected onto the grayscale image, which is the position of “x” in FIG.

In S3503, the measurement viewpoint determination unit 104 determines the end point in the moving direction. In the present embodiment, the end point is the region boundary position (xe, ye) of the measurement object 130 on the grayscale image, which is the position of “◯” in FIG. The following equation (1) is used as the calculation method.

ここでｔは実数である。ここでｔを変更して、物体領域内の座標値（ｘｅ，ｙｅ）を算出する。ここで物体領域であるかどうかは、濃淡画像を大津法やｐタイル法などを用いて二値化して計測対象物の領域を検出する。

Here, t is a real number. Here, t is changed to calculate the coordinate values (xe, ye) in the object area. Here, whether or not it is an object region is determined by binarizing the grayscale image using the Otsu method, the p-tile method, or the like to detect the region of the object to be measured.

In S3504, the measurement viewpoint determination unit 104 converts the end point calculated in S3503 into the coordinate system in the coordinate system of the moving device 120, and determines the next viewpoint position. First, the measurement viewpoint determination unit 104 calculates the coordinate position of the end point (xe, ye) calculated in S3503 in the three-dimensional space. Specifically, the line-of-sight vector from the origin to the end point of the photographing apparatus 111 is calculated, and the three-dimensional coordinates (Xe, Ye, Ze) of the intersection of the line-of-sight vector and the model used for alignment are calculated. When there are a plurality of intersections, the point closest to the origin of the photographing apparatus 111 is adopted. Next, the measurement viewpoint determination unit 104 uses the transformation matrix from the coordinate system of the measurement device 110 that has been read in advance to the coordinate system of the moving device 120, and uses the three-dimensional coordinates (Xe, Ye) as in the following equation (2). , Ze) is converted to the coordinate system of the moving device (Xe', Ye', Ze').

ここでｒ₁₁〜ｒ₃₃は、移動装置１１０に対する計測装置１２０の姿勢を表し、ｔ_x，ｔ_y，ｔ_zは移動装置１１０に対する計測装置１２０の位置を表している。

Here, r _{11 to r 33} represent the posture of the measuring device 120 with respect to the mobile device 110, and t _x , _ty , and t _z represent the position of the measuring device 120 with respect to the mobile device 110.

The above is the details of the processing of the measurement viewpoint determination unit 104 in S3500.

The evaluation value in the measurement feature evaluation unit 103 in the third embodiment evaluates the bias of the distribution when the three-dimensional point cloud used for the position / orientation estimation is projected on the grayscale image by using the principal component analysis. , Not limited to that. For example, the ratio of the three-dimensional point cloud projected on the grayscale image to the entire surface of the measurement object 130 on the grayscale image may be used. In order to specify the position and orientation of the object to be measured 130, it is only necessary to evaluate whether the three-dimensional points on the surface of the object to be measured have been measured.

Further, when the evaluation value was calculated by the measurement feature evaluation unit 103, the distribution when the three-dimensional point cloud used for the position / orientation estimation was projected on the grayscale image was observed, but the projection target is not limited to the grayscale image. For example, an image obtained by rendering the model from the viewpoint in which the model and the measurement feature are aligned in step S1300 may be used. Other methods may be used as long as the area occupied by the object on the image can be estimated.

In the third embodiment, the viewpoint determined by the measurement viewpoint determination unit 104 is determined to move in the direction in which the distribution of the three-dimensional point cloud is largely biased, but the present invention is not limited to this. If there is a portion where the three-dimensional point cloud is not projected on the entire surface of the measurement object 130 on the grayscale image, it may be determined to move to the position of the center of gravity of that portion. At this time, the viewpoint may be determined so as not to change the posture or distance of the measuring device 110 with respect to the measuring object 130. In any case, it is only necessary to be able to move to an area where the three-dimensional point cannot be measured with respect to the area occupied by the object on the image.

As described above, in the third embodiment, the spatial distribution of the three-dimensional point cloud used at the time of position / orientation estimation is measured, and if the distribution is biased, the viewpoint of the measuring device is set a plurality of times in a direction that makes it uniform. It is a method of changing and integrating a three-dimensional point cloud. By doing so, it becomes possible to estimate the position and orientation with high accuracy while reducing the number of measurements and movements when measuring from a plurality of viewpoints as much as possible.

[Modification 3]
In the third embodiment, the three-dimensional point cloud is used as a measurement feature, and the measurement is performed by moving to the viewpoint so that the spatial distribution becomes uniform. However, the measurement feature group that can be used is not limited to this, and the image feature group on the grayscale image may be used. In variant 3, the spatial distribution of the image feature group used in the position and orientation estimation is measured, and if the distribution is biased, the viewpoint of the measuring device is changed in a direction that makes the distribution uniform, and the measurement features are integrated. To do.

Since the configuration of the information processing device in the third modification is the same as that of the third embodiment, the description thereof will be omitted. The image features used in the third modification are the same as the image features used in the first modification. The measuring device 110 in the third modification captures a shade image in which the measurement object 130 is captured.

The position / orientation estimation unit 102 in the third modification is described in [T. Drummond and R. Cipolla, “Real-time visual tracking of complex structures,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.24, no.7, pp. 932-946, 2002.], the image feature and the local line feature are obtained as corresponding points, and the position and orientation of the measurement object 130 are estimated. Not limited to this, it may be used in combination with the method of acquiring a three-dimensional point cloud as in the first embodiment and performing alignment using local surface features.

The evaluation value V evaluated by the measurement feature evaluation unit 103 in the modification 3 is the absolute value of the difference between the first principal component and the second principal component of the principal component analysis for the bias of the image feature used in the position / orientation estimation unit 103. Obtained by calculation.

The viewpoint determined by the measurement viewpoint determination unit 104 in the modification 3 is moved so that the average value of the coordinate values of the local line feature group does not exist within a certain threshold range in the two-dimensional space and is centered on the image. To do. At this time, the viewpoint may be restricted so as not to change the posture or distance of the measuring device 110 with respect to the measuring object 130.

The measurement feature integration unit 205 in the third modification adds the image feature group obtained from the viewpoint determined by the measurement viewpoint determination unit 104 to the measurement feature holding unit 206. First, the line-of-sight vector (unit vector) from the origin of the photographing device 111 to the image feature is calculated, and the line-of-sight vector is multiplied by the Z coordinate value of the local line feature geometrically similar to the image feature at the time of position / orientation estimation. Virtually convert to coordinates in 3D space. Next, as in the third embodiment, the measurement feature integration unit 205 converts the relative position / orientation of the measurement object 130 before and after the viewpoint change into the coordinates before the viewpoint change by reverse conversion. Finally, the measurement feature integration unit 205 projects the image coordinates after the coordinate conversion onto the grayscale image and adds them to the measurement feature holding unit 206.

The above is the method of measuring the spatial distribution of the image feature group in the modified example 3 and integrating the image features by changing the viewpoint of the measuring device in a direction that makes the distribution uniform when the distribution is biased. By doing so, it becomes possible to estimate the position and orientation with high accuracy while reducing the number of measurements and movements when measuring from a plurality of viewpoints as much as possible.

[Fourth Embodiment]
As a suitable application example of the information processing devices 1 and 2 in the embodiment, an industrial robot arm having a movable axis including a rotating and / or translational moving axis is used as the moving device 120, the measuring device is moved, and the measuring device is moved from the viewpoint. An application example of integrating the obtained measurement features can be mentioned.

FIG. 1 is a robot that performs grasping / assembling processing of an object by demonstrating a configuration example of a robot system that performs component assembling processing using a robot that is an information processing device 1, a measuring device 110, and a moving device 120. The measuring device 110 is mounted on the robot to acquire the shape of the measurement object 130. The measuring device 110 is moved by the robot so that it can be observed from the viewpoint designated by the information processing device 1. The measurement features are acquired again by the measuring device 110 from the moved viewpoint and integrated. Further, the information processing device 1 acquires the movable range of the joint position of the robot at the time of initializing the S1000, and if it is outside the movable range of the joint position of the robot determined by the measurement viewpoint determination unit 104, it is within the movable range of the robot. Instruct the robot to move within the range that can be moved to that viewpoint. Further, when the information processing device 1 moves to the position determined by the measurement viewpoint determination unit 104 and the measurement device 110 or the robot itself physically interferes with the measurement object 130, the range in which the information processing device 1 does not physically interfere with the measurement object 130. Instruct the robot to move within the range that can be moved to that viewpoint.

As described above, the method of acquiring and integrating measurement features by changing the viewpoint by the robot during the assembly work by the robot system by the information processing apparatus of the present invention has been described. By using a robot, it is possible to efficiently move to a position where measurement features can be obtained, so that highly accurate position and orientation estimation can be performed.

The coordinate system of the three-dimensional point cloud held by the measurement feature holding unit 102 in the present embodiment is based on the position and orientation of the moving device 120 when the three-dimensional point cloud is first acquired after initialization. , Not limited to that. For example, if it is a parameter that can be expressed by 6 degrees of freedom of position / orientation, such as the position / attitude value of the measurement object 130 when first estimated by the position / attitude estimation unit 130 or the position / attitude value defined by the user. There is no limit.

[Effect of Embodiment]
According to the first embodiment, the position where a sufficient number of three-dimensional point clouds can be obtained can be known, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible.

According to the first modification, the position where a sufficient number of image feature groups can be obtained can be known, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible.

According to the second embodiment, a larger number of measurement feature groups can be obtained by rotating the measuring device according to the shape, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible. become.

According to the second modification, more measurement features can be obtained by bringing the measuring device closer to the shape, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible.

According to the third embodiment, the three-dimensional three-dimensional point cloud indicating the shape of the object to be measured can be measured without bias, so that the position and orientation can be estimated with high accuracy while reducing the number of measurements and movements as much as possible. Become.

According to the third modification, the image feature group is increased by moving the viewpoint to the part where the image feature group on the shading image of the measurement object is insufficient. Therefore, the position and orientation with high accuracy while reducing the number of measurements and movements as much as possible. You will be able to make estimates.

According to the fourth embodiment, the robot system enables the robot to change the viewpoint and acquire the measurement feature group, so that the position and orientation can be estimated with high accuracy.

<Definition>
The model feature of the measurement object held by the model information holding unit in the present embodiment is local three-dimensional plane information on the object surface, local on the object contour composed of the three-dimensional position and the three-dimensional line segment direction. There is no particular limitation as long as it is composed of three-dimensional line segment information. Further, it may be represented by a set of simple three-dimensional points, a set of three-dimensional lines representing ridges, shape information of a polygon composed of three three-dimensional points, and the like.

Further, the measurement feature group acquired by the measurement feature acquisition unit in the present embodiment may be a three-dimensional point cloud and / or an image feature group showing geometric information of the measurement object. Further, the three-dimensional point group showing the geometric information of the object to be measured may have three-dimensional coordinates based on the real space, or coordinates based on an imaging device whose position and orientation in the real space at the time of imaging are known. It may be the three-dimensional coordinates in. Further, the image feature group on the grayscale image may be a method of detecting the contour of the measurement object such as the edge feature of the measurement object detected on the grayscale image.

The position / orientation estimation unit in the present embodiment is not particularly limited as long as it is a method of obtaining the position / orientation of the measurement object from the model information held by the model information holding unit and the measurement feature group input by the measurement feature input unit. The local surface feature held by the information holding unit may be aligned with the three-dimensional point cloud input by the measurement feature input unit, or the image feature group and the local area on the grayscale image input by the measurement feature input unit may be performed. Alignment may be performed using line features, or both a 3D point cloud and an image feature group may be input, and the position and orientation of the object to be measured can be obtained using the local line features and local surface features of the model. May be good.

The measurement feature evaluation unit in the present embodiment includes the measurement object model information held by the model information holding unit, the measurement feature group input by the measurement feature input unit, and the position of the measurement object calculated by the position / orientation estimation unit. Based on the posture, it is only necessary to evaluate that there is a region on the surface of the object to be measured for which measurement features have not been acquired. For example, the number of measurement feature groups closest to the model feature in the three-dimensional space after the positioning is performed by the position / orientation estimation unit is evaluated. Alternatively, the spatial distribution of the model feature group in which the measured feature point group does not exist within a certain threshold range in the three-dimensional space is measured, and the bias or spread on the two-dimensional image from the measured viewpoint is evaluated. Alternatively, the bias or spread of the measurement feature group distribution projected on the measurement object from the measured viewpoint is evaluated.

Based on the result of evaluation by the measurement feature evaluation unit, the measurement viewpoint determination unit in the present embodiment determines the viewpoint at which the measurement feature in the region where the measurement feature cannot be acquired can be obtained on the surface of the measurement object. For example, it may be determined that the number of model feature groups that can be observed from a viewpoint different from the measured viewpoint moves with respect to the viewpoint that can be obtained more than the currently acquired measurement feature group. Further, the viewpoint may be determined so that the bias of the distribution of the unmeasurable region and the direction of the illumination pattern are parallel to the unmeasurable region so that the measuring device can measure the region. Alternatively, the viewpoint may be determined so that the image is projected larger than the minimum area required on the image for determining the illumination pattern. Further, the viewpoint may be determined in the direction in which the deviation of the measurement point distribution is small so that the three-dimensional points are measured all over the surface on which the measurement object is observed.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1 ... Information processing device, 110 ... Measuring device, 120 ... Moving device, 130 ... Measurement object, 100 ... Model information holding unit, 101 ... Measurement feature acquisition unit, 102 ... Position / orientation estimation unit, 103 ... Measurement feature evaluation unit, 104 ... Measurement viewpoint determination unit, 205 ... Measurement feature integration unit, 206 ... Measurement feature holding unit

Claims (12)

A model information holding means for holding a model feature group representing the shape of a measurement object, a measuring feature acquiring means for acquiring measurement features measured from the first measurement viewpoint, and a measurement feature acquiring means.
A position / orientation estimating means for estimating the position / orientation of the measurement object from the model feature group and the measurement feature group of the measurement object acquired by the measurement feature acquisition means.
A measurement feature evaluation means that evaluates a measurement feature group of a measurement object acquired by the measurement feature acquisition means based on the position / attitude value estimated by the position / orientation estimation means, the model feature group, and the measurement feature group.
Based on the result of evaluation by the measurement feature evaluation means, a measurement viewpoint determining means for determining a second measurement viewpoint different from the first measurement viewpoint, and a measurement viewpoint determining means.
An information processing device characterized by being equipped with. The measurement feature evaluation means calculates the number of measurement feature groups without geometrically similar model feature groups obtained by the position / orientation estimation means as an evaluation value.
The claim for determining the measurement viewpoint is characterized in that the second measurement viewpoint is determined so that the number of model feature groups that can be observed from a viewpoint different from the first measurement viewpoint is larger than the evaluation value. Item 1. The information processing apparatus according to item 1. The measurement feature evaluation means evaluates the bias of the distribution of the model feature group having no geometrically similar measurement features obtained by the position / orientation estimation means.
The measurement viewpoint determining means determines the second measurement viewpoint so that the direction in which the distribution of the model feature group is largely biased and the measurement feature group distribution acquired by the measurement feature acquisition means are parallel to each other. The information processing apparatus according to claim 1. The measurement feature evaluation means evaluates the spread of the distribution of the model feature group having no geometrically similar measurement features obtained by the position / orientation estimation means.
In the measurement viewpoint determining means, based on the evaluation result of the measurement feature evaluation means, the minimum area of the measurement feature acquired by the measurement feature acquisition means has a wider distribution of the measurement feature group and the model feature group. The information processing apparatus according to claim 1, wherein the second measurement viewpoint is determined so as to be smaller. It further has a measurement feature integration means for adding the measurement feature group of the measurement object acquired from the second measurement viewpoint and a measurement feature holding means for holding the measurement feature group added by the measurement feature integration means.
The position / orientation estimation means measures based on the measurement feature group of the model feature group, the measurement feature group of the measurement target acquired by the measurement feature acquisition means, and the measurement feature group of the measurement target held by the measurement feature holding means. Estimate the position and orientation of the object and
The measurement feature evaluation means is a measurement feature group of a measurement object acquired by the measurement feature acquisition means and the measurement feature group, based on the position / posture value estimated by the position / posture estimation means, the model feature group, and the measurement feature group. Evaluate the measurement feature group of the measurement object held by the measurement feature holding means,
The information according to claim 1, wherein the measurement viewpoint determining means determines a second measurement viewpoint different from the first measurement viewpoint based on an evaluation value calculated by the measurement feature evaluation means. Processing equipment. The measurement feature evaluation means projects and evaluates the bias of the measurement feature group geometrically similar to the model feature group obtained by the position / orientation estimation means onto the measurement object at the first measurement viewpoint.
The measurement viewpoint determining means determines the second measurement viewpoint in a direction in which the bias of the measurement feature group is small and at a position where the measurement object is observed.
The information processing apparatus according to claim 5, wherein the measurement feature integration means integrates a group of measurement features of the measurement object measured from the second measurement viewpoint. The measurement feature evaluation means has a spread of the distribution of the model feature group that can be observed from the first measurement viewpoint and a distribution of the measurement feature group that is geometrically similar to the model feature group obtained by the position / orientation estimation means. Evaluate the spread of
The measurement viewpoint determining means is the second measurement viewpoint so that a region where the distribution of the observable model feature group and the distribution of the measurement feature group geometrically similar to the model feature group do not overlap can be observed. The information processing apparatus according to claim 5, wherein the information processing apparatus is determined. The information processing apparatus according to any one of claims 1 to 7, wherein the measurement feature acquisition means acquires an image feature group of a two-dimensional image obtained by photographing the measurement object. The information processing apparatus according to any one of claims 1 to 8, wherein the measurement feature acquisition means acquires a three-dimensional point cloud on the surface of the measurement object. A measurement device for acquiring the measurement feature group and / or a measurement viewpoint for moving a measurement object to a measurement viewpoint determined by the measurement viewpoint determination means by a robot having a movable axis including a rotation and / or translational movement axis. Means of change and
The information processing apparatus according to any one of claims 5 to 7, further comprising a measurement feature integration means for integrating the measurement features acquired from the second measurement viewpoint into the measurement feature holding means. It is a control method for information processing equipment.
A measurement feature acquisition process that acquires measurement features measured from the first measurement viewpoint, and
A model feature group representing the shape of the measurement target, a position / orientation estimation process for estimating the position / orientation of the measurement target from the measurement feature group of the measurement target acquired in the measurement feature acquisition process, and a position / orientation estimation process.
Based on the position / orientation value estimated in the position / orientation estimation process, the model feature group, and the measurement feature group, a measurement feature evaluation step for evaluating the measurement feature group of the measurement object acquired in the measurement feature acquisition step, and a measurement feature evaluation step.
Based on the result of evaluation in the measurement feature evaluation step, a measurement viewpoint determination step of determining a second measurement viewpoint different from the first measurement viewpoint, and a measurement viewpoint determination step.
A method for controlling an information processing device, which comprises. A program for causing a computer to execute each step of the control method of the information processing apparatus according to claim 11.

Images