JP2021085781A - Information processing device, information processing method, measuring device, program, system, and article manufacturing method - Google Patents

Abstract

To provide an information processing device that accurately calculates the attitude of a target object.SOLUTION: Provided is an information processing device that calculates the attitude of a target object. The information processing device comprises: image acquisition means for acquiring an image of the target object and acquiring three-dimensional information of the target object; acquisition means for acquiring, from a model of the target object, information of a first specific portion that is a portion for specifying an attitude around a first axis with regard to the target object and information of a second specific portion that is a portion for specifying an attitude around a second axis, different from the first axis, with regard to the target object; and calculation means for rotating the model around the first axis and performing fitting of the model to the three-dimensional information on the basis of information of the first and second specific portions acquired by the acquisition means, and rotating the model around the second axis and performing fitting of the model to the three-dimensional information, thereby calculating the attitude of the target object.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device, an information processing method, a measuring device, a program, a system, and a method for manufacturing an article.

In a production line such as a factory, one individual is identified from a pile of objects (objects, workpieces) using a vision system, and by measuring the three-dimensional position and posture, it can be grasped by a hand attached to the robot. The technology to do has been developed in recent years.

As a method of measuring the three-dimensional position and orientation of an object, the approximate position and orientation of one individual is detected from the captured image of the target object, and the three-dimensional shape model of the object is applied to the image data with the position and orientation as the initial value. There is a (model fitting) method. With this method, depending on the shape of the object, it may not be possible to calculate the correct position and orientation even if the model fitting in the subsequent stage is performed.

For example, in the case of an object having similar shapes on the front and back sides, it is assumed that the position and orientation of the back side are erroneously detected in the detection process even though the front side is observed. At this time, even if the model fitting is performed with the position / orientation as the initial value, the solution converges to an erroneous solution, and the correct position / orientation cannot be calculated.

On the other hand, Patent Document 1 describes a method of suppressing erroneous recognition by performing fitting from two symmetrically arranged postures, which are easily error-prone to each other, and comparing the results of the two postures. .. Specifically, the three-dimensional shape model is fitted to the image, and at that time, the axis that greatly contributes to the posture convergence (the posture convergence is easy) is obtained. Then, a position / orientation in which the target object is rotated so that the direction of the obtained axis is reversed with respect to the position / orientation calculated by fitting is created as a new initial value, and fitting is performed separately from there. Then, by comparing these fitting results and selecting the best one, false recognition is suppressed.

Further, in the case of an object having substantially rotational symmetry around a certain axis, a similar position / orientation around the rotation axis may be erroneously detected in the detection process, and the correct position / orientation cannot be calculated. On the other hand, in Patent Document 2, the rotation axis is first specified, then the portion that characterizes the posture around the rotation axis is focused on, and the feature portion is searched around the rotation axis in the captured image, thereby substantially rotationally symmetric. It suppresses erroneous recognition around the rotation axis of a property.

Japanese Unexamined Patent Publication No. 2012-163450 JP-A-2017-967479

However, in the methods described in Patent Document 1 and Patent Document 2, for an object having substantially rotational symmetry (first symmetry) around a certain axis and having a second symmetry different from that. In some cases, it may not be possible to obtain the correct posture. The object having the second symmetry refers to an object having substantially rotational symmetry around the z-axis and substantially inversion symmetry around the x-axis as shown in FIG. 2, for example.

When the method of Patent Document 1 is applied to this object, a posture in which the directions of the axes in the longitudinal direction of the object are exchanged is created as a candidate. Therefore, the posture can be correctly estimated for the substantially inversion symmetry around the x-axis, but since there is no means for specifying the posture for the substantially rotational symmetry around the z-axis, the method of Patent Document 1 is applied. It is difficult to calculate the correct posture of an object.

Further, when the method of Patent Document 2 is applied to this object, the posture can be correctly estimated with respect to the substantially rotational symmetry around the z-axis. However, since there is no means for specifying the posture with respect to the substantially inversion symmetry around the x-axis, it is difficult to calculate the correct posture of the object by applying the method of Patent Document 2.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an information processing device that accurately calculates the posture of a target object.

In order to achieve the object, the present invention is an information processing apparatus for calculating the posture of a target object, which is an image acquisition means for acquiring an image of the target object and acquiring three-dimensional information of the target object, and an object. To identify the information of the first specific part, which is a part for specifying the posture around the first axis of the target object, and the posture of the target object around the second axis, which is different from the first axis. Based on the acquisition means for acquiring the information of the second specific part, which is the part of, and the information of the first specific part and the information of the second specific part acquired by the acquisition means, a model is formed around the first axis. It is provided with a calculation means for calculating the posture of the target object by rotating the model to fit the three-dimensional information and rotating the model around the second axis to fit the model and the three-dimensional information. It is characterized by that.

According to the present invention, for example, it is possible to provide an information processing device that accurately calculates the posture of a target object.

It is a figure which shows an example of the hardware configuration of an information processing apparatus. It is a figure which shows an example of the target object. It is a figure which shows an example of the structure of an information processing apparatus. It is a figure which shows the flowchart which shows the processing procedure which calculates the position | position | position | position | position | posture of the target object in the position / posture calculation part. It is a figure explaining the extraction method of the part which characterizes the 1st symmetry. It is a figure explaining the extraction method of the part which characterizes the second symmetry. It is a figure which shows an example of the target object in Example 2. FIG. It is a figure which showed the example of the control system which includes the gripping apparatus equipped with the information processing apparatus.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings with reference to examples. In each figure, the same members or elements are given the same reference numbers, and duplicate explanations are omitted or simplified.

[Example 1]
In the present embodiment, when the position and orientation of the piled object objects (workpieces, objects) are accurately obtained and the present invention is applied when the object object is gripped by the robot hand based on the obtained position and orientation. Will be described. In this embodiment, the position / posture includes position or posture information.

FIG. 1 is a hardware configuration diagram of the information processing device 100 in this embodiment. In FIG. 1, the CPU 1030 comprehensively controls each device connected via the bus 1000. The CPU 1030 reads and executes the processing steps and programs stored in the read-only memory (ROM) 1020. The operating system (OS), each processing program, device driver, etc. according to this embodiment are stored in the ROM 1020, temporarily stored in the random access memory (RAM) 1030, and appropriately executed by the CPU 1030. Further, the input I / F 1040 is input as an input signal from an external device (imaging device, operating device, etc.) in a format that can be processed by the information processing device 100. Further, the output I / F 1050 is output as an output signal to an external device (display device) in a format that can be processed by the display device. Further, the output I / F 1050 outputs the position / orientation information of the target object to the externally connected robot controller.

FIG. 2 is a diagram showing a target object in this embodiment. Prior to explaining the configuration of the information processing apparatus 100 in this embodiment, the target object in this embodiment will be described below with reference to FIG. The target object targeted in this embodiment is, for example, an object that is easily erroneously recognized when rotated with respect to a predetermined axis (z-axis). Further, it is an object that is easily erroneously recognized when rotated by 180 degrees with respect to an axis (x-axis) different from that. The target object in FIG. 2 has substantially rotational symmetry around the z-axis and substantially inversion symmetry around the x-axis. Further, the portion 201 in FIG. 2 is a characteristic portion (first specific portion) that uniquely determines the position / orientation around the z-axis (uniquely specifies the posture). The portion 202 is a characteristic portion (second specific portion) that uniquely determines the substantially inverted position posture (uniquely specifies the posture).

FIG. 3 is a diagram showing a configuration of an information processing device 100 according to this embodiment. Here, the configuration of the information processing apparatus 100 will be described below with reference to FIG.

The information processing device 100 is composed of a recognition target information acquisition unit 30, a three-dimensional shape model holding unit 31, a specific part acquisition unit 32, a rough position / orientation recognition unit 33, and a position / orientation calculation unit 34. Then, it is connected to the image pickup device 35, the display device 36, the operation device 37, and the robot controller 38. In this embodiment, the image pickup device 35, the display device 36, the operation device 37, and the robot controller 38 are connected to the outside of the information processing device 100 in this way. In this embodiment, the image pickup device 35, the display device 36, the operation device 37, and the robot controller 38 may be further configured as an integrated information processing device. Hereinafter, each part constituting the information processing apparatus 100 will be described.

The recognition target information acquisition unit (image acquisition unit) 30 acquires an image (shade image, distance image) from the image pickup device 35, performs image processing based on the image, and performs measurement information (target) which is three-dimensional information of the target object. Acquires and holds edge information and distance point group information of an object.

The three-dimensional shape model holding unit 31 holds a three-dimensional shape model of the target object to be picked in piles. As the three-dimensional shape model, for example, a polygon model in which the three-dimensional shape of the target object is approximately represented by a combination of a plurality of polygons can be used. Each polygon is composed of the position of a point (three-dimensional coordinates) on the surface of the target object and the connection information of each point for forming a polygon that approximates the surface. The polygon is generally composed of a triangle, but may be a quadrangle or a pentagon. In addition, any polygon model that can approximate the shape of the object by the three-dimensional coordinates of the surface points and the connection information thereof may be used. Alternatively, as the three-dimensional shape model, a model such as CAD data may be used in which the shape is represented by a set of divided parameter curved surfaces called a B-Rep (Boundary-Representation) representation. In addition, any model that can express the three-dimensional shape of the object may be used. The three-dimensional shape model is also used when the position / orientation calculation unit 34 measures the position / orientation of the target object. It is assumed that a reference coordinate system (model coordinate system) for representing the coordinates of points on the surface of the target object is set in the model in advance. The three-dimensional shape model holding unit 31 is composed of a memory, a hard disk, or the like, but a three-dimensional shape model may be acquired from a storage medium or the like.

The specific part acquisition unit 32 is a characteristic part (first specific part) that uniquely identifies the posture with respect to the first symmetry (approximately rotational symmetry) around the first axis from the three-dimensional shape model of the target object. Is specified (extracted). Further, a characteristic portion (second specific portion) that uniquely specifies the posture with respect to the second symmetry (substantially inverted symmetry) around the second axis different from the first axis is specified (extracted). After that, the three-dimensional shape model information of the specific part in the first symmetry and the second symmetry is acquired and held. As described above, in the case of the target object shown in FIG. 2, it is substantially rotationally symmetric around the z-axis (around the first axis). Then, in order to uniquely determine the position and orientation of the 201 part around the z-axis, the three-dimensional shape model information of the corresponding part is used as a specific part (characteristic part) that uniquely specifies the posture with respect to the first symmetry. Get and hold. Further, the target object in FIG. 2 is substantially inverted symmetric around the x-axis (around the second axis). Then, since the 202 part uniquely determines the substantially inverted position posture, the three-dimensional shape model information of the corresponding part is acquired and held as a specific part (characteristic part) that uniquely specifies the posture with respect to the second symmetry. To do.

The approximate position / orientation recognition unit 33 acquires a three-dimensional shape model of the target object from the three-dimensional shape model holding unit 31. In addition, measurement information (edge information of the target object, distance point cloud information) is acquired from the recognition target information acquisition unit 30. In this embodiment, it is assumed that the image includes a pile of target objects. Then, the approximate position / orientation recognition unit 33 detects one individual from the piled up target objects included in the three-dimensional shape model image, and calculates approximate values of the position and orientation of the target object with respect to the imaging device 35 described later. As a result, the approximate position and posture of one individual is recognized. It is assumed that the image pickup apparatus 35 is defined with a three-dimensional coordinate system (reference coordinate system) that serves as a reference for position and attitude measurement.

In this embodiment, the center of the sensor used in the image pickup apparatus 35 is the origin, and the coordinate system with the horizontal direction of the acquired image as the x-axis, the vertical direction as the y-axis, and the optical axis of the sensor as the z-axis is the reference coordinate system. And. The position / orientation of the target object with respect to the image pickup apparatus 35 represents the position / orientation of the symmetrical object in the reference coordinate system. In this embodiment, the rough position and orientation of one individual is calculated by performing pattern matching on the distance image and the shading image acquired by the sensor using images observed from a plurality of viewpoints as templates. However, the method of recognizing the approximate position / posture may be another method. In addition, any method other than those described here may be used as long as one individual can be found in the pile and the three-dimensional position and orientation can be calculated. In this embodiment, since the target object is an object that is easily erroneously recognized when the target object is rotated with respect to a predetermined axis, there is a high possibility that the approximate position and orientation acquired here are erroneously recognized. The position / orientation calculated by the approximate position / posture recognition unit 33 is input to the position / orientation calculation unit 34, which will be described later, as the approximate position / posture.

The position / orientation calculation unit 34 acquires a three-dimensional shape model of the target object from the three-dimensional shape model holding unit 31. Further, the position / orientation calculation unit 34 acquires the three-dimensional shape model information of the characteristic portion that uniquely specifies the posture with respect to the symmetry from the specific portion acquisition unit 32. Further, the position / posture calculation unit 34 acquires the approximate position / posture from the approximate position / posture recognition unit 33. Further, the position / orientation calculation unit 34 acquires measurement information from the recognition target information acquisition unit 30. Then, the position and orientation of the target object are calculated from the acquired information. In this embodiment, the position / posture calculation unit 34 calculates both the position and the posture, but only the posture may be calculated.

Specifically, first, based on the acquired approximate position and orientation, the measurement points (measurement information) in the measurement target object and the three-dimensional shape model are aligned, and the approximately overlapped points are used as the first reference. The position and orientation of the 3D shape model are changed so that the error between the measurement point and the point on the 3D shape model is minimized. As a method of changing the position and the posture, a method called an ICP (Iterative Closet Point) algorithm is known. The ICP algorithm is divided into two steps: associating the measurement point with the nearest neighbor point on the 3D shape model, and calculating the attitude conversion that minimizes the error between the measurement point and the point on the 3D shape model. By repeating these two steps, the error between the measurement point and the point on the model data MD becomes smaller and smaller, and when the error becomes less than a certain value, the iterative process is terminated and the final result is obtained. However, the method of changing the position and the posture may be a method other than ICP. After that, the symmetry around the axis is determined, and the correct position and orientation of the target object are calculated.

The image pickup device (imaging unit) 35 is a sensor that acquires measurement information necessary for recognizing the position and orientation of the target object. For example, a camera that captures a two-dimensional image may be used, a distance sensor that captures a distance image in which each pixel has depth information, or a combination of these may be used. As the distance sensor, there is a method of measuring the distance by triangulation by photographing the reflected light of the laser light or the slit light irradiating the target with a camera, and a Time-of-light method using the flight time of the light. In addition, a method of calculating the distance by triangulation from an image taken by a stereo camera can also be used. In addition, any sensor may be used as long as it can acquire the information necessary for recognizing the three-dimensional position and orientation of the object.

The image pickup device 35 may be fixed upward or sideways with respect to the target object, or may be provided on a robot hand or the like. In this embodiment, it is assumed that a sensor capable of acquiring both a distance image and a shade image is used. As described above, the measurement information acquired by the imaging device 35 is input to the recognition target information acquisition unit 30. The coordinate system set in the image pickup apparatus 35 will be referred to as a sensor coordinate system hereafter.

The display device 36 acquires the three-dimensional shape model from the three-dimensional shape model holding unit 31 and displays it. Further, the image acquired from the image pickup apparatus 35 and the position / orientation calculated by the position / orientation calculation unit 34 may be displayed so that the user can confirm the position / orientation. As the display device 36, for example, a liquid crystal display, a CRT display, or the like is used. The operation device 37 is a keyboard or a mouse, and is used for inputting an instruction from the user. In particular, the mouse is used for operating the GUI. The robot controller 38 acquires the position / posture information of the target object calculated by the information processing device 100, and instructs the robot on the gripping position / posture of the target object based on the acquired position / posture information.

FIG. 4 is a flowchart showing a processing procedure for calculating the position and orientation of the target object by using the information processing device 100 as a measuring device in combination with the imaging device 35 that images the target object. Here, a method of calculating the position and orientation of the target object in the position and orientation calculation unit 34 of the information processing apparatus 100 will be described in detail with reference to FIG.

First, in step S401, the position / orientation calculation unit 34 acquires the three-dimensional shape model of the target object from the three-dimensional shape model holding unit 31. Next, in step S402, the recognition target information acquisition unit 30 acquires the distance image and the shading image of the captured target object through the interface by the image pickup device 35, and acquires the measurement feature of the target object from the image. .. In this step, the imaging device 35 also functions as an imaging means for capturing an image of the target object. Further, the recognition target information acquisition unit 30 also functions as an acquisition means for acquiring an image of the target object and further performing a process of acquiring measurement features which are three-dimensional information of the target object. Next, in step S403, the specific part acquisition unit 32 extracts and acquires information on a characteristic part (specific part) that uniquely specifies the posture with respect to the first symmetry from the three-dimensional shape model of the target object. Further, the information of the characteristic part (specific part) that uniquely specifies the posture with respect to the second symmetry is extracted and acquired. In this step, the specific part acquisition unit 32 acquires information on the first specific part, which is a part for uniquely specifying the posture around the first axis of the target object. Further, it also functions as an acquisition means for performing a process of acquiring information on a second specific part, which is a part for uniquely specifying the posture of the target object around the second axis.

FIG. 5 is a diagram illustrating a method of extracting a portion that characterizes the first symmetry. Here, the extraction method of the first specific portion will be described in detail with reference to FIG. First, from the viewpoint of interest, an image (reference image) obtained by projecting a three-dimensional shape model of the target object onto the image coordinate system of the image pickup apparatus 35 is generated. As for the image, it is assumed that 1 is stored in the pixel on which the model is projected, and 0 is stored in the other pixels. Next, the three-dimensional shape model of the target object is rotated 360 degrees around the rotation axis (first axis) showing the first symmetry of the target object at predetermined angular intervals, and similarly to the image coordinate system of the image pickup device 35. Generate a rotated image, which is a projected image. Here, when generating the rotated image, a plurality of rotated images rotated at predetermined angular intervals (360 / N degrees) are generated around the rotation axis. The total number of rotated images generated at this time is N. The number of rotated images generated may be specified by the user. Therefore, the generation of the rotated image will generate one or more images. The predetermined angle interval may be set to a constant angle interval, or an angle interval having a certain regularity may be set. Next, a difference image obtained by extracting the difference between the reference image and each rotated image is generated. Finally, all the difference images are compared to generate an AND image, a three-dimensional model point of the target object corresponding to the region whose value is not 0 is extracted, and this is acquired as information of the first specific part. Here, the acquired information of the first specific part is held as the difference data 1.

FIG. 6 is a diagram illustrating a method of extracting a portion that characterizes the second symmetry. Subsequently, with reference to FIG. 6, a method for extracting the second specific site will be described in detail. From the viewpoint of interest, an image (reference image) obtained by projecting a three-dimensional shape model of the target object onto the image coordinate system of the image pickup device 35 is generated. For example, it is assumed that the image stores 1 in the pixel on which the model is projected and 0 in the other pixels. Next, a inverted image is generated, which is an image in which the three-dimensional shape model of the target object is inverted (rotated) by 180 degrees around the second axis orthogonal to the first axis. Then, with this inverted posture as the initial posture, the three-dimensional shape model of the target object is rotated 360 degrees around the first axis at predetermined angular intervals, and each posture is projected onto the image coordinate system of the image pickup apparatus 35. Generate a rotated image. Here, when generating the rotated image, a plurality of rotated images rotated at predetermined angular intervals (360 / N degrees) are generated around the first axis. The total number of rotated images generated at this time is N. The number of rotated images generated may be specified by the user. Therefore, the generation of the rotated image will generate one or more images. The predetermined angle interval may be set to a constant angle interval, or an angle interval having a certain regularity may be set.

Next, a difference image is generated by extracting the difference between the reference image generated in the reference posture before inversion and each rotation image generated in the inversion and rotation around the rotation axis. Finally, all the difference images are compared to generate an AND image, a three-dimensional model point of the target object corresponding to the region whose value is not 0 is extracted, and this is acquired as information of the second specific part. The information of the second specific part acquired here is held as the difference data 2.

Here, the difference data 1 includes information on a characteristic portion (first specific portion) that uniquely specifies the posture with respect to the first symmetry. The difference data 2 includes information on a characteristic portion (second specific portion) that uniquely identifies the posture with respect to the second symmetry. The difference data 2 includes both information on a characteristic part that uniquely identifies the posture with respect to the first symmetry and information on a characteristic part that uniquely specifies the posture with respect to the second symmetry. You can stay.

Further, the position / orientation calculation unit 34 is information on a characteristic portion that uniquely identifies the posture with respect to the first symmetry for each point belonging to the difference data 2, or the posture is uniquely specified with respect to the second symmetry. It also functions as a classification means for classifying whether the information is characteristic parts to be specified. In the classification, for example, if the distance from the point of the difference data 1 which is the closest to the point belonging to the difference data 2 is within a predetermined threshold value, the point uniquely identifies the posture with respect to the first symmetry. Classify into information on characteristic parts. The other points are classified into the information of the characteristic part that uniquely identifies the posture with respect to the second symmetry. The predetermined threshold value may be set in advance by the user, for example.

Returning to FIG. 4, next, in step S404, the approximate position / orientation recognition unit 33 detects one of a large number of piled up target objects existing in the captured image, and detects the target object in the sensor coordinate system. Six parameters representing the approximate position and orientation are calculated and recorded. In the coordinate transformation from the model coordinate system to the sensor coordinate system based on the six parameters calculated here, the 3 × 3 rotation matrix represented by the three parameters representing the posture is represented by RSM and the three parameters representing the position. Let TSM be the translation vector of the three columns. At this time, the conversion from the model coordinate system XM = [XM, YM, ZM] T to the sensor coordinate system XS = [XS, YS, ZS] T is performed by using the following equation (4 × 4 matrix T0'). It can be expressed as 1).
XS'= T0'XM' (1)
In the above formula (1), XS'= [XS, YS, ZS, 1] T and XM'= [XM, YM, ZM, 1] T. Here, T0'is the approximate position and posture.

Next, in step S405, the position / posture calculation unit 34 performs model fitting between the three-dimensional shape model of the target object and the target object in the image with the approximate position / posture T0'as the initial value, thereby performing the position / posture of the target object. Is calculated. Specifically, the three-dimensional shape model is projected onto the captured image based on the parameters of the image pickup apparatus 35 and the approximate position and orientation. Then, the information of the specific part which is the feature of the projected three-dimensional shape model is associated (fitting) with the measurement feature which is the feature of the target object in the image. That is, the difference data 1 which is the information of the first specific part acquired around the first axis and the measurement information are fitted, and the difference data 2 which is the information of the second specific part acquired around the second axis is combined with. Fit with measurement information. Then, the position and orientation of the target object are calculated by reducing the residual. This makes it possible to calculate the highly accurate position and orientation of the target object. The 4 × 4 rotation matrix T0 that performs coordinate conversion from the model coordinate system to the sensor coordinate system, which can be expressed by the six parameters of the position / orientation calculated here, is the position / orientation T0 calculated by the position / orientation calculation unit 34. In this step, the position / posture calculation unit 34 functions as a calculation means for performing a process of calculating the position / posture of the target portion by using the calculation method described above.

Next, in step S406, the position / posture calculation unit 34 calculates a predetermined value (score) for the calculated position / posture T0, compares it with a predetermined threshold value, and determines whether the position / posture is correct or incorrect. It is determined again whether or not to perform the process of calculating the correct position / orientation. As a predetermined threshold value, for example, the amount of deviation is acquired by using the three-dimensional distance between the geometric feature, which is a characteristic part on the model surface in the position and orientation after fitting, and the geometric feature in the image as a residual. .. Then, the average value E of the residuals of all geometric features can be used as the score. In this step, the position / orientation calculation unit 34 also functions as a determination means for determining whether or not the position / orientation of the target object is correct and whether or not to perform the process of calculating the correct position / orientation again.

When the average value E of the calculated residuals is smaller than a predetermined threshold value (for example, 0.1 mm), it is determined that the correct position and orientation can be calculated, and the process is terminated. On the other hand, when the average value E of the residuals is larger than the threshold value, it is determined that the wrong position and orientation have been obtained, and the process proceeds to step S407. The predetermined threshold value may be set in advance by the user, for example. Further, the method of determining the correctness of the position / posture is not limited to this. For example, based on the calculated position / orientation T0, the normalized intercorrelation coefficient R of the brightness in the object region between the image rendered by projecting the model and the captured image can be obtained, and this value can be used. Good. In this case, if the normalized intercorrelation coefficient R is larger than a predetermined value (for example, 0.9), it is determined that the correct position / orientation has been calculated, and the process ends. On the other hand, if it is small, the process proceeds to step S407. In addition, any method may be used as long as it can distinguish the calculated correctness of the position and posture.

Next, in step S407, the position / orientation calculation unit 34 calculates the correct position / orientation of the target object in consideration of the first symmetry and the second symmetry of the target object. Let N1 be the number of model points of the part that characterizes the first symmetry and N2 be the number of model points that characterize the second symmetry extracted in S403. Further, the threshold value for performing the first symmetry determination is σ1, and the threshold value for performing the second symmetry determination is σ2. In this step, by switching the processing as follows according to the number of model points of the part that characterizes the symmetry, the redundancy of the processing can be eliminated and the processing can be performed at high speed.

In the case of N1 <σ1 and N2 <σ2, since the part that characterizes the symmetry is sufficiently small, the symmetry discrimination is not performed and no additional processing is performed.

When N1> σ1 and N2 <σ2, the part that characterizes the first symmetry is large, and there is a possibility that the symmetry around the first rotation axis (around the first axis) is incorrect. In this case, the score is recalculated at each of the rotated positions and postures by rotating 360 degrees around the first rotation axis with respect to the position and posture calculated in step S405. Then, the position / posture with the highest score is set as the correct position / posture.

In the case of N1 <σ1, N2> σ2, the part that characterizes the second symmetry is large, and there is a possibility that the symmetry around the second rotation axis (around the second axis) is incorrect. In this case, the score is recalculated based on the position / posture calculated in step S405 by rotating 180 degrees around the second rotation axis and the position / posture before and after the rotation. Then, the position / posture with the highest score is set as the correct position / posture.

When N1> σ1, N2> σ2, and 0 <N1 <N2, the influence of the posture error due to the second symmetry is larger than that of the first symmetry. In this case, first, with respect to the position / posture calculated in step S405, the position / posture is rotated 180 degrees around the second rotation axis, the score is recalculated based on the position / posture before and after the rotation, and the position / posture with the higher score is provisionally set. The correct position and posture of. Next, the score is recalculated at each of the rotated positions and postures by rotating 360 degrees around the first rotation axis at predetermined angular intervals with respect to the tentative correct position and posture. Then, the position / posture with the highest score is set as the correct position / posture.

When N1> σ1, N2> σ2, and 0 <N2 <N1, the influence of the posture error due to the first symmetry is larger than that of the second symmetry. In this case, first, with respect to the position / posture calculated in step S405, the score is recalculated at each of the rotated positions / postures by rotating 360 degrees around the first rotation axis at predetermined angular intervals. The position / posture with the highest score is saved as the correct position / posture candidate. Next, with respect to the position / orientation calculated in step S405 by 180 degrees around the second rotation axis, each rotation is rotated 360 degrees around the first rotation axis at predetermined angular intervals. Recalculate the score based on the position and posture. The position / posture with the highest score is saved as the correct position / posture candidate. Finally, among the candidates for the correct position and orientation, the position and orientation with the highest score is set as the correct position and orientation.

The predetermined interval (360 / N degrees) for rotating 360 degrees in the recalculation of the first or second specific part may be set to a constant angular interval as described above, or An angular interval with a certain regularity may be set. In this embodiment, the symmetry is determined based on the number of model points of the parts that characterize the symmetry, but other indexes may be used as long as the information is based on the size of the parts that characterize the symmetry. For example, the same determination may be made based on the (occupied) area of a specific portion, which is a portion that characterizes symmetry on the reference image. Further, although this embodiment is carried out by the flowchart, it may be carried out by a discrete circuit or the like.

By the above method, it is possible to erroneously recognize an object having a first symmetry around the first axis and a second symmetry around the second axis different from the first axis. It is possible to accurately calculate the correct position and orientation of the object. Further, in step S407, when determining the correct position and orientation of the target object in consideration of the first symmetry and the second symmetry, the processing is switched based on the size of the portion that characterizes each symmetry. I did. As a result, redundant processing can be eliminated, and processing can be performed at a higher speed than in the case of comprehensively discriminating the first symmetry and the second symmetry.

[Example 2]
In the first embodiment, as the second symmetry, a method of correctly calculating the position and orientation of an object whose front and back are liable to be mistaken without erroneous recognition has been described. In this embodiment, the case where the process of Example 1 is applied to a target object having substantially rotational symmetry of every 90 degrees as a predetermined angle around the axis as the second symmetry will be described. Further, the object is not limited to an object having 90 degrees around the axis, as long as it has a predetermined symmetry around the axis, and the first symmetry around the first axis and the second axis around the axis. Any object may have any second symmetry. An object having a predetermined symmetry is, for example, an object having a certain angular interval or an object having an angular interval having a certain regularity. Since the configuration of the information processing device 100 in this embodiment is the same as that of the information processing device 100 of FIG. 3 shown in the first embodiment, the description thereof will be omitted. Further, since the processing procedure in this embodiment is the same as the processing procedure of FIG. 4 shown in the first embodiment, the description thereof will be omitted.

FIG. 7 is a diagram showing a target object as a target in this embodiment. FIG. 7A shows the correct position and orientation of the target object, and FIGS. 7B and 7C show the position and orientation of the target object in the case of erroneous recognition. The target object of this embodiment has substantially rotational symmetry (first symmetry) every 90 degrees around the z-axis of FIG. 7, and substantially rotational symmetry every 90 degrees around the x-axis of FIG. Has (second symmetry). Reference numeral 701 shown in FIG. 7 is a characteristic portion (first specific portion) that uniquely specifies the posture with respect to the first symmetry. Further, reference numeral 702 shown in FIG. 7 is a characteristic portion (second specific portion) that uniquely specifies the posture with respect to the second symmetry. In this embodiment as well, since the target object has the first and second symmetry, the portion that characterizes the symmetry is extracted by the method described in the first embodiment. Based on this, rotation images are generated for each of the first and second specific parts, and the generated rotation images and reference images for each of the first and second specific parts are produced by the same method as in Example 1. By comparing and discriminating the symmetry, the correct position and orientation can be calculated without erroneous recognition.

By the above method, even if the object does not have the rotational symmetry of 360 degrees as shown in the first embodiment, the object has the first symmetry around the axis and the second symmetry different from the first symmetry. The correct position and orientation of the target object can be calculated accurately without erroneously recognizing the object.

[Examples relating to the article manufacturing method]
The information processing device 100 described above can be used in a state of being supported by a certain support member. In this embodiment, as an example, a control system attached to and used in the robot arm 800 (grip device) as shown in FIG. 8 will be described. Although not shown in FIG. 8, the information processing device 100 is configured as a measuring device using the image pickup device 35, and the pattern light is projected onto the object W placed on the support base T to take an image and acquire an image. .. Then, the control unit of the information processing device 100 or the arm control unit 810 that has acquired the image output from the control unit of the information processing device 100 determines the position and posture of the object W. The arm control unit 810 controls the robot arm 800 by sending a drive command to the robot arm 800 based on the position and posture information (measurement result). The robot arm 800 holds the object W with a robot hand (grip portion) at the tip and moves the object W in translation or rotation. Further, by assembling (assembling) the object W to other parts by the robot arm 800, it is possible to manufacture a predetermined article composed of a plurality of parts, for example, an electronic circuit board or a machine. In addition, an article can be manufactured by processing (processing) the moved object W. The arm control unit 810 has an arithmetic unit such as a CPU as a computer and a storage device such as a memory for storing a computer program. A control unit that controls the robot may be provided outside the arm control unit 810. Further, the measurement data measured by the information processing apparatus 100 and the obtained image may be displayed on a display unit 820 such as a display. Further, the object W may be gripped and moved in order to align the object W with respect to other parts. The object W has the same configuration as the target object shown in FIG. Further, the measuring device may be configured to include the robot arm 800.

[Other Examples]
Although the present invention has been described in detail based on the preferred examples thereof, the present invention is not limited to the above examples, and various modifications can be made based on the gist of the present invention. It is not excluded from the scope of the invention.

Further, a computer program that realizes a part or all of the control in each of the above-described embodiments may be supplied to the information processing device 100, the measuring device, or the like via a network or various storage media. Good. Then, the computer (or CPU, MPU, etc.) in the information processing device 100 or the measuring device may read the computer program from the recording medium and execute it. In that case, the computer program and the storage medium that stores the computer program constitute the present invention. As the recording medium, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, or the like can be used. Further, the computer program read from the storage medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer. After that, based on the instruction of the computer program, the function expansion board, the CPU provided in the function expansion unit, or the like performs a part or all of the actual processing, and the processing includes the case where the function of the above-described embodiment is realized. Is done.

100 Information processing device 30 Recognition target information acquisition unit 31 Three-dimensional shape model holding unit 32 Specific part acquisition unit 33 Approximate position / orientation recognition unit 34 Position / orientation calculation unit 35 Imaging device 36 Display device 37 Operation device

Claims (13)

An information processing device that calculates the posture of an object.
An image acquisition means for acquiring an image of the target object and acquiring three-dimensional information of the target object,
From the model of the target object, information on the first specific part, which is a part for specifying the posture around the first axis of the target object, and information on the target object around the second axis different from the first axis. An acquisition means for acquiring information on a second specific part, which is a part for specifying a posture, and
Based on the information of the first specific part and the information of the second specific part acquired by the acquisition means, the model is rotated around the first axis to fit the model and the three-dimensional information. And a calculation means for calculating the posture of the target object by rotating the model around the second axis and fitting the model and the three-dimensional information.
An information processing device characterized by being equipped with. The acquisition means includes a reference image which is an image of a viewpoint which is a reference of the model of the target object, and a plurality of rotations generated by rotating the model at predetermined intervals about the first axis of the target object. The information processing apparatus according to claim 1, wherein information on the first specific portion is acquired by comparing with an image. The acquisition means obtains a reference image which is an image of a viewpoint which is a reference of the model of the target object and an inverted image obtained by reversing the model by 180 degrees about the second axis of the target object. The information processing according to claim 1, wherein information on the second specific portion is acquired by comparing with a plurality of rotated images generated by rotating the first axis at predetermined intervals. apparatus. The calculation means is characterized in that the information of the first and second specific parts and the three-dimensional information are aligned based on the approximate position and orientation of the target object and the three-dimensional shape model information. Item 1. The information processing apparatus according to item 1. The claim is characterized in that after calculating the posture of the target object, the calculation means determines whether or not to recalculate the posture of the target object by comparing the calculation result with a predetermined threshold value. Item 4. The information processing apparatus according to item 4. The information of the first specific part and the information of the second specific part are at least one of the information about the point of the three-dimensional shape model information of the target object or the information about the area of the part of the specific part on the reference image. The information processing apparatus according to claim 2 or 3, wherein the information processing apparatus includes. In the target object having specific parts around the first and second axes, a rotation image generated by rotating the target object around the first and second axes at predetermined intervals is displayed around the first and second axes. The information processing apparatus according to claim 1, wherein a rotation image for each specific part around the first and second axes generated for each part is compared with the reference image, respectively. The information processing apparatus according to any one of claims 1 to 7, wherein the calculation means also calculates information on the position of the target object. An information processing method that calculates the posture of an object.
An image acquisition step of acquiring an image of the target object and acquiring three-dimensional information of the target object, and
From the model of the target object, information on the first specific part, which is a part for specifying the posture around the first axis of the target object, and information on the target object around the second axis different from the first axis. The acquisition process of acquiring information on the second specific part, which is the part for specifying the posture,
Based on the information of the first specific part and the information of the second specific part acquired in the acquisition step, the model is rotated around the first axis to fit the model and the three-dimensional information. And the calculation step of calculating the posture of the target object by rotating the model around the second axis and fitting the model and the three-dimensional information.
An information processing method characterized by having. An imaging means that captures an image of a target object,
The information processing device according to any one of claims 1 to 8 and
A measuring device characterized by being provided with. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 8. The information processing device according to any one of claims 1 to 8 and
A system characterized by having a robot that grips and moves the target object based on the posture of the target object calculated by the information processing device. A calculation step of calculating the posture of the target object by the information processing apparatus according to any one of claims 1 to 8.
A method for manufacturing an article, which comprises a step of manufacturing a predetermined article by grasping the target object and performing an assembly process of the target object based on the calculated position and orientation of the target object. ..

Images