JP2017111019A - Model generator, position/posture calculator, and handling robot device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a model generator that generates a three-dimensional model of a target article for which no CAD data exists.SOLUTION: A model generator for generating a three-dimensional model of an article includes: a three-dimensional measuring machine A04 that three-dimensionally measures multiple articles respectively arranged in arbitrary postures in a three-dimensional space and acquires measurement data; a model measurement data extracting part that considers the measurement data for the articles measured by the three-dimensional measuring machine to be a set of model measurement data obtained by measuring one article from multiple virtual measurement positions, and extracts corresponding model measurement data with respect to each of the virtual measurement positions; and a model measurement data integrating part that generates a three-dimensional model of the article by integrating the model measurement data extracted by the model measurement data extracting part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a model generation apparatus that measures a target article with a three-dimensional measuring machine and generates a three-dimensional model of the target article using measurement data. Furthermore, the present invention relates to a position and orientation calculation apparatus that calculates the position and orientation of a target article based on a three-dimensional model of the target article and measurement data. Furthermore, the present invention relates to a handling robot apparatus that calculates a position and orientation of a target article based on a three-dimensional model of the target article and measurement data, and a robot handles the target article based on the calculated position and orientation of the article.

  In an apparatus for calculating the position and orientation of a target article (hereinafter sometimes simply referred to as “article”) by collating measurement data measured by a three-dimensional measuring machine with a three-dimensional model, The model needs to be created in advance. The three-dimensional model is created mainly by the following two methods.

  In the first method, CAD data of the target article is used. Patent Document 1 discloses a technique for generating a three-dimensional model for position and orientation estimation from CAD data. In Patent Literature 1, measurement data that can be expected to be acquired when three-dimensional measurement of the target article is generated from CAD data, and a three-dimensional model for position and orientation estimation is generated based on the generated measurement data. ing.

  In the second method, the target article is actually measured with a three-dimensional measuring machine to generate a three-dimensional model. However, a general three-dimensional measuring instrument measures only the surface of an article exposed upward. For this reason, in order to calculate various article positions and orientations, it is necessary to measure the articles from various viewpoints in advance. In Patent Document 2, a three-dimensional model of an article is generated by repeating three-dimensional measurement and calculation of a position and orientation while gradually changing the posture of the article, and integrating each measurement data based on the calculated position and orientation. doing.

JP 2010-077943 A JP 2010-113398 A

However, the conventional method described above has the following problems.
In the method of Patent Document 1, it is necessary to prepare CAD data in advance. Since the CAD data of the target article is not always available, a 3D model cannot be created in such a case. In addition, there is no guarantee that all article surfaces can be measured three-dimensionally as expected. It is difficult for a three-dimensional measuring machine using an optical system to perform three-dimensional measurement on a specific surface characteristic, for example, a black region partially existing on the surface of an article. However, when a three-dimensional model is generated from CAD data, such surface characteristics cannot be taken into consideration. Even if such surface characteristics are modeled, a modeling error occurs, and there is a possibility that a divergence occurs between the generated three-dimensional model and actual measurement data.

  In the method of Patent Document 2, it is necessary to repeat the three-dimensional measurement and the calculation of the position and orientation every time the posture of the article is changed little by little. For example, when measurement is performed while rotating a rotating body such as a cam shaft by 20 degrees around the axis, it is necessary to measure 18 times in total. Furthermore, when creating a three-dimensional model that can calculate not only the axis but also any article position and orientation, the number of measurements is enormous, and the effort to create a three-dimensional model is proportionally large. become. Therefore, a great amount of time is required to create a three-dimensional model.

  The present invention has been made in view of such circumstances, and it is possible to easily generate a three-dimensional model of a target article for which CAD data does not exist by measuring once or a small number of times using a three-dimensional measuring machine. It is an object of the present invention to provide a model generation apparatus, and a position and orientation calculation apparatus and a handling robot apparatus including such a model generation apparatus.

In order to achieve the above-described object, according to a first invention, there is provided a model generation device for generating a three-dimensional model of an article, wherein a plurality of articles respectively arranged in an arbitrary posture in a three-dimensional space are three-dimensionally arranged. A three-dimensional measuring instrument that measures and acquires measurement data, and measurement data of the plurality of articles measured by the three-dimensional measuring instrument, model measurement data obtained by measuring one article from a plurality of virtual measurement positions It is regarded as a set, and for each of the plurality of virtual measurement positions, the model measurement data extraction unit that extracts corresponding model measurement data, and the model measurement data extracted by the model measurement data extraction unit are integrated to integrate the article. And a model measurement data integration unit for generating the three-dimensional model.
According to a second aspect, in the first aspect, the model measurement data extraction unit includes a drawing unit that outputs the measurement data to a teaching screen, and a region of one article selected from a plurality of articles. An area setting unit that is set on the teaching screen from which measurement data is output, a temporary model generation unit that generates a temporary model based on the measurement data existing in the area set by the area setting unit, A detection unit that detects the position and orientation of the remaining articles of the plurality of articles using a temporary model, and the model measurement data extraction unit is based on the position and orientation of the remaining articles detected by the detection unit Model measurement data is extracted, and the model measurement data integration unit generates a three-dimensional model by integrating the model measurement data into the temporary model.
According to a third invention, in the second invention, the detection unit sequentially detects articles in a posture close to the initial posture of the temporary model, and the model measurement data extraction unit detects the detection result of the detection unit. The model measurement data is extracted in the order close to the initial posture of the temporary model, and the model measurement data integration unit sequentially integrates the model measurement data into the temporary model each time the model measurement data is extracted. To do.
According to a fourth invention, in the second or third invention, the model measurement data extraction unit includes a detection success / failure instruction unit for an operator to instruct success / failure of the detection result of the detection unit.
According to a fifth aspect, in any one of the second to fourth aspects, the model measurement data extraction unit is based on the measurement data existing in the vicinity of the position and orientation of the article detected by the detection unit. The model measurement data is extracted.
According to a sixth invention, in any one of the second to fifth inventions, the model measurement data extraction unit illegally extracts erroneous measurement data as model measurement data and illegally extracted illegal model measurement data. Even if the model measurement data is integrated with the temporary model, the model measurement data integration unit extracts and removes the incorrect model measurement data from the temporary model based on the consistency between the measurement data and the temporary model. An unauthorized model measurement data removal unit is provided.
According to a seventh invention, in any one of the second to sixth inventions, when the model measurement data integration unit integrates the model measurement data into the temporary model, the model measurement data and the temporary model If there is an overlap between the model measurement data, the model measurement data is integrated into the temporary model after removing the overlap from the model measurement data.
According to an eighth invention, in any one of the second to sixth inventions, when the model measurement data integration unit integrates the model measurement data into the temporary model, the model measurement data and the temporary model If there is an overlap between the model measurement data, the overlap of the temporary model is updated based on the overlap of the model measurement data.
According to a ninth aspect, in any one of the first to eighth aspects, the model measurement data integration unit calculates the virtual measurement position and the model measurement calculated from the position and orientation of the article detected by the detection unit. Optimization is performed using at least one of the data as a parameter to improve the accuracy of the model to be generated.
According to a tenth invention, in any one of the first to ninth inventions, after the three-dimensional measuring device three-dimensionally measures a plurality of articles arranged in a three-dimensional space, Change and repeat the new three-dimensional measurement of the plurality of articles one or more times, and the model measurement data extraction unit and the model measurement data integration unit perform a plurality of measurements obtained by a plurality of three-dimensional measurements. A three-dimensional model of the article is generated based on the data.
According to the eleventh aspect of the invention, any one of the first to tenth model generation devices, the three-dimensional model generated by the model generation device, and the measurement data acquired by the three-dimensional measuring device are associated with each other. And a position / orientation calculation unit that calculates the position / orientation of the article.
According to a twelfth aspect of the invention, a robot capable of handling the article, any one of the first to tenth model generation devices, a three-dimensional model generated by the model generation device, and the three-dimensional measuring device A position / orientation calculation unit that calculates the position / orientation of the article based on the association with the measured data, and the robot is configured to calculate the position / orientation of the article based on the position / orientation of the article calculated by the position / orientation calculation unit. There is provided a handling robot apparatus characterized by handling an article.

In the first invention, a three-dimensional model used for calculating the position and orientation of an article can be generated even for an article for which CAD data does not exist. When generating a three-dimensional model of an article using a three-dimensional measuring machine, model measurement is performed by measuring measurement data of a plurality of articles arranged in an arbitrary posture from a plurality of virtual measurement positions. It is considered as a collection of data. For this reason, a three-dimensional model can be generated by a single measurement.
In the second invention, the temporary model is first taught from the measurement data of one article, the position and orientation of the remaining article is detected using the temporary model, and the model measurement data is extracted based on the detection result. And integrated into a temporary model. For this reason, it becomes possible to generate a three-dimensional model without increasing the burden on the operator.
In the third aspect of the invention, model measurement data is extracted by detecting from an article in a posture close to the initial posture of the temporary model and sequentially integrated into the temporary model. That is, the temporary model can be expanded by detecting in order from articles that can be detected accurately and stably. As a result, the model measurement data can be finally extracted and integrated even on the surface of an article for which no model measurement data exists in the initial provisional model.
In the fourth invention, it is possible to prevent erroneous model measurement data from being added to the three-dimensional model by instructing whether or not the detection result using the temporary model is correct.
In the fifth invention, when model measurement data is extracted based on the detection result using the temporary model, points in the vicinity of the existing model measurement data are extracted as new model measurement data. Therefore, it is possible to automatically extract model measurement data without teaching.
In the sixth aspect of the invention, even if the model measurement data contains an incorrect three-dimensional point and is integrated into the temporary model, the consistency between the temporary model and the measurement data is confirmed Remove the integrated model measurement data. Thereby, a correct three-dimensional model can be generated.
In the seventh aspect, when the newly extracted model measurement data is integrated into the temporary model, if there is an overlapping 3D point between the model measurement data and the temporary model, the overlapping 3D point is Try not to integrate. For this reason, it is possible to prevent the data of the three-dimensional model from becoming heavy and to generate a three-dimensional model in which information is not excessive or insufficient.
In the eighth aspect of the invention, when newly extracted model measurement data is integrated into the temporary model, if there is an overlapping 3D point between the model measurement data and the temporary model, a new one is added from the overlapping 3D point. 3D points are calculated and the temporary model is updated. For this reason, it is possible to prevent the data of the three-dimensional model from becoming heavy and to generate a three-dimensional model in which information is not excessive or insufficient. Furthermore, a three-dimensional model with a reduced measurement error can be generated.
In the ninth aspect, after the integration processing of the model measurement data is completed, the optimization processing of the virtual measurement position and the position of each three-dimensional point belonging to the model measurement data is performed, so that the higher accuracy three-dimensional A model can be generated.
In the tenth invention, even when the number of articles that can be measured at a time is small, a three-dimensional model is generated by performing a three-dimensional measurement a plurality of times and using a plurality of measurement data. Thereby, it becomes possible to similarly generate a three-dimensional model obtained by measuring an article from various measurement positions. Even in such a case, it is possible to generate a three-dimensional model in a shorter time than measurement while changing the posture of one article little by little.
In the eleventh aspect, the position and orientation of an article can be calculated even for an article for which CAD data does not exist. Further, since the three-dimensional model is generated by measuring the article with the three-dimensional measuring machine, it is possible to greatly reduce the labor of teaching at the time of generating the three-dimensional model. Furthermore, since a three-dimensional model is created using a three-dimensional measuring machine that actually measures compared to a position / orientation calculation device that uses CAD data, the three-dimensional model and the measurement data at the time of position / orientation calculation are used. It is hard to produce wrinkles.
In the twelfth aspect, the position and orientation of an article arranged in a three-dimensional space can be calculated even for an article for which CAD data does not exist, and the article can be handled accurately based on the calculation result. Further, since the three-dimensional model of the article is generated by performing the three-dimensional measurement of the article, it is possible to greatly reduce the teaching effort when generating the three-dimensional model.

  These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the present invention illustrated in the accompanying drawings.

It is the schematic of the handling robot apparatus based on this invention. It is a functional block diagram of the three-dimensional recognition apparatus containing the model production | generation apparatus based on this invention. It is a figure for demonstrating measurement data. It is a figure for demonstrating a virtual measurement position. It is a flowchart which shows operation | movement of a handling robot apparatus. It is a 1st figure for demonstrating a temporary model. It is a 2nd figure for demonstrating a temporary model. It is a 3rd figure for demonstrating a temporary model. It is a 4th figure for demonstrating a temporary model. It is a figure for demonstrating removing incorrect model measurement data.

Embodiments of the present invention will be described below with reference to the accompanying drawings. In the following drawings, the same members are denoted by the same reference numerals. In order to facilitate understanding, the scales of these drawings are appropriately changed.
FIG. 1 is a schematic view of a handling robot apparatus according to the present invention. As shown in FIG. 1, a handling robot apparatus A00 includes a robot A01, a robot control apparatus A02 that controls the robot A01, a hand A03 attached to the arm tip of the robot A01, a three-dimensional measuring machine A04, and a three-dimensional Recognizing device A05 is mainly included.

  As shown in FIG. 1, the three-dimensional measuring machine A04 is fixed to the tip of the gantry A07. A plurality of articles A06 having the same or the same shape are stacked in the measurement range of the three-dimensional measuring machine A04. Alternatively, the plurality of articles A06 may be arranged in any posture in the three-dimensional space. The three-dimensional measuring machine A04 measures the surface of the article A06 stacked in bulk, and acquires position information of a plurality of three-dimensional points as measurement data.

  The hand A03 grips and releases one article from the plurality of articles A06, and is controlled by the robot controller A02. Note that the three-dimensional measuring machine A04 may be attached to the tip of the robot A01.

  The three-dimensional measuring machine A04 can use various methods for measuring the article A06 without contact. Such a method includes, for example, a stereo method using two cameras, a method of scanning laser slit light, a method of scanning laser spot light, a method of projecting pattern light onto an article using an apparatus such as a projector, and a light projector For example, there is a system that uses the time of flight from the time when the light is emitted to the time when the light is reflected on the surface of the article and enters the light receiver.

  As can be seen from FIG. 1, the three-dimensional measuring machine A04 and the three-dimensional recognition device A05 are connected to each other by a communication unit such as a communication cable. Similarly, the three-dimensional recognition device A05 and the robot control device A02 are communicably connected to each other by a communication unit such as a communication cable. Note that the robot control device A02 may function as the three-dimensional recognition device A05, and the robot control device A02 may generate a three-dimensional model B40, which will be described later, or calculate an article position and orientation B50.

  FIG. 2 is a functional block diagram of a three-dimensional recognition apparatus including a model generation apparatus according to the present invention. Strictly speaking, a portion B00 indicated by a broken line in FIG. 2 corresponds to the model generation device of the present invention. The model generation unit B10 shown in FIG. 2 regards the measurement data of a plurality of articles measured by the three-dimensional measuring machine A04 as a set of model measurement data obtained by measuring one article from a plurality of virtual measurement positions. For each of a plurality of virtual measurement positions, a model measurement data extraction unit B11 that extracts corresponding model measurement data and the model measurement data extracted by the model measurement data extraction unit B11 are integrated to obtain a three-dimensional model of the article. And a model measurement data integration unit B12 to be generated.

  Further, as shown in FIG. 2, the model measurement data extraction unit B11 outputs the measurement data to the drawing unit C11 that outputs the measurement data to the teaching screen C17, and the region of one article selected from a plurality of articles. Using a temporary model generation unit C13 for generating a temporary model based on the area setting unit C12 set on the taught screen, the measurement data existing in the area set by the region setting unit C12, and the temporary model And a detection unit C14 that detects the position and orientation of the remaining articles of the plurality of articles. The model measurement data extraction unit B11 includes a detection success / failure instruction unit C15 for the operator to instruct the success / failure of the detection result of the detection unit C14.

  Further, the model measurement data integration unit B12 is a case where the model measurement data extraction unit B11 illegally extracts erroneous measurement data as model measurement data and integrates the illegal model measurement data extracted illegally into the temporary model. In addition, the model measurement data integration unit B12 includes an unauthorized model measurement data removal unit C16 that extracts and removes unauthorized model measurement data from the temporary model based on the consistency between the measurement data and the temporary model.

  As can be seen from FIG. 2, measurement data B30 of a plurality of articles A06 is acquired by the three-dimensional measuring machine A04. Here, FIG. 3 is a diagram for explaining the measurement data. FIG. 3 shows a plurality of articles A06a to A06h. In the example shown in FIG. 3, these articles A06a to A06h include a shaft portion, an elliptical flange portion attached to one end of the shaft portion, and a portion near the peripheral surface of the flange portion. And a projecting portion extending in the opposite direction.

  For the purpose of facilitating understanding, the articles A06a to A06h in FIG. 3 are arranged such that their shaft portions are oriented in substantially the same direction and the flange portions are at different rotational positions around the shaft portion. As a matter of course, the plurality of articles A06a to A06h are actually stacked and may have different positions and postures. Further, the article A06 may have another shape.

  A plurality of black circles are shown on each surface of each article A06. Each of the black circles shown in FIG. 3 is measurement data B30 that means position information of a three-dimensional point on the surface of the article A06 measured by the three-dimensional measuring machine A04. The model generation unit B10 shown in FIG. 2 generates a three-dimensional model B40 of the article using the measurement data B30 as will be described later.

  As can be seen from FIG. 3, the measurement data B30 is obtained for a surface facing upward of one article, for example, the article A06a. The measurement data B30 is acquired for an article in another posture, for example, the article A06b that faces upward, which includes information on the face that is a blind spot in the article A06a. Therefore, the measurement data B30 acquired by measuring a plurality of articles A06a to A06h at the same time is regarded as including information obtained by measuring one article from various virtual measurement positions. Can do.

Here, the virtual measurement position will be described in more detail with reference to FIG. In FIG. 4, virtual measurement positions C06a to C06h are shown. These virtual measurement positions C06a to C06h are calculated when a reference coordinate system is provided on one article A06 based on the relative positional relationship between the articles A06a to A06h and the measurement positions shown in FIG. It corresponds to each of the virtual measurement positions to be performed.
In the present invention, the measurement data B30 of the articles A06a to A06h shown in FIG. 3 is obtained by measuring one article A06 from a plurality of virtual measurement positions C06a to C06h shown in FIG. Hereinafter, it is regarded as a set of model measurement data B13).

  Referring to FIG. 2 again, the model measurement data extraction unit B11 extracts the model measurement data B13 measured from one virtual measurement position C06 from the measurement data B30 as described later. Further, the model measurement data integration unit B12 integrates the model measurement data B13 based on a plurality of virtual measurement positions, thereby generating a three-dimensional model B40 of the article.

  FIG. 5 is a flowchart showing the operation of the handling robot apparatus. Hereinafter, with reference to FIG. 5, processing executed by the handling robot apparatus A00 shown in FIG. 1 will be described. When the handling robot apparatus A00 is started up, a three-dimensional model B40 is generated using the three-dimensional measuring machine A04 as shown on the left side of FIG.

  In step S101 in FIG. 5, a plurality of articles A06 are stacked in the measurement range of the three-dimensional measuring machine A04 as described above. At this time, it is preferable that the plurality of articles A06 have various postures. In such a case, since it can be determined that the measurement is the same as measuring one article A06 from various measurement positions, a highly reliable three-dimensional model can be generated. After the plurality of articles A06 are stacked, the process proceeds to step S102.

  In step S102, the surfaces of the plurality of articles A06 stacked in bulk are measured by the three-dimensional measuring machine A04. Measurement data B30 (see FIG. 3) acquired by the three-dimensional measuring machine A04 is held in the three-dimensional recognition apparatus A05. If acquisition of measurement data B30 is completed, it will progress to Step S103.

  In step S103, the drawing unit C11 outputs the measurement data B30 to the teaching screen C17. In this case, as shown in FIG. 3, it is preferable to display the measurement data B30 in an overlapping manner on the images of the plurality of articles A06a to A06h imaged by the three-dimensional measuring machine A04 or another imaging unit.

  Then, the operator sets a region surrounding one article from the plurality of A06a to A06h using the region setting unit C12. FIG. 3 shows a region surrounding the article A06a set by the region setting unit C12.

  Then, the temporary model generation unit C13 generates a temporary model D01 based on the measurement data B30 in the set area of the article A06a. In FIG. 6A, it is assumed that the temporary model D01 is generated from the model measurement data B13 measured from the virtual measurement position C06a corresponding to the article A06a. The black circles shown in FIGS. 6A to 6D are the model measurement data B13, and the set of black circles in FIGS. 6A to 6D is the temporary model D01.

  Here, the method for teaching the temporary model D01 is not limited. For example, a point obtained by projecting a three-dimensional point on a camera image may be plotted, and an effective three-dimensional point may be taught by area designation or mask teaching. When the teaching of the temporary model D01 is completed, the process proceeds to step S104.

  In step S104, the detection unit C14 collates the temporary model D01 with the measurement data B30 that has not been used in step S103, and detects the position and orientation of the remaining article A06, particularly the remaining article. For example, as illustrated in FIG. 6B, the detection unit C14 sequentially detects a virtual measurement position close to the virtual measurement position C06a, for example, the position and orientation of the article corresponding to the measurement positions C06b and C06h.

  If the detection unit C14 succeeds in detection in step S105, the process proceeds to step S106. In step S106, model measurement data B13 measured from a virtual measurement position C06 corresponding to the detection result is extracted based on the detection result. Specifically, first, the temporary model D01 at that time is subjected to coordinate conversion based on the position and orientation of the article calculated by detection.

  If there is a three-dimensional point in the measurement data B30 that is close to any three-dimensional point of the coordinate-converted temporary model D01, the three-dimensional point is extracted as new model measurement data B13. Specifically, the distance between each three-dimensional point in the temporary model D01 and each three-dimensional point in the measurement data B30 is calculated. If the calculated distance is smaller than the predetermined threshold, a three-dimensional point in the measurement data B30 is extracted as model measurement data.

  By the way, when there is an erroneous detection in the detection in step S104 and a 3D model is generated using the erroneous detection result, a final 3D model is erroneously generated. Therefore, the operator confirms that the detection result is correct and does not use the incorrect result by the detection success / failure instruction unit C15. Then, it is preferable to perform the processes of steps S106 and S107 using only the detection result guaranteed to be correct. Thereby, since an erroneous detection result can be excluded, it is possible to prevent the erroneous model measurement data B13 from being added to the three-dimensional model. If the extraction of the model measurement data B13 is completed, the process proceeds to step S107.

  In step S107, the extracted model measurement data B13 is integrated into the temporary model D01. In this regard, the newly extracted model measurement data B13 and temporary model D01 have overlapping portions. For this reason, if all newly extracted model measurement data B13 is integrated every time, the data of the final three-dimensional model becomes redundant and very heavy.

  For this reason, when generating a three-dimensional model, it is desirable to integrate so that there is no excess or deficiency of information. In order to achieve this, there are the following two methods.

  In the first method, when the 3D point of the temporary model D01 already exists in the vicinity of the 3D point of the newly extracted model measurement data B13, the 3D point is extracted from the newly extracted model measurement data B13. So that it is not integrated into the temporary model.

  In the second method, when the 3D point of the temporary model D01 already exists in the vicinity of the 3D point of the newly extracted model measurement data B13, the 3D point of the newly extracted model measurement data B13 and the temporary point One new three-dimensional point is calculated from the three-dimensional point of the model D01. For example, the new three-dimensional point may be the center of gravity of the three-dimensional point of the newly extracted model measurement data B13 and the three-dimensional point of the temporary model D01. In this case, it is possible to generate a model with reduced measurement errors.

  FIG. 7 is a diagram for explaining the removal of erroneous model measurement data. In FIG. 7, three three-dimensional points B30a to B30c are acquired as measurement data B30 for one article. When three three-dimensional points B30a to B30c form a triangular patch by a general triangular patch generation method, the three-dimensional measuring machine A04 and the three-dimensional points B30a to B30c are formed in a quadrangular pyramid. Can expect nothing to exist. For this reason, when the model measurement data B13a exists in the quadrangular pyramid, the model measurement data B13a can be removed by the unauthorized model measurement data removal unit C16 as the unauthorized model measurement data B13.

  Therefore, even when an incorrect three-dimensional point is extracted as the model measurement data B13, it is possible to remove the incorrect model measurement data B13 by confirming the consistency with the measurement data B30. For this reason, in the present invention, a correct three-dimensional model can be generated.

  It is also effective to teach a region in which model measurement data does not exist in advance on a teaching image displayed on a camera image or voxel. When the integration of the model measurement data is completed, the process returns to step S104. And the process of step S104-S107 is repeated using the updated temporary model D01.

  Here, measurement data far from the virtual measurement position C06a where the temporary model D01 shown in FIG. 6A is initially registered, for example, model measurement data measured from the virtual measurement position C06e shown in FIG. It is impossible to detect using the model D01 or its detection accuracy is low.

  For this reason, as shown in FIG. 6B, articles measured from virtual measurement positions close to the virtual measurement position C06a, for example, measurement positions C06b and C06h are sequentially detected. By repeating such an operation, the temporary model D01 is gradually expanded as can be seen from FIGS. 6B to 6C. And finally, as shown to FIG. 6D, the articles | goods measured from the virtual measurement position C06e are detected, and the model measurement data are integrated in step S107. Thereby, the temporary model D01 reflecting the model measurement data B13 of all the virtual measurement positions C06a to C06h can be created.

  In other words, in the present invention, the provisional model D01 is expanded by detecting in order from articles that can be detected accurately and stably. As a result, the model measurement data B13 can be finally extracted and integrated even on the surface of the article in which the model measurement data B13 does not exist in the initial temporary model.

  If the detection fails in step S105, the remaining article A06 does not exist, and the process proceeds to step S108. In step S108, the provisional model D01 after integration is optimized to generate a three-dimensional model B40.

  In the processing described so far, the estimation parameters used for generating the three-dimensional model are the virtual measurement position C06 (including posture information) and the position of each three-dimensional point belonging to the temporary model D01. In step S108, it is possible to generate a highly accurate three-dimensional model by using all these parameters as estimation targets and obtaining a solution that minimizes the total matching error (least square error).

  For example, an optimization method generally known as bundle adjustment can be applied. In bundle adjustment, nonlinear optimization is performed by numerical analysis, and the initial value at that time may be a virtual measurement position C06 grasped by prior detection and a three-dimensional point belonging to the provisional model D01. As described above, after the integration process of the model measurement data B13 into the temporary model D01 is completed, the process of optimizing the virtual measurement position and the position of each three-dimensional point belonging to the temporary model D01 is performed with higher accuracy. It becomes possible to generate the three-dimensional model B40.

  In this way, in the present invention, a three-dimensional model used for calculating the position and orientation of an article can be generated even for the article A06 for which CAD data does not exist. In the present invention, when a three-dimensional model of the article A06 is generated using the three-dimensional measuring machine A04, measurement data of a plurality of articles arranged in an arbitrary posture is used, and one article is converted into a plurality of virtual measurement positions. It is regarded as a set of model measurement data measured from For this reason, in the present invention, a three-dimensional model can be generated by a single measurement using the three-dimensional measuring machine A04.

  However, in another embodiment, the three-dimensional measurement may be performed a plurality of times. When performing the three-dimensional measurement again, after failing the detection in step S105, the process returns to step S101 without proceeding to step S108, the article is rearranged, and the subsequent processing is repeated.

  For example, when only a small number, for example, only two articles A06 can be arranged in the measurement range of the three-dimensional measuring machine A04, a three-dimensional model similar to that measured from various virtual measurement positions can be obtained by one-time three-dimensional measurement. It is difficult to generate. However, when the measurement is repeated a plurality of times after the two articles are rearranged, a three-dimensional model that can be acquired by measuring from various measurement positions can be similarly generated. Even in such a case, it is possible to generate a three-dimensional model in a shorter time than measurement while changing the posture of one article little by little.

  Here, it is difficult to extract model measurement data B13 by calculation from measurement data in which a plurality of articles whose three-dimensional models are unknown are measured. Further, when the operator teaches extraction of all model measurement data, the burden on the operator is too great. Therefore, in the present invention, as described above, the temporary model D01 is taught, the position and orientation of the remaining articles are detected using the temporary model D01, and the model measurement data B13 is extracted based on the detection result. Are integrated into the temporary model D01. For this reason, it is possible to generate a three-dimensional model without increasing the burden on the operator.

  Incidentally, the article transport process shown on the right side of FIG. 5 is started by, for example, operating an operation switch (not shown) and inputting a handling start command for the article A06. In step S201 in FIG. 5, three-dimensional measurement of a plurality of articles A06 arranged in a three-dimensional space is performed. This process is substantially the same as the process of step S102.

  Next, the process proceeds to step S202, and one article A06 is detected from the plurality of articles A06 arranged in the three-dimensional space. This process is substantially the same as the process of step S104. When the detection of one article A06 is completed, the process proceeds to step S203.

  Then, the article position / orientation calculation unit B20 collates the three-dimensional model B40 generated in advance by the model generation unit B10 with the measurement data B30 of the article A06 separately measured as described above, and the position / orientation of the article A06. B50 is calculated.

  Any method may be used for calculating the position and orientation of the article A06 by collating the three-dimensional model with the measurement data. For example, three-dimensional features such as PPF (Point Pair Feature) features and SHOT (Signature of Histogram of Orientation) feature values are calculated from model points of the three-dimensional model. Similarly, three-dimensional features are calculated for the measurement data. Then, a three-dimensional point having the same feature amount may be searched from the measurement data, and the position and orientation may be calculated therefrom.

  Alternatively, coordinate conversion is performed on a three-dimensional point constituting the three-dimensional model within a predetermined search position range or search posture range, and a difference from the three-dimensional point of the measurement data is obtained. If there is a three-dimensional point having a difference equal to or less than a predetermined value, it is counted that the three-dimensional points of such a three-dimensional model coincide. Then, this process may be performed on a three-dimensional point that can be expected to be measured in the three-dimensional model, and when the number of matches reaches a predetermined number or more, the position and orientation at that time may be output as a calculation result.

  In the present invention, a three-dimensional model is created using a three-dimensional measuring machine A04 that actually performs measurement, as compared with a position and orientation calculation apparatus that uses CAD data. For this reason, there is an advantage that wrinkles hardly occur between the three-dimensional model and the measurement data at the time of position / orientation calculation.

  Next, in step S203, the robot A01 is controlled, whereby the hand A03 of the robot A01 is moved to a position corresponding to the position and orientation of the article A06 detected in step S202. Then, the article A06 is held by the hand A03, and the robot A01 is caused to perform a predetermined handling operation. When the handling process of the article A06 is completed, the article transport operation is finished.

  As described above, according to the present invention, the position and orientation of the article A06 arranged in the three-dimensional space is calculated even for an article for which CAD data does not exist, and the article A06 is accurately handled based on the calculation result. it can. Further, since the three-dimensional model of the article A06 is generated by performing the three-dimensional measurement of the article A06, it is possible to greatly reduce the teaching effort when generating the three-dimensional model.

  Alternatively, in step S203, the reference robot position and orientation with respect to the reference article position and orientation, which is the reference article position and orientation, is stored in advance in the robot controller A02. Next, a robot position / posture that matches the relative positional relationship between the reference article position / posture and the reference robot position / posture with respect to the position / posture of the article A06 recognized in step S202 is obtained. Then, the robot A01 may be controlled to move the hand A03 to such a robot position and orientation.

  In the present embodiment, the system is terminated by gripping and transporting one detected article A06. However, the position and orientation of the plurality of articles A06 may be calculated by one detection, and the detected plurality of articles A06 may be continuously held and conveyed. Alternatively, all of a plurality of articles arranged in the three-dimensional space may be sequentially conveyed. When handling an article, as described in JP-A-2002-331480, the hand A03 may be prevented from interfering with a box in which the article A06 is stored.

  Although the present invention has been described using exemplary embodiments, those skilled in the art can make the above-described changes and various other changes, omissions, and additions without departing from the scope of the invention. You will understand.

A00 Handling robot device A01 Robot A02 Robot control device A03 Hand A04 3D measuring machine A05 3D recognition device A06, A06a to A06h Article A07 Mounting B00 Model generation device B10 Model generation unit B11 Model measurement data extraction unit B12 Model measurement data integration unit B13, B13a Model measurement data B20 Article position and orientation calculation unit (position and orientation calculation unit)
B30a-B30c 3D point (measurement data)
B40 Three-dimensional model B50 Article position and orientation C06, C06a to C06h Virtual measurement position C11 Drawing unit C12 Area setting unit C13 Temporary model generation unit C14 Detection unit C15 Detection success / failure instruction unit C16 Unauthorized model measurement data removal unit D01 Temporary model

Claims (12)

A model generation device (B00) for generating a three-dimensional model of an article,
A three-dimensional measuring machine (A04) that three-dimensionally measures a plurality of articles respectively arranged in an arbitrary posture in a three-dimensional space and acquires measurement data;
The measurement data of the plurality of articles measured by the three-dimensional measuring machine is regarded as a set of model measurement data obtained by measuring one article from a plurality of virtual measurement positions, and each of the plurality of virtual measurement positions is A model measurement data extraction unit (B11) for extracting corresponding model measurement data,
A model generation apparatus comprising: a model measurement data integration unit (B12) that generates a three-dimensional model of the article by integrating the model measurement data extracted by the model measurement data extraction unit. The model generation device according to claim 1,
The model measurement data extraction unit
A drawing unit (C11) for outputting the measurement data to a teaching screen;
An area setting unit (C12) for setting an area of one article selected from a plurality of articles on the teaching screen from which the measurement data is output;
A temporary model generation unit (C13) that generates a temporary model based on measurement data existing in the region set by the region setting unit;
A detection unit (C14) that detects the position and orientation of the remaining articles of the plurality of articles using the temporary model,
The model measurement data extraction unit extracts model measurement data based on the position and orientation of the remaining article detected by the detection unit,
The model measurement data integration unit generates a three-dimensional model by integrating the model measurement data into the temporary model. The model generation device according to claim 2,
The detection unit detects in order from an article in a posture close to the initial posture of the temporary model,
The model measurement data extraction unit extracts model measurement data in the order close to the initial posture of the temporary model based on the detection result of the detection unit,
The model measurement data integration unit integrates model measurement data into the temporary model sequentially each time the model measurement data is extracted.   4. The model generation apparatus according to claim 2, wherein the model measurement data extraction unit includes a detection success / failure instruction unit (C15) for an operator to instruct success / failure of the detection result of the detection unit. A model generation apparatus characterized by that.   5. The model generation device according to claim 2, wherein the model measurement data extraction unit is based on the measurement data existing in the vicinity of the position and orientation of the article detected by the detection unit. A model generation apparatus that extracts the model measurement data. The model generation device according to any one of claims 2 to 5,
Even if the model measurement data extraction unit illegally extracts incorrect measurement data as model measurement data and the model measurement data integration unit integrates the illegal model measurement data extracted illegally into the temporary model, The model measurement data integration unit includes an unauthorized model measurement data removal unit (C16) that extracts and removes the unauthorized model measurement data from the temporary model based on the consistency between the measurement data and the temporary model. A model generation device characterized by the above. The model generation device according to any one of claims 2 to 6,
When the model measurement data integration unit integrates the model measurement data into the provisional model, if there is an overlap between the model measurement data and the provisional model, the model measurement data is integrated with the overlap part. A model generation apparatus that integrates the model measurement data into the temporary model after removing the model. The model generation device according to any one of claims 2 to 6,
When the model measurement data integration unit integrates the model measurement data into the temporary model, if there is an overlap between the model measurement data and the temporary model, the overlap of the model measurement data A model generation apparatus that updates the overlapping portion of the temporary model based on the model.   9. The model generation apparatus according to claim 2, wherein the model measurement data integration unit calculates the virtual measurement position and the model measurement calculated from the position and orientation of the article detected by the detection unit. A model generation apparatus characterized in that optimization is performed using at least one of data as a parameter to improve the accuracy of a model to be generated. The model generation device according to any one of claims 1 to 9,
After the three-dimensional measuring device three-dimensionally measures a plurality of articles arranged in a three-dimensional space, changing the loading state of the articles and performing a new three-dimensional measurement of the plurality of articles at least once Repeatedly, the model measurement data extraction unit and the model measurement data integration unit generate a three-dimensional model of the article based on a plurality of measurement data obtained by a plurality of three-dimensional measurements. apparatus. The model generation device according to any one of claims 1 to 10,
A position and orientation calculation unit (B20) that calculates the position and orientation of the article based on the correspondence between the three-dimensional model generated by the model generation device and the measurement data acquired by the three-dimensional measuring machine;
A position and orientation calculation apparatus comprising: A robot (A01) capable of handling the article;
The model generation device according to any one of claims 1 to 10,
A position and orientation calculation unit (B20) that calculates the position and orientation of the article based on the correspondence between the three-dimensional model generated by the model generation device and the measurement data acquired by the three-dimensional measuring machine;
A handling robot apparatus (A00), wherein the robot handles the article based on the position and orientation of the article calculated by the position and orientation calculation unit.

Images