JP6832694B2 - Database generator and 3D position / orientation recognition system - Google Patents

Description

The present invention relates to a database generator that generates a database showing posture data, and a three-dimensional position / posture recognition system that estimates the three-dimensional posture of an object using the database.

Conventionally, when picking a bulked object with a robot, the three-dimensional position (x, y, z) and the three-dimensional posture (α, β, γ) of the object are taken from the image of the bulked object. ) Needs to be recognized by estimating. Examples of the object recognition method include a recognition method based on two-dimensional matching (appearance base) and a recognition method based on three-dimensional matching (model base).
As a device using the model base, for example, there is a three-dimensional position / orientation recognition device disclosed in Patent Document 1 (see, for example, Patent Document 1). When this three-dimensional position / orientation recognition device estimates the three-dimensional attitude of an object, the above image and the three-dimensional model are compared while changing the attitude data of the three-dimensional model. The posture of the three-dimensional model indicated by the posture data having a high degree of coincidence is defined as the three-dimensional posture of the object.

Japanese Unexamined Patent Publication No. 2010-210511

However, in the conventional three-dimensional position / orientation recognition device, it is necessary to compare the image with a huge amount of attitude data, and there is a problem that the number of calculations increases. The same applies when using the appearance base.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a database generation device capable of narrowing down posture data with respect to a conventional configuration.

The database generator according to the present invention has a basic posture estimation unit that estimates the three-dimensional posture of the object when the object is dropped alone, and a three-dimensional posture of the object when the objects are stacked separately. Based on the statistical data acquisition unit that acquires statistical data and the three-dimensional attitude estimated by the basic posture estimation unit, the probability of occurrence of the three-dimensional attitude of the object when it is stacked separately from the statistical data acquired by the statistical data acquisition unit. Based on the distribution calculation unit that calculates the probability distribution showing, and the probability distribution calculated by the distribution calculation unit, the attitude data of the three-dimensional posture that can occur when the objects are stacked separately is extracted, and the three-dimensional of the object. It is characterized by including a database generation unit that generates a database showing the attitude data from the model.

According to the present invention, since the configuration is as described above, the posture data can be narrowed down with respect to the conventional configuration.

It is a block diagram which shows the structural example of the 3D position attitude recognition system which concerns on Embodiment 1 of this invention. It is a figure which shows the hardware configuration example of the 3D position attitude recognition system which concerns on Embodiment 1 of this invention. It is a figure which shows the coordinate system used in the 3D position attitude recognition system which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the dimension and width dimension of the container used in the 3D position attitude recognition system which concerns on Embodiment 1 of this invention. It is a flowchart which shows the operation example of the database generator which concerns on Embodiment 1 of this invention. It is a figure explaining the operation by the basic posture estimation part in Embodiment 1 of this invention. It is a figure which shows an example of the estimation result by the basic posture estimation part in Embodiment 1 of this invention. It is a figure explaining the operation by the statistical data acquisition part in Embodiment 1 of this invention. It is a figure explaining the operation by the distribution calculation part in Embodiment 1 of this invention. It is a figure which shows an example of the calculation result by the distribution calculation part in Embodiment 1 of this invention. It is a figure explaining the operation of the 3D position posture recognition apparatus which concerns on Embodiment 1 of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing a configuration example of a three-dimensional position / orientation recognition system according to the first embodiment of the present invention, and FIG. 2 is a diagram showing a hardware configuration example of the three-dimensional position / orientation recognition system.
As shown in FIG. 1, the three-dimensional position / orientation recognition system includes a database generation device 1, a photographing device 2, and a three-dimensional position / orientation recognition device 3. As shown in FIG. 2, the database generation device 1 and the three-dimensional position / orientation recognition device 3 are realized by a computer 4 equipped with a processing circuit such as a system LSI and a CPU or the like that executes a program stored in a memory or the like. Will be done.

The database generation device 1 generates a database related to the posture data used in the three-dimensional position / posture recognition device 3. At this time, the database generation device 1 narrows down the posture data used in the three-dimensional position / posture recognition device 3 with respect to the conventional configuration. The database generated by the database generation device 1 has data indicating an image holding the posture data when the three-dimensional position / orientation recognition device 3 recognizes the object 50 on an appearance basis, and the object is based on a model. When recognizing 50, it is set to have data indicating a three-dimensional model holding posture data. The database generation device 1 generates the database before the three-dimensional position / orientation recognition device 3 performs the processing. The details of the database generation device 1 will be described later.

As shown in FIG. 2, the photographing device 2 is installed at a position where the objects 50 piled up in bulk can be photographed, and photographs are taken. The photographing device 2 acquires a two-dimensional image when the three-dimensional position / orientation recognition device 3 recognizes the object 50 on an appearance basis, and obtains depth information when the object 50 is recognized on a model basis. It is configured to acquire a distance image held by a pixel.
The distance image is a known light cutting method (a method of irradiating an object with slit light and photographing the reflected light to measure the distance by triangulation) and a Time-of-Flight method (distance from the flight time of light). It can be obtained by using a method for measuring) or a measurement method by triangulation using a stereo camera.

The three-dimensional position / orientation recognition device 3 uses the database generated by the database generation device 1 and estimates the three-dimensional position and the three-dimensional posture of the object 50 from the image taken by the photographing device 2. Recognize 50. The method of recognizing the object 50 by the three-dimensional position / orientation recognition device 3 may be either appearance-based or model-based. The three-dimensional position / orientation recognition device 3 includes a database storage unit 31, an image acquisition unit 32, and a comparison calculation unit 33.

The database storage unit 31 stores the database generated by the database generation device 1.

The image acquisition unit 32 acquires data indicating an image captured by the photographing device 2.
The center of the image obtained by the image acquisition unit 32 is set as the origin, the horizontal direction of the image is set as the x-axis, and the vertical direction is set as the y-axis. Further, the vertical direction with respect to the reference plane is the z-axis, and the coordinate system is the reference coordinate system.

The comparison calculation unit 33 compares the image obtained by the image acquisition unit 32 with the database stored in the database storage unit 31 to compare the three-dimensional position (x, y, z) and the three-dimensional posture of the object 50. Estimate (α, β, γ).
As shown in FIG. 3, the three-dimensional position (x, y, z) of the object 50 is the three-dimensional position in the reference coordinate system. The three-dimensional posture (α, β, γ) of the object 50 is the roll angle around the x-axis, the pitch angle around the y-axis, and the yaw angle around the z-axis at the three-dimensional position (x, y, z). is there. This three-dimensional posture (α, β, γ) can be replaced by Euler angles and the like. The method of estimating the three-dimensional position and the three-dimensional posture by the comparison calculation unit 33 is the same as that of the conventional configuration.

As shown in FIG. 1, the database generation device 1 includes a three-dimensional model storage unit 11, a bulk stacking condition storage unit 12, a basic posture estimation unit 13, a statistical data acquisition unit 14, a distribution calculation unit 15, and a database generation unit 16. There is.

The three-dimensional model storage unit 11 stores data indicating a three-dimensional model of the object 50. Here, as the data showing the three-dimensional model, for example, CAD data can be mentioned.

The bulk loading condition storage unit 12 stores data indicating conditions (bulk loading conditions) relating to the bulk loading of the object 50. The bulk picking conditions are the dimensions (W, D, H) and width dimensions (dW, dD, dH) of the container 51 for bulk picking the objects 50 as shown in FIG. 4, and the number of the objects 50 to be bulked. Assuming that the inner dimension of the container 50 in the x-axis direction is Win, dW = (W-Win) / 2. Further, assuming that the inner dimension of the container 50 in the y-axis direction is Din, dD = (D-Din) / 2. Further, assuming that the inner dimension of the container 50 in the z-axis direction is Hin, dH = (H—Hin) / 2. Moreover, the number may be the number at the time of bulk picking.

The basic posture estimation unit 13 estimates the three-dimensional posture (hereinafter, the basic three-dimensional posture) of the object 50 when the object 50 is dropped by itself. The basic three-dimensional posture is a three-dimensional posture that the object 50 can take on a horizontal plane when the object 50 is dropped by itself. At this time, the basic posture estimation unit 13 first obtains statistical data of the three-dimensional posture of the object 50 by simulation using a physics engine or the like based on the three-dimensional model stored in the three-dimensional model storage unit 11. get. Then, the basic posture estimation unit 13 estimates the basic three-dimensional posture from this statistical data.

The statistical data acquisition unit 14 acquires statistical data of the three-dimensional posture of the object 50 when the objects 50 are piled up in bulk. At this time, the statistical data acquisition unit 14 is based on the three-dimensional model stored in the three-dimensional model storage unit 11 and the bulk stacking condition stored in the bulk stacking condition storage unit 12, and is subjected to the above-mentioned target by simulation using a physics engine or the like. The statistical data of the three-dimensional posture of the object 50 is acquired.

The distribution calculation unit 15 is based on the basic three-dimensional posture estimated by the basic posture estimation unit 13, and the probability of occurrence of the three-dimensional posture of the object 50 when the objects 50 are stacked separately from the statistical data acquired by the statistical data acquisition unit 14. Calculate the probability distribution that indicates. At this time, the distribution calculation unit 15 obtains the probability distribution by calculating a mixed Gaussian distribution consisting of the same number of Gaussian distributions as the number of patterns of the basic three-dimensional posture by averaging the basic three-dimensional postures (mixing). Modeled by Gaussian distribution).

The database generation unit 16 extracts attitude data from the three-dimensional model stored in the three-dimensional model storage unit 11 based on the probability distribution calculated by the distribution calculation unit 15, and generates a database showing the attitude data. At this time, the database generation unit 16 extracts, for example, the attitude data of the three-dimensional postures belonging to a certain range centered on the average of each Gaussian distribution constituting the mixed Gaussian distribution which is the probability distribution from the three-dimensional model.

Next, an operation example of the database generation device 1 according to the first embodiment will be described with reference to FIG. It is assumed that the three-dimensional model storage unit 11 stores data indicating the three-dimensional model of the object 50, and the bulk stacking condition storage unit 12 stores data indicating the bulk stacking conditions.
In the operation example of the database generation device 1, as shown in FIG. 2, first, the basic posture estimation unit 13 is simulated by using a physics engine or the like based on the three-dimensional model stored in the three-dimensional model storage unit 11. , The statistical data of the three-dimensional posture of the object 50 when the object 50 is dropped by itself is acquired (step ST1). That is, the basic attitude estimation unit 13 performs a fall simulation as shown in FIG. 6A. As a result, the basic posture estimation unit 13 can obtain the three-dimensional posture of the object 50 as shown in FIGS. 6B and 6C, and by trying this a plurality of times, statistical data can be obtained. The number of simulation trials n can be obtained from the following equation (1) from the response ratio p, the error rate d (%), and the reliability level λ (%).

Further, as shown in FIG. 3, when the roll angle, pitch angle, and yaw angle are used as the three-dimensional postures (α, β, γ), if the roll angle and the pitch angle are equal, the yaw angle (around the axis of the photographing apparatus 2) is used. Even if the angles) are different, they can be regarded as the same three-dimensional posture. Therefore, here, it is assumed that statistics are taken only for the roll angle and the pitch angle. FIG. 7 shows the statistical data acquired by the basic posture estimation unit 13.

Next, the basic posture estimation unit 13 estimates the basic three-dimensional posture from the acquired statistical data (step ST2). When the object 50 is dropped by itself, a peak is obtained at a specific location as shown in FIG. 7. Then, the basic posture estimation unit 13 estimates the three-dimensional posture of this peak as the basic three-dimensional posture.

Further, the statistical data acquisition unit 14 is based on the three-dimensional model stored in the three-dimensional model storage unit 11 and the bulk stacking condition stored in the bulk stacking condition storage unit 12, and the object 50 is simulated by using a physics engine or the like. The statistical data of the three-dimensional posture of the object 50 when the objects are stacked in bulk is acquired (step ST3). That is, the statistical data acquisition unit 14 performs a bulk stack simulation as shown in FIG. As a result, the statistical data acquisition unit 14 can obtain the three-dimensional posture of the object 50, and obtains statistical data by trying this a plurality of times. The number of simulation trials n is the same as above. Also in this case as well, statistics are taken only for the roll angle and the pitch angle.

Next, the distribution calculation unit 15 is based on the basic three-dimensional posture estimated by the basic posture estimation unit 13, and from the statistical data acquired by the statistical data acquisition unit 14, the three-dimensional posture of the object 50 when separately stacked. The probability distribution indicating the probability of occurrence is calculated (step ST4). FIG. 9 is a diagram illustrating distribution calculation by the distribution calculation unit 15. The scatter plot shown in FIG. 9 shows the statistical data acquired by the statistical data acquisition unit 14. Further, reference numeral 901 indicates the position of the basic three-dimensional posture estimated by the basic posture estimation unit 13. Then, the distribution calculation unit 15 obtains the above probability distribution by calculating a mixed Gaussian distribution having the same number of Gaussian distributions as the number of patterns of the basic three-dimensional posture 901 by averaging the basic three-dimensional posture 901. In the example of FIG. 9, since there are two basic three-dimensional postures 901, the distribution calculation unit 15 obtains a mixed Gaussian distribution consisting of two Gaussian distributions obtained by averaging each basic three-dimensional posture 901. .. FIG. 10 shows the probability distribution calculated by the distribution calculation unit 15.

Here, the distribution calculation unit 15 uses a maximum likelihood estimation method such as an EM algorithm as a modeling method based on a mixed Gaussian distribution. The three-dimensional postures that the object 50 can take when the objects 50 are piled up in bulk can be expressed by adding rotation to the basic three-dimensional postures, except for some special postures. Here, the special posture is a three-dimensional posture that does not normally occur, and is a three-dimensional posture that occurs exceptionally only under specific conditions when piled up in bulk. Further, even if the object 50 in bulk has a special posture, the object 50 in bulk is picked by the robot from the state in which the object 50 is in bulk, so that the object 50 in bulk becomes a more stable posture. It can be assumed that it approaches, that is, it approaches the basic three-dimensional posture. Therefore, such a special three-dimensional posture can be ignored when modeling. Therefore, the number of Gaussian distributions constituting the mixed Gaussian distribution used in modeling is the number of patterns of the basic three-dimensional posture, and the average is the basic three-dimensional posture.

Next, the database generation unit 16 extracts attitude data from the three-dimensional model stored in the three-dimensional model storage unit 11 based on the probability distribution calculated by the distribution calculation unit 15, and generates a database showing the attitude data. (Step ST5). At this time, the database generation unit 16 is, for example, three-dimensional belonging to a certain range (for example, within the range of reference numeral 902 shown in FIG. 9) centered on the average of each Gaussian distribution constituting the mixed Gaussian distribution which is the probability distribution. The posture data of the posture is extracted from the three-dimensional model. As a result, the posture data can be narrowed down.

FIG. 11 is a diagram illustrating the operation of the three-dimensional position / orientation recognition device 3. As shown in FIG. 11, in the three-dimensional position / orientation recognition device 3, when estimating the three-dimensional attitude of the object 50, the image 1001 obtained by the image acquisition unit 32 is stored in the database storage unit 31. Compare with 1002.
Here, conventionally, it is necessary to compare the image 1001 with all the posture data of the three-dimensional model. On the other hand, in the first embodiment, the database generator 1 removes unnecessary posture data 1003 (which cannot normally occur) in advance. Therefore, in the three-dimensional position / orientation recognition device 3, the image 1001 may be compared with less attitude data than the conventional configuration. Therefore, the number of operations can be reduced and the processing time can be shortened.
Further, in the first embodiment, the database generation device 1 removes unnecessary posture data 1003 in advance. Therefore, since the three-dimensional position / orientation recognition device 3 does not use this unnecessary posture data 1003, erroneous recognition can be reduced.

As described above, according to the first embodiment, when the basic posture estimation unit 13 that estimates the three-dimensional posture of the object 50 when the object 50 is dropped by itself and the object 50 are stacked separately. From the statistical data acquisition unit 14 that acquires the statistical data of the three-dimensional posture of the object 50, and the statistical data acquired by the statistical data acquisition unit 14 based on the three-dimensional attitude estimated by the basic posture estimation unit 13. Attitude data from the three-dimensional model of the object 50 based on the distribution calculation unit 15 that calculates the probability distribution showing the probability of occurrence of the three-dimensional posture of the object 50 when stacked in bulk and the probability distribution calculated by the distribution calculation unit 15. Is provided, and a database generation unit 16 for generating a database showing the attitude data is provided, so that the attitude data can be narrowed down with respect to the conventional configuration. As a result, in estimating the three-dimensional posture of the object 50 in the three-dimensional position / posture recognition device 3, the number of calculations can be reduced and the processing time can be shortened.

In the above, the case where the database generation device 1 and the three-dimensional position / orientation recognition device 3 are realized by a common computer 4 is shown. However, the present invention is not limited to this, and the database generation device 1 and the three-dimensional position / orientation recognition device 3 may be realized by different computers.

Further, in the above, the case where the three-dimensional model storage unit 11 for storing the data indicating the three-dimensional model is provided inside the database generation device 1 is shown. However, the present invention is not limited to this, and the database generation device 1 may not be provided with the three-dimensional model storage unit 11, and the database generation device 1 may be configured to acquire data indicating the three-dimensional model from the outside. The same applies to the data indicating the bulk loading conditions.

Further, in the above, the case where the basic posture estimation unit 13 acquires the statistical data of the three-dimensional posture by simulation using the three-dimensional model and estimates the basic three-dimensional posture from the statistical data is shown. However, the present invention is not limited to this, as long as the basic posture estimation unit 13 can acquire statistical data, it may be performed experimentally instead of simulation. The same applies to the statistical data acquisition unit 14.

In the present invention, within the scope of the invention, it is possible to modify any component of the embodiment or omit any component of the embodiment.

1 Database generation device 2 Imaging device 3 Three-dimensional position / orientation recognition device 4 Computer 11 Three-dimensional model storage unit 12 Bulk condition storage unit 13 Basic posture estimation unit 14 Statistical data acquisition unit 15 Distribution calculation unit 16 Database generation unit 31 Database storage unit 32 Image acquisition unit 33 Comparison calculation unit 50 Object

Claims (6)

A basic posture estimation unit that estimates the three-dimensional posture of the object when the object is dropped by itself,
A statistical data acquisition unit that acquires statistical data of the three-dimensional posture of the object when the objects are piled up in bulk,
Based on the three-dimensional posture estimated by the basic posture estimation unit, a distribution that calculates a probability distribution indicating the probability of occurrence of the three-dimensional posture of the object when the objects are stacked separately from the statistical data acquired by the statistical data acquisition unit. Calculation department and
Based on the probability distribution calculated by the distribution calculation unit, the posture data of the three-dimensional posture that can occur when the objects are stacked in bulk is extracted, and a database showing the posture data is generated from the three-dimensional model of the target. A database generator with a database generator. The basic posture estimation unit acquires statistical data of the three-dimensional posture of the object by simulation based on the three-dimensional model, and estimates the three-dimensional posture of the object from the statistical data. Item 1. The database generator according to item 1. The database generation device according to claim 1 or 2, wherein the statistical data acquisition unit acquires statistical data of the three-dimensional posture of the object by simulation based on the three-dimensional model and bulk stacking conditions. The distribution calculation unit obtains the probability distribution by calculating a mixed Gaussian distribution consisting of the same number of Gaussian distributions as the number of patterns of the three-dimensional posture by averaging the three-dimensional postures estimated by the basic posture estimation unit. The database generator according to any one of claims 1 to 3, wherein Wherein the database generation unit according to claim 4 database generating device, wherein the to extract the attitude data of the three-dimensional position falling within a predetermined range around the mean of each Gaussian distribution constituting the mixed Gaussian distribution .. The database generator according to any one of claims 1 to 5.
An image acquisition unit that acquires data indicating an image,
A three-dimensional position / orientation equipped with a three-dimensional position / orientation estimation unit that estimates the three-dimensional attitude of the object by comparing the image obtained by the image acquisition unit with the database generated by the database generator. Recognition system.

Images