JPH06262568A - Recognition method for three-dimensional position and attitude based on visual sensation and device thereof - Google Patents

Abstract

PURPOSE:To make it possible to seek a parameter showing the position and attitude of a camera in a relatively simple and quick manner by constituting it from an image processing operation, a search area setting process, a through-vision transformation matrix operating process, a through-vision transformation operating process, an error operating process and a data outputting process. CONSTITUTION:This recognition method of a three-dimensional position-and-attitude based on visual sensation is featured that the image characteristic point of a work picked up by a camera 4 is operated by an image processing part 7. In succession, a space search area with a quantized space point set at the specified density by a quantization search part 10, is set by a search area setting part 11, and an estimate image characteristic point to lee securable at a time when the work is photographed by the camera 4 is operated for seeking a through-vision transformation matrix at the respective quantized space points, while a slippage with the image characteristic point is operated by an error evaluation function. Thus, when the specified request error accuracy is satisfied, the quantized space point and an attitude parameter at that time are outputted from a data output part 16 as the position and attitude of the camera 4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is based on vision for recognizing the three-dimensional position and orientation of a monocular camera based on image information of a known object taken by a monocular camera mounted on an autonomous mobile robot or the like. The present invention relates to a three-dimensional position / orientation recognition method and apparatus.

[0002]

2. Description of the Related Art In recent years, for example, an active operation such as an autonomous mobile robot which moves so as to approach a target object whose shape and absolute coordinate position are known and which grips the target object with an arm has been performed. There are things I tried to do. In such an autonomous mobile robot, the CCD
It is necessary to mount an image pickup means such as a camera and recognize the own three-dimensional relative position with respect to the target object based on the two-dimensional image information. In this case, in a device for position recognition, it is required to perform a highly accurate and quick calculation to obtain three-dimensional position data, and a configuration that can be realized at low cost is desired.

Therefore, conventionally, as a relatively simple one, for example, ones disclosed in JP-A-3-166072 or JP-A-3-166073 are considered. That is, in these, a special geometric shape (called a marker) whose shape and dimensions are fixed to the target device is photographed by a camera, and the center of gravity of the marker is determined based on the two-dimensional image information of the marker. The relative three-dimensional positional relationship of the camera with respect to the target device is obtained by a method such as calculation.

[0004]

However, according to the conventional method as described above, since it is necessary to provide the marker on the target device, it is necessary to make preparations for its installation each time, and at the same time, for the marker. Since it is necessary to always set the optical axis of the camera to be vertical, there is a problem that it is difficult to apply it to a camera that takes an arbitrary posture.

As a theoretical method capable of solving such a problem, in general, a method of back-calculating the parameters representing the position and orientation of the camera from the components of the perspective transformation matrix estimated by least-squares in the perspective n-point problem is a computer vision method. It exists as a method. According to this method, even if the relationship between the optical axis of the camera and the origin of the absolute coordinates that describe the target object is unknown, it is possible in principle to obtain parameters that represent the position and orientation of the camera. .

However, in such a computer vision method, when an explicit calculation is actually performed using a computer, the applied formula becomes very complicated and the amount of calculation is enormous. Therefore, rapid arithmetic processing becomes difficult and practically difficult to apply.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a camera even when the relationship between the optical axis of the camera and the origin of absolute coordinates describing a target object of a known shape is unknown. Based on the two-dimensional image information taken by
It is an object of the present invention to provide a method and apparatus for recognizing a three-dimensional position and orientation based on visual information, which makes it possible to relatively easily and quickly obtain parameters representing the position and orientation of a camera.

[0008]

A method of recognizing a three-dimensional position and orientation based on the present invention is based on image information when a target object whose shape has a known three-dimensional absolute coordinate is known. And an image processing step of extracting a two-dimensional image feature point indicating the feature of the shape of the target object, and a density corresponding to a calculation accuracy required according to a rough estimated distance between the image pickup means and the target object. A search area setting step of setting a search area having a quantized space point; and calculating a two-dimensional estimated image feature point to be obtained when the target object is photographed by the imaging means at the quantized space point of the search area. The perspective transformation matrix for performing the above-mentioned transformation is a posture parameter indicating the estimated position of the quantized space point with respect to the three-dimensional absolute coordinates of the target object stored in advance in the storage unit and the image pickup posture of the image pickup unit. Based on the perspective transformation matrix calculation step, and using the perspective transformation matrix obtained in this perspective transformation matrix calculation step, the two-dimensional estimated image feature points of the target object at the quantized space point The perspective transformation calculation step to be calculated, and the correspondence between the two-dimensional image feature points of the target object extracted in the image processing step and the two-dimensional estimated image feature points calculated in the perspective transformation calculation step. And an error calculation step of calculating an evaluation error value indicating the degree of conformity between the image feature points and the estimated image feature points for which the corresponding relationship is obtained, based on a predetermined error evaluation function,
The evaluation error value obtained in this error calculation step is compared with the required error accuracy set corresponding to the search area,
An error determination step of repeatedly executing the search area setting step, the perspective transformation matrix operation step, the perspective transformation operation step, and the error operation step under the condition that the evaluation error value is smaller than the required error accuracy. , When the determination condition is satisfied in this error determination step,
It is characterized in that it is composed of a three-dimensional absolute coordinate of the quantized space point at that time and a data output step of outputting the attitude parameter of the image pickup means.

Further, in the visual-based three-dimensional position and orientation recognition apparatus of the present invention, an image pickup means for outputting image information and a target object whose three-dimensional absolute coordinates indicating a shape are known are photographed by the image pickup means. Image processing means for extracting a two-dimensional image feature point indicating the shape feature of the target object based on the image information at the time, and necessary according to a rough estimated distance between the imaging means and the target object. Search area setting means for setting a search area having a quantized spatial point having a density corresponding to calculation accuracy, and two-dimensional to be obtained when the target object is imaged by the imaging means at the quantized spatial point of the search area. A perspective transformation matrix for calculating the estimated image feature points of the estimated position of the quantized space point with respect to the three-dimensional absolute coordinates of the target object stored in advance in the storage unit and the imaging Perspective transformation matrix computing means for computing based on a posture parameter indicating the imaging posture of the stage, and two-dimensional of the target object at the quantized space point using the perspective transformation matrix calculated by the perspective transformation matrix computing means A perspective transformation calculation means for calculating the estimated image feature points of
While obtaining the correspondence between the two-dimensional image feature points of the target object extracted by the image processing means and the two-dimensional estimated image feature points calculated by the perspective transformation calculation means, Error calculation means for calculating an evaluation error value indicating the matching degree between the obtained image feature point and the estimated image feature point based on a predetermined error evaluation function, and the evaluation error obtained by the error calculation means. The value is compared with the required error accuracy set corresponding to the search area, and the search area setting means and the perspective transformation matrix operation means are set on the condition that the evaluation error value is smaller than the required error accuracy. Error determining means for repeatedly performing the steps of the perspective transformation calculating means and the error calculating means, and the quantization space point at that time when the determination condition is satisfied in the error determining means Characterized in was composed of a data output means for outputting the three-dimensional absolute coordinate and the posture parameter of the imaging means.

[0010]

According to the vision-based three-dimensional position and orientation recognition method of the present invention, the three-dimensional absolute coordinates are known by the image pickup means mounted on the device or the like that moves and approaches the target object to perform work. When the target object is photographed, in the image processing step, a two-dimensional image feature point indicating the shape feature of the target object is extracted based on the image information. Subsequently, in the search area setting step, the search area having the quantized spatial points with the density corresponding to the required calculation accuracy is set according to the approximate estimated distance between the imaging means and the target object.

Next, in the perspective transformation matrix calculation step,
A perspective transformation matrix for calculating a two-dimensional estimated image feature point that should be obtained when the target object is photographed by arranging the image pickup unit at the quantized space point in the search area is stored in advance in the storage unit. The calculated position based on the estimated position of the quantized spatial point with respect to the three-dimensional absolute coordinates of the target object and the posture parameter indicating the image-capturing posture of the image-capturing means, and in the subsequent perspective transformation calculation step, is calculated as described above. The two-dimensional estimated image feature points of the target object at the quantized spatial points are calculated using the obtained perspective transformation matrix.
Thereby, when it is assumed that the target object is imaged by the imaging unit at the quantized space point of the set search area,
An image feature point obtained from the image information of the image pickup means can be obtained as an estimated image feature point.

Then, in a subsequent error calculation step, the two-dimensional image feature points of the target object extracted in the image processing step and the two-dimensional estimated image feature points calculated in the perspective transformation calculation step are The correspondence error is calculated, and an evaluation error value indicating the degree of matching between the image feature points for which the correspondence relation is obtained and the estimated image feature point is calculated using, for example, a sum of squared errors as a predetermined error evaluation function. .

Thereafter, in the error determining step, whether the evaluation error value obtained in the error calculating step is smaller than the required error accuracy set corresponding to the search area is satisfied. If it is not matched, the search area setting step, the perspective transformation matrix calculation step, the perspective transformation calculation step, and the error calculation step are repeatedly executed until it is matched.

In this case, the required error accuracy is in a relationship contradictory to the final calculation processing speed, and is required in consideration of the processing speed required according to the distance of the image pickup means to the target object at that time. By setting the value to any value, it is possible to execute the operation so as to satisfy both.

Finally, when the above conditions are met in the error determination step, the data output step outputs the three-dimensional absolute coordinates of the quantized space point at that time and the attitude parameter of the image pickup means. As a result, the quantized space point closest to the actual three-dimensional absolute coordinates of the image pickup means can be obtained as an approximate point, and the attitude parameter indicating the deviation of the optical axis of the image pickup means can be detected. Therefore, the movement control of the device with respect to the target object and the like can be appropriately performed based on these data.

Further, according to the visual-based three-dimensional position / orientation recognizing device of the present invention, the same operation can be performed by the means for carrying out the above-mentioned steps, and the three-dimensional position of the actual imaging means. It becomes possible to obtain the quantized space point closest to the absolute coordinates as an approximate point, and it becomes possible to detect the posture parameter representing the deviation of the optical axis of the image pickup means, and based on these data The movement control of the device can be performed appropriately.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a three-dimensional position recognition device mounted on an autonomous mobile robot running on a track in a factory or the like will be described below with reference to the drawings. In FIG. 2 showing the outline of the overall configuration, the autonomous mobile robot 1 is configured to move between the work areas 2 along a trajectory provided on a floor surface in a factory, for example, and the three-dimensional structure of the present invention is provided. A position recognition device (see FIG. 1) is installed. An arm 3 is provided on the top of the autonomous mobile robot 1, and a hand 6 for performing work such as gripping a camera 4 as an image pickup means and a work 5 as a target object is provided at the tip of the arm 3. Has been done. In this case, the work area 2 is provided in a recess formed in a state of being recessed from the wall surface, and the work 5 having a predetermined shape is placed at a predetermined position on the bottom surface 2 a of the work area 2.

Now, the three-dimensional position recognizing device is constructed as shown in FIG. Image processing unit 7 as image processing means
Is composed of a preprocessing calculation unit 8 and a feature point extraction unit 9. The preprocessing calculation unit 8 inputs an image signal of a scene including the work 5 photographed by the camera 4 and performs preprocessing, and the feature point extraction unit 9 based on the image information calculated by the preprocessing calculation unit 8. As will be described later, two-dimensional image feature points corresponding to the feature points of the work 5 are extracted.

The quantization space search unit 10 includes a search area setting unit 11 as a search area setting unit, a perspective transformation matrix calculation unit 12 as a perspective transformation matrix calculation unit, a perspective transformation calculation unit 13 as a perspective transformation calculation unit, and an error. The error calculation unit 14 as a calculation unit, the error determination unit 15 as an error determination unit, and the data output unit 16 as a data output unit.

In this case, the search area setting unit 11 performs a space search having a plurality of quantized space points E having a density based on the rough position information of the autonomous mobile robot 1 with respect to the work 5 in the work area 2 as a search area setting step. Area S is set. The perspective transformation matrix calculation unit 12 performs, as a perspective transformation matrix calculation step, image feature points in the image obtained from the image signal when the work 5 is photographed by the camera 4 at the quantization space point E of the set space search area S. The perspective transformation matrix M for obtaining the coordinates of is calculated according to the rough position information of the autonomous mobile robot 1 and the posture parameter of the camera 4. The perspective transformation calculation unit 13 uses, as a perspective transformation calculation step,
Using the obtained perspective transformation matrix M, a two-dimensional estimated image feature point to be obtained corresponding to the feature point when the work 5 is photographed by the camera 4 at each position of the quantized space point E is calculated. .

The error calculation section 14 performs the error calculation step as follows.
The image feature points extracted by the image processing unit 7 and the estimated image feature points are associated with each other, and the position error between them is calculated based on an error evaluation function for obtaining the sum of squared errors. The error determination unit 15 determines whether or not the calculated sum of squared errors is smaller than the required error accuracy ε as an error determination step, and when it is smaller, the position and orientation data of the camera 4 at that time is output to the output terminal 17. When it is larger, when it is larger, it returns to the search area setting unit 11 again to execute the calculation in each of the above units. Then, the data output unit 16 outputs the quantized spatial point E in the spatial search region S at that time based on the result of the above determination as a data output step.

Further, the quantization space search unit 10 has a memory 18 as a storage means for storing various data described later.
Are connected to each other so that various data can be written and read out, an external information input terminal 19 is provided, and a control unit (not shown) or a position detection sensor (not shown) that controls traveling of the autonomous mobile robot 1 is provided. A signal indicating the approximate position of the autonomous mobile robot 1 is input from. In this case, the data stored in advance in the memory 18 includes a feature point matching list, three-dimensional absolute coordinate data of the work 5 and the work area 2, parameters of the camera 4, a list of the quantization space points E and their coordinate data. is there.

Next, the operation of this embodiment will be described with reference to FIGS. 3 to 10. The three-dimensional position recognition device
As the flow of the operation, the existence position of the autonomous mobile robot 1 is recognized according to the steps shown in FIG. First, in the state of FIG. 2, when the autonomous mobile robot 1 moves on the orbit and approaches the work area 2, the camera 4
When work area 2 enters the shooting scene of
The image processing unit 7 inputs the image signal of the camera 4 (step T1). In this case, the autonomous mobile robot 1
Is moving in a predetermined direction on a predetermined trajectory, and a rough stop position is known from internal information of a control unit that controls the movement. Can be set within the field of view.

Next, the work 5 photographed by the camera 4
The image signal of is subjected to the noise removal processing (step T2) as the preprocessing and the image cutting processing (step T3) as the preprocessing in the preprocessing calculation portion 8 of the image processing portion 7, and then the feature point extraction is performed. Based on the image signal, the section 9 obtains and outputs a set Q of m image feature points Qj as follows (step T4). Here, the image feature point Q
The set Q of j is defined as follows. Q = {Qj; j = 1,2, ..., m} (1) Qj = (uj, vj) (2) u, v; Each element value of the two-dimensional coordinates in pixel units on the display screen is shown.

When extracting the image feature point Qj, the following calculation is performed based on the rough position information given from the input terminal 19 as external information. That is, as the rough position information, for example, a work plan in which the moving route of the autonomous mobile robot 1 is stored, the history information of the spatial movement moved by the traveling control, and the like are input, and therefore the autonomous mobile robot is based on these external information. As approximate three-dimensional absolute coordinates of the center position of 1, the approximate positions A0, A0 = (X0, Y0, Z0) (3) expressed in absolute coordinates can be obtained. The three-dimensional absolute coordinates of the rough position A0 represent a predetermined position of the work area 2 as the origin Op.

Further, the value of the rough position A0 is selected from the various appearance patterns (see FIGS. 4A to 4F) of the work 5 placed on the work area 2 stored in the memory 18 in advance. Select the appearance pattern corresponding to.
Now, for example, when the appearance pattern corresponding to the approximate position A0 corresponds to FIG. 4B, a set of three-dimensional feature points Pi of the work 5 appearing in the appearance pattern, P = {Pi I = 1, 2, ..., N} (4) The image feature points Qj corresponding to (4) are obtained and sequentially extracted. The extracted image feature points Qj are stored in the memory 18.

Now, in the quantization space search unit 10,
The search area setting unit 11 causes the approximate position A0 (X0, Y0
, Z0) as a reference, and spatial search areas Sc, Sc (c = 1, 2, ..., C) (5) are set (step T5). This space search area Sc
5, a plurality of quantized spatial points E (k, c), E (k, c) (k = 1, 2, ..., Quantized and arranged in a lattice at a predetermined density as shown in FIG. K, c = 1, 2, ..., C) (6).

In this case, the camera 4 faces the work 5 in the plane of the coordinate value X = XL1 in the X-axis direction (direction facing the work area 2) in the absolute coordinate system with the origin Op of the work area 2 as a reference. Can be assumed. Here, XL1 is an estimated value of the X coordinate value indicating the position of the viewpoint of the camera 4, and this value is calculated from the coordinate value X0 of the approximate position A0 of the autonomous mobile robot 1 in the X direction and the relative coordinates of the camera 4 on the robot. I can decide. Further, in the case of the autonomous mobile robot 1 in the present embodiment, it is assumed that the coordinate value Z0 of the approximate position A0 in the height direction does not deviate significantly from a fixed value, and therefore the coordinate of the camera 4 in the Z direction. The search range for obtaining can be narrowed down to a substantially narrow range.

Next, the search range of various parameters of the camera 4 is set (step T6). Here, as the parameters of the camera 4, as shown in the correspondence relationship in FIG. 6, (a) posture parameter CP (α, β, γ) of the camera 4, ... (7) (b) focal length characteristic of the camera 4 f (8) (c) There is a position (r, θ, φ) of the camera 4 (9). Then, in this case, the focal length characteristic f of (b) is determined in advance by the characteristic of the camera 4, so that it can be stored as data in the memory 18, for example. The value of the position (r, θ, φ) of the camera 4 in (c) is a value that can be determined in the subsequent calculation process.

Therefore, here, the search range of the posture parameter CP (α, β, γ) representing the deviation of the optical axis from the center position of the work 5 of the camera 4 may be determined.
That is, the search range of the posture parameter CP of the camera 4 is, for example, as shown in FIG.
Image position Ops of the origin Op as the predetermined position of the work 5
In the two-dimensional coordinates (u, v) in pixel units on the display screen of the image, (convert to a coordinate system having the included feature point as the origin if not included in the image information) is, for example, (Na , Nb), the estimated values of α and β can be calculated by the following equation.

Α = 2 × arctan {(Na / NH) × tan (wh / 2)} (10) β = 2 × arctan {(Nb / NV) × tan (wV / 2)} (11) , NH; the image size in the horizontal direction on the image display screen NV; the image size in the vertical direction on the image display screen wH; the horizontal view angle size of the camera wV; the vertical view angle size of the camera.

In this case, when the accuracy of the required error is not so high, a linear approximation is made by the following equation: α = (Na / NH) × wH (12) β = (Nb / NV) × wV (13) It can be substituted.

When γ = 0 is not satisfied, the work 5
An annular region Sa (Op including the image position Ops at the center of
Since it is considered that Opsc when γ is corrected exists in s), the quantized space point Ops in Sa (Ops) is Ops.
For c, equations (10), (11) or (12),
Based on (13), α and β are determined, and γ = γpsc (angle formed by Ops and Opsc) is set, and the subsequent search process is performed. In the present embodiment, the autonomous mobile robot 1 is capable of moving on a substantially horizontal floor and adjusting the rotation angle of the camera 4 with high accuracy using the coordinate value of the approximate position A0. Of the posture parameters (α, β, γ) for the optical axis shift of, the largest search capability is required for α, which represents the angular shift in the horizontal direction. Therefore, a small range may be searched for β and γ.

Next, in the perspective transformation matrix calculation unit 12, the quantized space point E is set as the initial search point in the space search area S1.
A quantization space point E (1,1), which is one of (k, 1) (k = 1,2, ..., K), is selected (step T7).
The three-dimensional absolute coordinates and the posture parameter CP of the camera 4
Using the initial values of (α, β, γ) and the perspective transformation matrix M (k, c) (k = 1, 2, ..., K, c = 1, 2, ..., C) (14) Calculate (step T8).

In this case, the perspective transformation matrix M is calculated as follows. That is, first, the origin Op of the work 5
3D absolute coordinates (X, Y, Z) set for
With reference to, the initial search area S1 is set within the plane of X = X0 which is the X coordinate value of the approximate position A0. A transformation matrix M1 for transforming the three-dimensional absolute coordinates (X, Y, Z) in the initial search area S1 into the three-dimensional polar coordinates (r, θ, φ) is obtained. Based on the posture parameter CP (α, β, γ) of the camera 4 and the viewpoint position of the camera 4 obtained from the focal length characteristic f and the distance d with respect to the projection plane, the perspective transformation matrix M based on the relationship shown in FIG. Find (k, c).

Here, the perspective transformation matrix M (k, c) can be expressed as follows. M (k, c) = T · M1 · M2 · M3 (15) where T: transformation matrix for rotation, parallel movement, enlargement, reduction, etc. (here, rotation represented by the posture parameter CP of the camera 4 Calculation) M1; Perspective transformation of the workpiece being photographed into the projective space M2; Perspective transformation from a plane in the projective space to the projective plane M3: Coordinate transformation from the projective plane to the display surface of the image The product of the perspective transformation matrices M1 and M2 and the coordinate transformation matrix M3 for calculating the perspective transformation between the surface and the work 5 are the following equations (16) and (17), respectively.
Can be expressed as

[0037]

[Equation 1]

Further, the transformation matrix T for calculating the rotation according to the posture parameter CP (α, β, γ) corresponding to the optical axis shift of the camera 4 has the relationship shown in FIG. 6 and FIG. It can be expressed by the following equations (18), (19) and (20). At this time, in the conversion of the optical axis shift, assuming that the distance to the work 5 does not change, it is possible to set T0 = T1. Therefore, the relationship of Expression (21) is established.

[0039]

[Equation 2]

Next, based on the perspective transformation matrix M (k, c) thus calculated, the perspective transformation calculation unit 13
N of the work area 2 and the work 5 placed there
Perspective transformation of each feature point Pi (see equation (4)) of the set P of individual feature points is performed on the coordinates of the two-dimensional image display unit,
An estimated image feature point QEi for each feature point Pi is obtained (step T9). A set QE of these estimated image feature points QEi
Is QE = {QEi; i = 1, 2, ..., N} (n ≧ m) (22)

Thus, using the homogeneous coordinates, the conversion from the set of feature points P to the set of estimated image feature points QE can be expressed by the following equation from equation (16). (UEi, vEi, 1) = (Xi, Yi, Zi, 1) · M (23) where (Xi, Yi, Zi); three-dimensional absolute coordinates of feature point Pi (uEi, vEi); estimated image feature It is a two-dimensional coordinate on the display of the point QEi. In this case, the quantization space point E
The perspective transformation matrix M (1,
1) Using estimated image feature point set P of feature point set P
Calculate E.

Next, each image feature point Qj of the set Q of image feature points of the work 5 photographed by the camera 4 extracted by the image processing unit 7 (see the equation (1)) and the work 5 stored in advance. A feature point correspondence list is created which has a correspondence relationship with the three-dimensional feature points Pi of (see equation (4)) (step T10). As shown in FIGS. 9A and 9B, this is the position of the feature point Pi of the work 5 and the quantization space point E.
Appearance pattern according to (1, 1) (see FIG. 4B)
9B, when the image feature points Qj are associated with each other, as shown in FIG. 9B, the image feature points Qj appearing on the display surface as image information and the feature points Pi corresponding to the image feature points Qj are shown in the following table.
Correspondence is obtained with.

[0043]

[Table 1]

In this case, there are cases where all the observed image feature points Qj corresponding to the feature point Pi do not exist, so that when the corresponding image feature point Qj exists, the "presence flag" is " It becomes 1 ”. In addition, the extraction order is sequentially from the upper left to the lower right on the image display unit.

Subsequently, the error calculating unit 14 calculates an error evaluation function between the image feature points Qj for which the correspondence is obtained and the above-described set of estimated image feature points QE (= P · M (1,1)). D
(1,1) is calculated (step T11). in this case,
The error evaluation function D (k, c) is for calculating a value corresponding to the distance between the image feature point Qj and the estimated image feature point QEi. Next, at the current quantized spatial point E (1,1), α and β are changed while leaving γ out of the posture parameters CP (α, β, γ), and the error evaluation function is performed in the same manner as described above. D (nE, 1) (nE = 1, 2, ..., NE) is calculated (steps T12, 13, 8 to 11).

Thereafter, these error evaluation functions D (nE,
From 1), DE1 (αE, βE, γE) is calculated as the optimum value (step T14). Here, usually, the determination of the optimum value is performed by using the sum of squared errors between QE and Q for all observed feature points as an evaluation function and performing the minimum value calculation. .

Subsequently, in the error judging section 15, the calculated value of the optimum value DE1 obtained from the calculated value of the error evaluation function D (k, 1) is the required error accuracy as a preset detection condition. For the value of ε, it is determined whether or not the determination condition of DE k <ε (24) is satisfied (step T).
15).

Then, when the above-mentioned judgment condition (24) is not satisfied, the above calculation is similarly executed for other quantized spatial points E (k, 1) in the initial search area S1 ( After steps T16 and T17, step T
7 to T15). In addition, another quantization space point E
When the judgment condition (24) is not satisfied for (k, 1), the quantization space point E (k, which gives the value closest to the required error accuracy ε from the optimum value DE k at that time is obtained. 1)
A new space search area S2 is set based on the position of (steps T18 and T19), and the above-described calculation process is executed again (steps T5 to T15).

On the other hand, when the condition of the judgment conditional expression (24) is satisfied (step T15), the data output unit 16 causes the quantized spatial point E corresponding to the optimum value DE k at that time.
The value of the position (rE, θE, φE) of (k, c) and the value of the posture parameter (αE, βE, γE) of the camera 4 are output to the output terminal 17 as detection data (step T2).
0), the search program ends.

If the determination condition expression (24) is not satisfied despite the setting of the final spatial search area SC (step T18), the quantized spatial point E satisfying the required error accuracy ε is obtained.
After outputting a message that (k, c) does not exist (step T21), the optimum value D calculated up to that point
The position (rE, θE, φE) of the quantized space point E (k, c) that gives the value closest to the required error accuracy ε from E k
Value and the pose parameters of the camera 4 (αE, βE, γ
The value E) is output to the output terminal 17 as detection data (step T20), and the search program is terminated.

In the above case, the quantized space point E (k, 1) satisfying the required error accuracy ε at a fixed distance (X = XL1) from the origin Op of the work 5 in the work area 2.
So that the spatial search area S is set to S1, S2,
Although the search process of setting ... Is shown, for example, FIG.
As shown in FIG. 5, when the autonomous mobile robot 1 moves closer to the work 5 (X <XL1), a search process for setting a space search area S (k, c) corresponding to the position is executed. You can also

That is, for example, in the course of progress of the autonomous mobile robot 1, one space search area S1 is set in correspondence with the distance X from the origin Op, and the quantized space point E (k, 1) in the space search area S1 is set. Of these, the one that is closest to the required error accuracy ε is obtained as provisional detection data indicating the position and orientation parameters of the camera 4, the movement amount of the autonomous mobile robot 1 is controlled based on the detection data, and the distance X from the origin Op is set. Is further reduced, a new space search area S2 is set, and the quantization space point E closest to the required error accuracy ε is set.
(K, 2) is obtained, and in the same manner, when the autonomous mobile robot 1 comes closest to the work 5, the search process is finally set so as to satisfy the required error accuracy ε. good.

According to the present embodiment as described above, the work area 2 and the work 5 as target objects of known shape are imaged by the camera 4, the image feature points Qj are extracted from the image information, and the quantized spatial points are extracted. Space search region S having a plurality of
c as well as the posture parameter CP of the camera 4
(Α, β, γ) is set and the estimated image feature point QE i of the work 5 to be obtained when the work 5 is imaged at the quantized space point E (k, c) is transformed into the perspective transformation matrix M (k,
c) is calculated and the correspondence between them is calculated, and it is determined whether or not the required error accuracy ε is satisfied by the error evaluation function D (k, c), and matching is performed to determine the position and orientation of the camera 4. Since the parameter CP is estimated, the following effects can be obtained.

That is, firstly, unlike the prior art, it is not necessary to provide a special marker, the moving time for marker recognition is unnecessary, and the calculation time can be shortened and the speed can be increased. Secondly, since it is not the method of calculating the parameters from the components of the perspective transformation matrix, the least square estimation of the perspective transformation matrix, which requires a large amount of computation, is not required, and thus the arithmetic processing can be executed quickly. Thirdly, it can be applied even when the posture parameter of the camera 4 is unknown. Fourthly, the accuracy of the calculation for obtaining the position of the camera 4 can be adjusted by the method of taking the quantized space point E. That is, for example, the accuracy can be set to a coarse accuracy at a long distance in consideration of the work content and the actual circumstances of the environment, and a fine accuracy at a close distance to the work 5, so that the accuracy can be set to an appropriate accuracy according to the situation. Fifth
In addition, since the calculation is performed in this way, when the camera 4 actively moves with respect to the work 5 as the target object,
From long-range optical axis alignment (coarse estimation) to high-precision estimation at short distances, it can be used in multiple stages in conjunction with a moving system.

[0055]

As described above, according to the visual-based three-dimensional position and posture recognition method of the present invention, the image processing step is used to image the target object whose three-dimensional absolute coordinates indicating the shape are known. A two-dimensional image feature point indicating the feature of the shape of the target object is extracted based on the image information when the image is captured, and a search region setting step, a perspective transformation matrix calculation step,
By the perspective transformation calculation step, the error calculation step, the error determination step and the data output step, a search area is set according to a rough estimated position of the image pickup means, and the image pickup means is set at a plurality of quantized spatial points in the search area. The error between the estimated image feature point and the image feature point that should be obtained when the target object is imaged is evaluated by an error evaluation function, and when the required error accuracy is satisfied, the quantized spatial point at that time is the third order of the image pickup means. Since it is estimated as the original absolute coordinate position and the posture parameter of the image pickup unit is output as the posture, the image pickup unit can be swiftly and highly accurately without using a special marker even when the optical axis of the image pickup unit is deviated. The position and orientation of the robot can be recognized, and the balance between the calculation speed and the calculation accuracy can be set appropriately according to the distance by setting the required error accuracy. You can be one in which an excellent effect that a like movement control device for the target object can be performed accurately based on these data.

Further, according to the visual-based three-dimensional position / orientation recognizing apparatus of the present invention, the same operation can be performed by the means for executing the above-mentioned steps, and the three-dimensional position of the actual image pickup means can be obtained. It becomes possible to obtain the quantized space point closest to the absolute coordinates as an approximate point, and it becomes possible to detect the posture parameter representing the deviation of the optical axis of the image pickup means, and based on these data It has an excellent effect that the movement control of the device can be performed accurately.

[Brief description of drawings]

FIG. 1 is a block diagram of a functional configuration showing an embodiment of the present invention.

FIG. 2 is a diagram showing a positional relationship between an autonomous mobile robot and a work area.

FIG. 3 is a flowchart illustrating a process flow.

[Fig. 4] Various appearance patterns

FIG. 5 is an operation explanatory view showing a positional relationship between a space search area and a work area.

FIG. 6 is a diagram showing a relationship between absolute coordinate axes and camera and image coordinate axes.

FIG. 7 is an explanatory diagram showing a relationship between a work on the image display unit and a posture parameter of a camera.

FIG. 8 is an operation explanatory view showing a relationship of coordinate axes for calculating a perspective transformation matrix.

FIG. 9 is an operation explanatory view showing a correspondence relationship between (a) definition of feature points of a work area and a work, and (b) image feature points by image signals.

FIG. 10 is an operation explanatory view when the search area is set according to the distance to the work.

[Explanation of symbols]

1 is an autonomous mobile robot, 2 is a work area, 4 is a camera (imaging means), 5 is a work (target object), 7 is an image processing unit, 8 is a preprocessing calculation unit, 9 is a feature point extraction unit, and 10 is quantum. Generalized space search unit, 11 a search region setting unit (search region setting unit), 12 perspective transformation matrix calculation unit (perspective transformation matrix calculation unit), 13 perspective transformation calculation unit (perspective transformation calculation unit), 1
Reference numeral 4 is an error calculation unit (error calculation unit), 15 is an error determination unit (error determination unit), 16 is a data output unit (data output unit), 18 is a memory (storage unit), and 19 is an external information input terminal.

Claims (2)

[Claims] 1. An image for extracting a two-dimensional image feature point indicating a feature of a shape of a target object based on image information when the target object whose shape has a known three-dimensional absolute coordinate is photographed by an image pickup means. A processing step; a search area setting step of setting a search area having a quantized spatial point having a density corresponding to a necessary calculation accuracy in accordance with a rough estimated distance between the imaging means and the target object; A perspective transformation matrix for calculating a two-dimensional estimated image feature point to be obtained when the target object is photographed by the image pickup means at the quantized spatial point of the third order of the target object stored in advance in the storage means. A perspective transformation matrix computing step of computing based on the estimated position of the quantized spatial point with respect to the original absolute coordinates and a posture parameter indicating the image pickup posture of the image pickup means; And a perspective transformation calculation step of calculating the two-dimensional estimated image feature points of the target object in the quantized space point using the perspective transformation matrix obtained in The correspondence between the two-dimensional image feature points of the target object and the two-dimensional estimated image feature points calculated in the perspective transformation calculation step is obtained, and the image feature points and the estimated image features for which the corresponding relations are obtained. An error calculation step of calculating an evaluation error value indicating a goodness of fit between the points based on a predetermined error evaluation function, and the evaluation error value obtained in the error calculation step is set corresponding to the search area. Compared with the required error accuracy,
An error determination step of repeatedly executing the search area setting step, the perspective transformation matrix operation step, the perspective transformation operation step, and the error operation step under the condition that the evaluation error value is smaller than the required error accuracy. A three-dimensional position based on vision, which comprises a data output step of outputting the three-dimensional absolute coordinates of the quantized space point at that time and the attitude parameter of the imaging means when the determination condition is satisfied in this error determination step. And attitude recognition method. 2. An image pickup means for outputting image information, and a feature of the shape of the target object based on image information when the image of the target object whose three-dimensional absolute coordinates showing the shape are known is picked up by the image pickup means. An image processing unit for extracting a two-dimensional image feature point, and a search region having a quantized spatial point having a density corresponding to a required calculation accuracy according to a rough estimated distance between the image pickup unit and the target object are set. And a perspective transformation matrix for calculating a two-dimensional estimated image feature point that should be obtained when the target object is photographed by the imaging unit at the quantized space point of the search region. Perspective transformation matrix calculated based on the estimated position of the quantized spatial point with respect to the three-dimensional absolute coordinates of the target object stored in the means, and an attitude parameter indicating the imaging attitude of the imaging means. Computing means, perspective transformation computing means for computing the two-dimensional estimated image feature points of the target object at the quantized spatial points using the perspective transformation matrix obtained by the perspective transformation matrix computing means, and the image The correspondence relationship between the two-dimensional image feature points of the target object extracted by the processing means and the two-dimensional estimated image feature points calculated by the perspective transformation calculation means is obtained, and the correspondence relationships are obtained. Error calculation means for calculating an evaluation error value indicating the degree of conformity between the image feature point and the estimated image feature point based on a predetermined error evaluation function, and the evaluation error value obtained by the error calculation means. Compared with the required error accuracy set corresponding to the search area,
An error determination for repeatedly executing the steps of the search region setting means, the perspective transformation matrix computing means, the perspective transformation computing means, and the error computing means under the condition that the evaluation error value is smaller than the required error accuracy. Means based on the visual sense, and three-dimensional absolute coordinates based on the visual sense, which output the three-dimensional absolute coordinates of the quantized spatial point at that time and the orientation parameter of the image pickup means when the determination condition is satisfied in the error determination means. Original position and posture recognition device.