JP3300092B2 - Image feature extraction device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image feature extracting device.

[0002]

2. Description of the Related Art In recent years, techniques relating to image processing have been applied in various fields for the purpose of automation. Techniques related to such image processing are disclosed in the following documents.

Reference 1: D.A. H. Ballard, "Generalized Hough Transform for Detecting Arbitrary Shapes" (DHBallard, "Ge
neralized Hough transform to detect arbitrary shap
es ″, Pattern Recognition, Vol.13, No.2,1981, pp.111-1
twenty two. Reference 2: Kosaka et al., "Navigation of Mobile Robots Using Visual Inference with Model Reasoning and Uncertainty Prediction" (A.Ko
saka and ACKak, ″ Fast vision-guidedmobile robot
navigation using model-based reasoning and predi
ction ofuncertainties ″, Computer Vision, Graphics,
and Image Processing--Image Understanding, Novembe
r 1992, 271-329) Reference 3: R.I. M. Haralick et al., "Computer and Robot Vision" (RMHaralick and LGShapiro, Com)
puter and Robot Vision, AddisonWesley, 1992, pp. 371-4
52.) Reference 4: O.M. I. Campus et al., "Object Recognition Using Prediction and Probabilistic Matching" (OICamps, LGShapiro, and
RMHaralick, ″ Object Recognition usingPrediction
and Probabilistic Matching ″ Proceedings of the 19
92IEEE International Conference on Intelligent Rob
ots and Systems, Raleigh, NC, 1992, pp. 1044-1052. 2. Description of the Related Art When an object is recognized using an image processing technique, analysis of a characteristic component that forms an image of the object has been most often used. At this time, the characteristic component can often be represented by a combination of a point or a linear component and a curved component. In general, an image having such a contour component is obtained by photographing an object with a television camera and digitally converting a video signal of the image through an A / D converter or the like. Digital signal processing is performed on the converted image data to calculate what shape the object has or where it is located. The first thing to do in such processing is to extract features such as contours formed by the object.

As a conventional method of extracting a contour component of an image, a differential operator such as a Sobel operator is uniformly applied to the entire image, and then the absolute value of the output of the operator is determined. A pixel having a value equal to or greater than the threshold is registered as an edge point. Next, a method is adopted in which an edge point corresponding to the contour is selected in consideration of the relationship between the edge point and a neighboring point, and a portion corresponding to a contour of a curve or a straight line is extracted.

After the edge points corresponding to the contour are obtained, a generalized Hough transform (see Document 1) can be considered as a method of obtaining a portion corresponding to a straight line or a curve. This is because when finding parameters that describe a curve or a straight line, by voting on all possible parameter candidates that pass through each point of the edge and detecting peaks in the parameter space, It is a method of estimating the optimal parameters for describing a straight line.

It is known that the generalized Hough transform requires relatively stable curve components and straight line components. However, the relationship of voting on any curve or straight line parameter having a possibility of passing through an edge point is known. Moreover, there is a disadvantage that the processing time and the storage capacity of the parameter space are enormous. Also, S
To use the obel differential operator and LOG operator,
When the image itself contains a lot of noise, the image itself contains a lot of edge components which are not the outline of the target object, and there is a disadvantage that the time required for the processing becomes enormous.

[0007]

Recently, a system for model-based image recognition has been developed. In such a system, there is a case where a target model to be extracted is provided in advance and it is known what features should be extracted from the image. In this method, the target object is recognized by using a CAD model of the target object, generating a prediction scene in which the target object can be seen in advance, and calculating the correspondence between the predicted scene and the feature extracted in the image. System.

For example, in the above-mentioned document 2, when estimating the position of a mobile robot, a line drawing of a scene that the robot would see is created, and the linear components constituting the line drawing are used as model features, and the linear component A method was devised to calculate the uncertainty of where is in the image and to extract the linear component from the image according to the uncertainty. In this method, the processing is performed by limiting the area to be processed in the image or the area of the parameter space, so that even in a noisy image, a linear component corresponding to the model feature can be relatively stably extracted. But
Model features are limited to straight-line components, and it is not possible to extract features of complex shapes composed of curve components and the like. Further, in the method described in this document, the target object is at a fixed position, and the position of the observer, that is, the position of the camera is movable only in a plane. Therefore, estimating the position of the robot was solved only as three parameter estimation problems that define the position and orientation of the robot. Therefore, it does not provide a more general method of recognizing a three-dimensional target object or estimating a position.

On the other hand, in the method described in Reference 4,
It has been proposed to perform a prediction diagram for the recognition of a three-dimensional object, and perform the recognition and position estimation of the object by comparing the prediction with image features actually extracted from the image. . In this method, feature extraction from an image is performed without directly relating to the target model. Therefore, if there is an object other than the model object, the contour features formed by the object are also extracted at the same time, and there is a disadvantage that the amount of calculation becomes enormous at the stage of associating with the model features in the prediction diagram. Was.

The image feature extraction apparatus of the present invention has been made in view of such a problem, and its object is to recognize and position an object in a three-dimensional space or a two-dimensional space. An object of the present invention is to provide an image feature extraction device capable of quickly and efficiently performing posture estimation.

[0011]

In order to achieve the above object, the present invention provides an apparatus for extracting an object whose shape is known deterministically or stochastically from an image in advance. Calculation means for probabilistically predicting possible values of parameters defining the posture and attitude and calculating the degree of uncertainty, and propagating the degree of uncertainty of the above parameters to the image plane to specify the target object Limiting means for limiting an area in an image where a feature to be present may exist or limiting an area in a parameter space created by a parameter whose feature is expressed in the image; and limiting the feature to the limiting means. Extraction means for extracting within an image area or a parameter space limited by

[0012]

That is, in the image feature extraction apparatus of the present invention, first, the possible values of the parameters defining the position and orientation of the object are stochastically predicted, and the degree of uncertainty is calculated. Next, the degree of uncertainty of the parameter is propagated to the image plane to limit an area in the image where a feature defining the object can exist, or a parameter in which the feature is expressed in the image. Restricts the existence area in the parameter space created by.
Then, the feature is extracted in the limited image area or the parameter space.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to the drawings. In the feature extraction device described in the present embodiment, it is assumed that the relative shape of the object from a certain reference point is known.

FIG. 2 is a block diagram showing an overview of an image feature extracting apparatus which realizes the first embodiment. An image is captured via a TV camera 1 that captures an object, and the image signal is converted into an image digital signal by a digitization (A / D) converter 2. The image data thus converted is once stored in the image memory 3 and then sent to the data processing device 4 for performing data processing. Of course, as long as the data processing device 4 itself has a memory corresponding to the image memory 3, the image memory 3 may be included in the data processing device 4. The data processing device 4 is a general-purpose computer.
Data system.

FIG. 1 is a diagram showing the configuration of the data processing device 4 of FIG. Pair Zomo Del storage module 11, object model is stored, the object model is sent to the image feature prediction module 13. The target object position / posture prediction module 12 is a module that predicts parameters that define the position and posture of the target object, and the prediction result is sent to the image feature prediction module 13. The image feature prediction module 13 is a visual model that defines the target when the target is photographed by the camera based on information on the position (viewpoint) of the photographing device such as a camera and information on the position and orientation of the target. A feature is predicted, and information of the model feature is sent to the model feature-image feature correspondence search module 15, and where the model feature is restricted in the camera image, or a parameter defining the feature is a parameter. It is a module that calculates how to be restricted in space.

Information on this restriction is sent to the next image feature extraction module 14 . Image feature extraction module 1
4 selects an appropriate method for extracting each of the features from within the image, and extracts the features in a limited image space or parameter space. The correspondence between the image feature extracted by the image feature extraction module 14 and the model feature sent from the image feature prediction module 13 is obtained by the model feature-image feature correspondence search module 15, and the correspondence is determined by the object position. A new estimated value is obtained by being sent to the posture estimation module 16 and updating the position / posture estimation value for the given object in advance.

This embodiment particularly describes how to efficiently extract image features among such modules. The model feature-image feature correspondence search module 1 is described.
5 and the object position / posture estimation module 16 will not be described in detail.

The method described in the aforementioned reference 2 is as follows:
When the position of a camera (robot) that can move only on a two-dimensional plane is uncertain, a method of estimating the camera position by recognizing a fixed environment is provided.
On the other hand, what is described in this embodiment is how to estimate the three-dimensional position and orientation of a three-dimensional object, especially when the position and orientation of the three-dimensional object are uncertain. The problem setting is essentially different.

Hereinafter, the processing performed in each module will be described focusing on the case where a linear component is extracted.

The object model storage module 11 stores a model describing an object. When the target model is a rigid body, the method for describing the model is a model of a surface region or a vertex representing the target object. For example, to represent a rectangular parallelepiped, a method of describing six surfaces constituting the rectangular parallelepiped (three-dimensional coordinates of four vertices characterizing the surface) or eight vertices (x1, y
1, z1),..., (X8, y8, z8), or a method in which twelve rectangular parallelepiped ridge lines are used as line segments and formed at the end points.

In any case, these features are expressed based on the parameters that describe them. FIG. 3 illustrates the target model using surface information. In the example of FIG. 3, each surface is configured as a basic element to represent a polyhedron such as a rectangular parallelepiped. Each surface is accompanied by vertices that define that surface. For example, surface 1 in FIG.
It is defined by P2, P3, and P4. In addition, each surface has an outgoing normal vector for that surface.

In this modeling, a reference point P ₀ and a reference rotation posture Q ₀ for the object are considered. By using the reference origin P ₀ and the reference attitude Q ₀ , a local coordinate system (x ₁ , y ₁ , z ₁ ) of the object can be formed. As shown in FIG. 4, by P ₀ ,
Defines the origin of the local coordinate system, is not the direction of the x _1, y _1, z ₁ axis of the coordinate system by Q ₀ is defined.
All points constituting the object can be represented as three-dimensional points P (x ₁ , y ₁ , z ₁ ) in this local coordinate system.

Next, the object is moved to the world coordinate system (x _w , y
_w , z _w ), the object exists in the world coordinate system with a certain position, that is, a posture. Therefore, each point of the object is defined in the world coordinate system (x
_w, y _w, in order to express at z _w) is the reference point P ₀ and the reference position Q ₀ of the object may be expressed and how it can express the world coordinate system. Now, the world coordinate system of the reference origin P ₀ (x
_w, y _w, z _w the position _{of) P 0 = (p x,} p y,
p _z ), and the reference attitude Q ₀ is Q ₀ = (φ _x , φ _y ,
φ _z ). Here, (p _x , p _y , p _z ) represents the x, y, z coordinate system in the world coordinate system of P ₀ , and φ _x , φ
_y, is phi _z, as shown in FIG. 4, respectively viewed from the world coordinate system, represents the rotation angle x _w-axis, y _w axis, with respect to z _w axis.

If the x, y, and z coordinates of each point P in the object take the value of P (x ₁ , y ₁ , z ₁ ) in the local coordinate system, the position in the world coordinate system is obtained. (X _w , y _w ,
z _w )

(Equation 1) And a homogeneous transformation matrix H = H (p _x , p _y , p _z ; φ _x ,
φ _y , φ _z ). Where H
Is

(Equation 2) Into a matrix containing the movement parameters and a matrix containing the rotation parameters.

(Equation 3) Can be written.

As described above, the position of each point on the object in the world coordinate system is determined by the reference origin P ₀ = (p _x ,
_py , p _z ) and the reference attitude Q ₀ = (φ _x , φ _y , φ _z ) in the world coordinate system, and each point constituting the object is expressed in the local coordinate system (x ₁ , y ₁ , It can be expressed using the relative coordinates in z ₁ ). In other words, this target model has parameters p = (p _x , p _y ,
p _z ; φ _x , φ _y , φ _z ) and parameters u of model features that can be expressed in the local coordinate system of the object, and these are sent to the image feature prediction module 13.

Similarly, for a line segment represented in the local coordinates of the object, if the end points of the line segment are represented in the local coordinate system of the object, the point conversion formula represented by the equation (1) is used. By doing so, an expression in the world coordinate system can be obtained.

More generally, patterns and features that define various objects are expressed by expressing the points constituting them in an object local coordinate system and expressing the points by the equation (1). The expression in the world coordinate system can be obtained by using the conversion formula.

When recognizing a certain object, the approximate position and posture of the object may be known. For example, there is a case where the movement of the target object can be roughly estimated by observation up to that time. Let me explain this case now.

At time t0, observation is performed on the position and orientation of the object, and the position and orientation of the object are p0 =
(P0 _x , p0 _y , p0 _z ; φ0 _x , φ0 _y , φ0 _z )
And the velocity vector v = (v _x , v _y ,
v _z ) and the rotation speed vector (ω _x , ω _y , ω _z )
W = (v _x , v _y , v _z from 0 to t1; ω _x ,
ω _y , ω _z ), the uncertainty is the average value W bar
When it considered to be the covariance matrix Σw the position and orientation of the object in the _{t1 p1 = (p1 x, p1} y, p
1 _z ; φ 1 _x , φ 1 _y , φ 1 _z ) can be written by the following relational expression. If Δt = t1−t0, then

(Equation 4) The uncertainty can be expressed using the average value and the covariance matrix.

(Equation 5) Determined by Here, Trans ( ^* ) and RPY ( ^* ) are matrices represented by Expression (3). On the other hand, to obtain the covariance matrix, the relationship between the average value p1 bar of p1 and V and the variation δp1 = p1−p1 bar and δW = W−W bar around the W bar is expressed as (4 ) Is obtained by linearizing (5).

[0030]

(Equation 6) Here, J is a Jacobi matrix for W of p1. When the variation relation is found in this way, the covariance matrix of p1 is

(Equation 7) Can be calculated. Here, E [ ^* ] represents an operator having an expected value. As described above, when the degree of uncertainty of the movement of the target object can be statistically modeled, the future prediction of the parameters representing the position and orientation can be expressed using the average value and the covariance matrix.

As described above, there are other cases where the position and orientation of the object can be statistically predicted in advance. When assembling parts using an industrial robot, etc., the parts to be assembled in advance are often placed in a specified location in advance, but the position is not always accurate. By statistically analyzing the error indicating how far the actual location of the target part is from the specified location, the estimated value of the position of the part is calculated as the average value And a covariance matrix.

The object position / posture prediction module 12
This is a module that predicts what value the parameter representing the position and orientation of the object can take statistically using an average value, a covariance matrix, and the like.

The image feature prediction module 13 captures an image of a target object based on the position and orientation information of the imaging device (camera or the like) and information on the estimated value of the position and orientation of the target object. Predict whether it will be reflected on the device. This is, for example, the average value p of predicted values where the object is located three-dimensionally.
And a method of generating a prediction image from the target model using Such a method of generating a predicted image can be generally realized by using a model of an object, as described in the above-mentioned document 4.

In this embodiment, how the prediction diagram is generated will be described with reference to the example of the polyhedron shown in FIG. In the case of a polyhedron, each surface is constituted by a line segment representing its boundary. Therefore, the prediction view, whether (or visible or invisible) which <br/> unit content of such a line segment is visible from a viewpoint camera is located may be determined. In order to realize this, it is sequentially checked whether or not the line segment connecting the point on each line segment and the viewpoint intersects with another surface. If the line segment does not intersect with any surface, the point is regarded as a visible point. By taking the method of registration, a predicted image can be generated. FIG. 5 shows an example of the output. As shown in this figure, nine line segments

(Equation 8) Is output as a visible line segment. What is important at this time is that the image feature prediction module 13 expresses the output as a parameter in the target object local coordinate system. The visible model feature groups L1, L2, L3,.
It is sent to the model feature-image feature correspondence search module 15 which will not be described in detail in this embodiment.

When one predicted image is generated in this way, it is possible to select a feature defining an object from the predicted image. These features include feature points, line segments,
Alternatively, a curve or the like can be considered. Next, description will be made on how such features show in an image or in a parameter space. In the method described in Reference 2, the degree of position uncertainty of the robot is propagated in the image in order to estimate the position of the robot moving in a completely modeled and immobile environment. The method described below is the reverse operation. That is, when an object whose position is uncertain is photographed by a fixed photographing device, the degree of position uncertainty in the image of the object is calculated.

In such a case, the estimated values can be used for the representation in the world coordinate system of the position P ₀ and the attitude Q ₀ as the reference of the object. Specifically, p = (p _x , p
_y , p _z ; φ _x , φ _y , φ _z ) can be stochastically expressed or expressed using a certain area restriction.

One method of stochastically expressing such parameters is to use the average value of (p _x , p _y , p _z ; φ _x , φ _y , φ _z ) and the covariance matrix. If such a stochastic expression is used, the range of the parameter can be limited by using an appropriate threshold value. Using the average p bar of p and the covariance matrix Σ _p of _p , using an appropriate threshold d,

(Equation 9) When a set is defined, the range in which p exists can be limited by controlling the threshold value d.

The method of restricting by the area is as follows.
(P _x , p _y , p _z ; φ _x , φ _y , φ _z ) means that the possible area is limited to a certain area. For example,

(Equation 10) There is a method of limiting by a space B defined by
Here, α _x , β _x ,..., Γ _z , ε _z are constants.

Now, let u be a parameter in the local coordinate system of the object that defines the characteristics of the object. As u
This is a parameter representing a position coordinate of a three-dimensional point, a direction vector describing a curve, or the like, which is a feature of a target object in a three-dimensional space.

For the sake of simplicity, a case where a three-dimensional point P having position coordinates u = (x, y, z) in the local coordinate system of an object is set as a feature of the object and the point feature is observed by a camera think of. Since the position P ₀ and posture Q ₀ of the object in the world coordinate system are uncertain, observing this feature point with the camera also makes the position and posture of the image in the camera image uncertain. To express how uncertain it is, P
_What is necessary is just to propagate the uncertainties of ₀ and Q _{0 to} the image in the camera image. As shown in the pages 281 to 282 of Document 2 described above, first, a three-dimensional point P (x,
y, z) and the relationship between the camera image P ′ are formulated. For simplicity, it is assumed that the photographing device is modeled as a TV camera 1 and is modeled as a pinhole type camera. Further, assuming that the camera is correctly calibrated, the position v = (r, c) of the point P ′ in the camera image is

[Equation 11] Here, T = (tij) (i = 1, 2, 3; j = 1,
(2, 3, 4) is a 3 × 4 execution sequence called a camera calibration matrix. H = H (p _x , p _y , p _z ; φ _x , φ _y ,
φ _z ) is a 4 × 4 matrix, which is a transformation matrix from the object local coordinate system shown in Expressions (2) and (3) to the world coordinate system.
Further, w is a parameter representing the perspective effect, and the actual position v (c, r) in the image is (rw / w, cw / w)
Can be obtained.

Now, in order to propagate the uncertainties included in P ₀ and Q ₀ to the camera image (r, c) of the three-dimensional feature point u, first, the uncertainties of P ₀ and Q ₀ are calculated. It is represented by the average value p of p and the covariance matrix Σ _{p of p} . This method is as described above. Then, v = (r,
The average value v bar of c) = (r bar, c bar) is obtained by using the average value w bar of the parameter w representing the perspective effect.

(Equation 12) Is determined by Further, after transforming equation (12) and removing the term of w, r and c are represented as functions of p. Then, with respect to the equation, considering the variation δp = pp bar, δv = vv bar around p bar, v bar = (r bar, c bar),

(Equation 13) Can be written. Here, F is a Jacobi matrix for (r, c) with respect to p. Then the covariance matrix Σv of v is
Assuming that the operator taking the expected value is E [ ^* ],

[Equation 14] Can be written. Here, a relational expression relating to the covariance of the position and orientation parameter p of the object,

(Equation 15) Was used.

As described above, the average value and the covariance matrix of v = (r, c) can be obtained. Furthermore, this v =
Based on the average value of (r, c) and the covariance matrix, the range in which v exists can be stochastically limited. For example,

(Equation 16) Is defined, and the area may be limited by A (v bar, Σ _v ; d) using a certain threshold d as a control variable. Here, if the control variable d is made large, v will be included in A (v bar, Σ _v ; d) with a higher probability.

FIG. 6 shows an example in which the degree of uncertainty included in the parameters of the position and orientation of the object is propagated in the camera image. FIG. 6A illustrates the position and orientation uncertainty of the origin P0 of the local coordinate system of the object by using an ellipsoid. FIG. 6B illustrates the degree of uncertainty of a feature point predicted to appear on the camera image plane when a point is selected as an image feature.

In the above description, a three-dimensional point is used as a feature. However, various features defining an object can be generally represented by a certain parameter u under the local coordinate system of the object. it can. When a feature can be parameterized, the parameter when the feature is expressed as a camera image
Find the formula to be converted to data. For example, feature points on the object are parameterized as points on the camera image, and line segment features on the object are expressed as line segments on the camera image,
It can be parameterized. Next, the uncertainty included in the object and the posture is propagated to the parameters of the camera image, and the camera image parameter space is restricted based on the uncertainty.

In the above example, the method of solving the point feature has been described in detail. However, in the general case where the object is described by the parameter u, the uncertainty is determined by the following method. Can be propagated.

(1) Parameterize the position and orientation of the object. For example, let that value be p. An average value p bar and a covariance matrix Σp regarding _p are obtained.

(2) Express the feature of the object using parameters. The parameters are expressed in the local coordinate system of the object. For example, let u be the parameter of the feature.

(3) A feature of the camera image corresponding to the feature of the object is selected, and a parameter v for expressing the feature in the camera image is set.

(4) An equation f (u, v, p) = 0 between parameters relating u, v, p is obtained. Next, the degree of uncertainty associated with the parameter v is calculated. First, to find the average value v bar, use the equation

[Equation 17] Can be solved. On the other hand, in order to obtain the covariance matrix Σv, the average value pbar, v of the equation of f (u, v, p) = 0
It is determined by taking the variation around the bar. In particular,

(Equation 18) F (u, p bar, v bar) = 0
By taking advantage of

[Equation 19] Available and finally

(Equation 20) The variation δv can be represented by the variation δp. Now, if the part of the transformation matrix of the previous equation is represented by F,

(Equation 21) Is obtained, and from this, the covariance matrix Σv of v is

(Equation 22) Can be expressed in the form Then, select an appropriate threshold d,
The parameter area A (v bar,
Σ _v ; d). The ellipse in the frame in FIG. 6B represents such an area where the image feature points are restricted.

In the above example, the case where a feature can be represented by a single parameter has been described. However, a single feature may be represented by combining a plurality of parameters. For example, when the position coordinates of the two end points are given as parameters in the linear component, the position of the entire linear component in the camera image is uncertain as described in the above-mentioned document 2. The degree can be determined as follows.

First, for each end point of the linear component feature, the degree of position uncertainty in the camera image is obtained, and a restricted area created from the degree of position uncertainty is combined.
It is possible to calculate a straight line presence restriction area. FIG.
(A), (b), and (c) show this method. FIG. 7A shows the construction of a restricted area where two end points P1 and P2 of the linear component exist. FIG. 7B shows that a convex hull including the end points is formed from the restricted areas, and is used as the restricted area of the linear component P ₁ P ₂ . This existence restriction area does not necessarily need to be constituted by a minimum closed area as shown in FIG. For example, FIG.
As shown in (c), it may be approximated by a rectangular area.

It is known that the Hough transform is an effective means for extracting a linear component (Ref. 2, page 2).
85). As shown on page 300 of Reference 2, the linear components are previously calculated in a three-dimensional space.

(Equation 23) It can be expressed as. Here, [a, b, c] ^T is a direction cosine that defines the direction of a straight line, and [x ₀ , y ₀ , z ₀ ] ^T
Is an arbitrary point on the straight line, and t is a parameter. When such a three-dimensional straight line is photographed by a camera that can be approximated by a pinhole model, the image also appears on the camera image as a straight line. As shown in FIG. 8, the Hough transform expresses the straight line by the distance ρ from the origin and the direction γ of the normal of the straight line, and the space created by (ρ, γ) is called the Hough space. If the values of the reference parameters P ₀ and Q ₀ for the position and orientation of the object are uncertain, the values of these Hough transform parameters in the camera image are also uncertain.

As described above, by transmitting the uncertainties of P ₀ and Q ₀ to the Hough transform parameter, the uncertainty of the Hough transform parameter can be calculated. Now, the average value of the parameters corresponding to v = (ρ, γ) is calculated as v bar
= (Ρ bar, γ bar), and let the covariance matrix be Σv. FIG.
Represents the degree of uncertainty in Hough space.

Once the degree of uncertainty of the Hough transform parameter can be calculated, the existence area of the Hough transform parameter can be limited by using the threshold value d as shown in the following equation.

[0055]

(Equation 24) This limiting method is the same as the method represented by Expression (19). Actually, in order to extract a linear component, it is sufficient to extract a corresponding linear component only in the range of the limited area. In this way, the image feature prediction module 13 calculates how the feature defining the object looks and what value the parameter expressing the feature can take.

Next, in the image feature extraction module 14,
The model feature selected by the image feature prediction module 13 is extracted from the camera image. Here, the extraction of a straight line component P ₁ P ₂ having P ₁ and P ₂ as end points will be described as an example. Information on a certain straight line created by the image feature prediction module 13 is as follows: (1) Uncertainty of the position of the end point of the straight line component in the camera image (2) Uncertainty of the Hough transform parameter of the straight line component 2 points. The image feature extraction module 14 uses such information to select a method for extracting the linear component feature.

To extract a linear component, the following method is used.

(1) A Hough accumulation matrix D (ρ, γ) corresponding to the area where the Hough transform parameter of the linear component exists is prepared. Here, the Huff space is appropriately discretized. The size of this matrix corresponds to the limited size of the Hough transform parameter. For example, the range that ρ can take now is [ρmin
, Ρmax] and the possible range of γ are [γmin, γmax]
In this case, the matrix only needs to include the above ranges of ρ and γ. Next, this Huff accumulation matrix D (ρ,
γ) is initialized to zero.

(2) The limited image space is converted into a rectangular area [rm
in, rmax, cmin, cmax].
A differential operator which is particularly sensitive is applied to a desired average slope (the Hough transform parameter γ indicates the normal direction of the linear component, and the direction orthogonal to γ corresponds to the slope of the linear component). An example of such an operator is described in, for example, the Integrated Directional Derivative in pp. 403-410 of Reference 3 described above.
As shown in Gradient Operator, it can be composed of linear filters. This special filter is applied in order to detect a linear component in a desired direction while suppressing the influence of noise as much as possible. Then, this filter is applied to the limited image area.

Regarding a pixel (r, c) whose absolute value of the output of the filter is larger than a certain threshold value, it is considered that the pixel may form an edge corresponding to a desired linear component.
The corresponding ρ is calculated for all possible angular directions of γ. That is, while changing γ from γmin to γmax,

(Equation 25) Is calculated, and the value D (ρ, γ) of the element of the Hough accumulation matrix of (ρ, γ) corresponding thereto is increased by one.

The above is repeated for all the pixels in the limited image area.

(3) A peak is searched for in the Hough accumulation matrix D (ρ, γ), and when a linear component corresponding to the peak really exists as a linear component in the limited image, the peak is searched for. -Registered as a linear component corresponding to The operation of step (3) is performed until a desired linear component can be detected or a peak of a certain size or more can be detected.
repeat.

The above operations (1), (2), (3)
By doing so, the area to be processed in the image is limited,
High-speed processing can be realized. In addition, by applying a special filter, a linear component can be extracted with as little influence of noise as possible. Another important point is that the method described in the above-mentioned document 2 only describes a method of extracting image features of linear components for estimating the position and posture of a robot moving on a plane. Does not provide a method for extracting the image features of a three-dimensional object in which is uncertain. In order to solve a more complicated problem than the method of Reference 2, in this embodiment, the degree of uncertainty of the position and orientation of the three-dimensional object is efficiently propagated to the image plane and the parameter space by the above-described method. It is.

The second embodiment will be described below. In the second embodiment, a method of extracting a characteristic of a target object as a curve is shown. Object model storage module 11 and object position / posture prediction module 12
The functions of the image feature prediction module 13 and the feature extraction module 14 will be described here.

After the object model storage module 11 sends the model information of the object to the image feature prediction module 13,
The image feature prediction module 13 selects a curve component appearing in the predicted image.

If the feature is displayed as a curve, the representative points constituting the curve are determined, the degree of uncertainty in the image with respect to the representative point is first determined, and the closure of these uncertain areas is considered. It is possible to limit the region where the curve exists in the image.

For predictive generation of a curve, first, the degree of uncertainty of a curve representative point serving as a model is obtained from an average value and a variance.

The model curve is generally calculated using parameters.

(Equation 26) It is described in the form of Here, (r, c) is a parameter representing the row / column direction of the image, and t is a parameter defining the length of the curve from one end point thereof. p is a parameter representing the characteristic of the curve.

When it is determined that it is difficult to extract the entire curve as one segment, the curve component is divided into a plurality of curve components. The curve component to be divided is now called a model curve segment. As this dividing method, for example, the following method can be considered.

(1) The model curve is divided at equal intervals. As shown in FIG. 10A, the curve is divided based on the length of the model curve to generate a curve segment.

(2) When the distance between the straight line connecting the end points of the model curve and the farthest point in the curve exceeds a certain threshold value,
The curve is divided into two at the farthest point, and the divided two
The same process is performed on the two curve components, and the curve division is repeated until the longest distance between the straight line connecting the point on the curve and the end point reaches a certain threshold. As shown in FIG. 10 (b), determine the most distant point P _m from the line segment connecting the end points P ₁ and P ₂ between the curve P ₁ P _2. Smaller than a certain threshold distance d from P _m to the line segment P ₁ P _2, it registers the curve P ₁ P ₂ as one curve segment. If it is greater than the threshold, the curve is divided into two segments, P ₁ P _m and P _m P ₂ , and a similar division is performed on the two segments.

(3) If the interior angle at the farthest point of the triangle formed by the two end points of the model curve and the farthest point in the curve is smaller than a certain threshold value, the farthest point is divided into two parts, which are divided. The same processing is performed on the two curve components. As shown in FIG. 10 (c), a point P _m on the curve farthest from the line segment P ₁ P ₂ is obtained, an angle formed by P ₁ P _m P ₂ is checked, and the value is calculated from a certain threshold value. If smaller, the curve P ₁ P ₂ is registered as one curve. If not, the curve is generated into _two curve segments P ₁ P _m and P _m P ₂ and the same division is repeated for each curve segment.

(4) Starting from one end point of the model curve, a representative point constituting the model curve is traced from the start point, and the tangential angle at the representative point is examined. When the angle change in the tangential direction with respect to the one end point exceeds a certain threshold value, the curve component from the end point to the representative point is registered as one model curve segment, and the representative point is set as a new start point. And the same processing is repeated. FIG.
0 (d), the representative points P ₁ ,
P ₂ ,..., P ₈ , and each representative point Pi (i = 1, 2, 2,
..., to calculate the tangent direction at 80) from the start point P ₁ in the order.
If regard is Pi, the angle of the tangent of P ₁ and P _i phi
_{When i} exceeds a certain threshold value, P ₁ P _i is registered as a curve segment, and the same processing is repeated with P _i as a starting point. This operation is performed until the start point coincides with the end point.

Of course, the division of the curve is not limited to the above method. After the division, the curve segments are extracted in a manner suitable for each feature in each region. With respect to such curve segments, the method is selected that will most easily extract the curve segments. More specifically, this is performed by segmenting a given model curve segment within a certain area as a component that is almost linear and has a certain angular direction. Let N be the total number of curve segments divided in this way.

A search area is first calculated which indicates where to find each model curve segment in the image.

Next, in the image feature prediction module 13, the line segment P formed by the end points P ₁ and P _{2 of} the model curve segment
The search area of P ₁ and P ₂ is reversely rotated by the angle of ₁ P ₂ . Then, a predicted image can be considered in the rotated new coordinate system. FIGS. 11A and 11B show this rotation. As shown in FIG. 11A, the curve segment P ₁ P ₂ is inclined from the column direction r by θ ° in the predicted image. The coordinate system is reversely rotated by θ ° so that the line segment P ₁ P ₂ is parallel to the column direction r ′. By adopting this method as shown in FIG. 11B, the curve segment P ₁ P ₂ is transformed into the predicted image in the column direction r ′.
Can be regarded as a straight line component substantially parallel to. Thereafter, the search areas of P ₁ and P ₂ are determined in the rotated coordinate system. The P _1, then form a search area of the curve segment P ₁ P ₂ from the search region of P _2. A method for generating this is shown in FIGS. 12 (a) and 12 (b). 12 (a) is seeking convex hull of the search area of P _1, P _2, are those which were expressed what you search area of the curve segment, FIG. 12 (b), P _1, P the rectangular closure of the _second search area is obtained by approximating the search area of the curve segment P ₁ P _2. Thus, the image feature prediction module 13 calculates a search area in the predicted image for each curve segment.

The output from the image feature prediction module 13 is calculated for each curve segment by calculating an uncertainty indicating in which area the model curve segment exists in the rotated prediction image system (search area). . On the other hand, the image feature extraction module 14 has a function of actually extracting an image curve segment corresponding to each model curve segment in the search area. Hereafter, the image feature extraction module
The processing in the rule 14 will be described with reference to a flowchart shown in FIG.

First, let N be the total number of model curve segments. Then, an index representing each model curve segment is set to k, and k = 1 is initialized (step S1).
Here, the steps are simply represented by S hereinafter. Next, the processing in S2 will be described. For the model curve segment k, the image feature prediction module 13 has calculated its search area in a rotated coordinate system. In this step, the original image is rotated so as to correspond to the rotated coordinate system. Actually, the model curve segment rotates in reverse by the angle of the line segment P ₁ P ₂ formed by the end points P ₁ and P ₂ . Now, assuming that the angle formed by the line segment P ₁ P ₂ is θ, the point (r, c) of the original image and its rotated corresponding point (r _p , c _p )
Takes into account the offset (r ₀ , c ₀ ) on a certain coordinate,

[Equation 27] Can be calculated by The point to be noted in this calculation is that the coordinate transformation (r _p , c _p ) of the image to be rotated is used, the corresponding point is found in the original image (r, c), and the grayscale value is calculated. In other words, the gray value of the rotation destination image is used. Actually, (r _p , c _p ) is an integer, but the coordinates (r, c) of the corresponding point in the original image are not integers.
After calculating c), a means for converting to an integer by rounding or the like is used, or an approximation is performed in consideration of the corresponding neighboring points. Here, the approximation will be briefly described. Now, it is assumed that the corresponding pixel coordinates (r, c) of the original image can be calculated by the above equation (30). The integer coordinate value obtained by truncating the decimal part of (r, c) is (r1, c1). At this time,

[Equation 28] Is calculated. If the gray value of the pixel of the original image corresponding to the coordinate value (R, C) is represented by g (R, C), the gray value corresponding to (r, c) to be obtained is

(Equation 29) , A linear approximation can be obtained.

Next, in S3, a binary image for the differentiation of 0 ° is created. First, to differentiate the original image, FIG.
In general, a window-type linear filter as shown in FIG. For example, the indexes indicating the row and column directions of the image are r and c, respectively, and the coordinates (r,
If the gray value in c) is a (r, c) and the output of the differential image is b (r, c), a window-type filter w
According to (i, j), b (r, c) can be obtained by the following calculation.

[0080]

[Equation 30] For example, a window filter for 0 ° is shown in FIG.
This is as shown in (c) (see Document 3). Furthermore, as shown in pp. 337-352 of the same reference 3, we consider the response of the desired filter,
The value of w (i, j) can be calculated. FIG.
(B) and (c) show a filter having a size of 5 × 5, but the size is not limited to 5 × 5.
A filter such as mxn may be used. Instead of a linear filter, a non-linear filter that performs spatial differentiation may be used.

The output b (r, c) of such a differential filter
Can be expected to have a large absolute value output for a pixel having an edge component in the direction of approximately 0 °.
In the next operation, two binary edge images are generated from the differential image in consideration of the sign of the differential. Specifically, the first
When the value of the differential image b (r, c) is larger than a threshold value threshold_positive, the corresponding edge image ep (r, c) is set to 1;
p (r, c) is set to 0.

On the other hand, in the second edge image, if the value of the differential image b (r, c) is smaller than a certain threshold value threshold_negative, the value of the corresponding edge image en (r, c) is set to 1, and otherwise. For example, the edge image is set so that the value of en (r, c) is set to 0. Here, the reason why the two edge images are separately prepared is to clarify the connection relationship for tracking the later edge components.

When the edge image is created from the differential image in this way, the entire image becomes a binary image composed of 0 or 1. Of course, this image is an image including an edge in a certain angular direction. However, when the image noise component is significant, the image may still include a pseudo edge component corresponding to the noise.

In the next S4, such noise components are removed. What is important here is that, in this binary edge image, only a curve portion corresponding to almost a specific direction 0 ° is extracted, and those corresponding to other angle components are removed as much as possible. As in the previous example, the angle of the curve segment P1P2 can be almost limited because the search area of the curve is limited. If the angle range is from minus a ° to plus a °, the center angle is 0 ° to ±
What has high sensitivity in the range of a ° may be used. Then, the removal of the noise may be limited to this range. In order to perform such noise removal on a binary image, a method of removing a noise component with reference to a pixel near the pixel of interest is conceivable.

As one example, this is realized by applying a window-type filter (or a template-type filter) consisting of a binary element having a specified size to the vicinity of the target pixel. For example, consider a template type filter that removes noise contradictory to an edge in the 0 ° direction. When it is considered that there is a curve passing through a pixel by referring to the vicinity of a certain center pixel point (r, c), a noise such that the output for the center pixel is 1 and otherwise 0. A template for removal may be prepared.

Hereinafter, this processing is referred to as template processing. For example, as shown in the example of FIG. 15A, when the center point is the target pixel (r, c), there is a high possibility that there is a curve passing through the target pixel in consideration of the neighboring 5 × 3 area. Sometimes, the output of the template processing is set to 1. On the other hand, when the center pixel is the point of interest as in the case of FIG. 15B, the output of the template processing is set to 0 because the possibility that the curve passes through the point is low. Such a template process can be realized by creating a table corresponding to the pixel values in the vicinity of the pixel in advance and performing a look-up table process.

For example, in order to realize a look-up table process using pixel values in the vicinity of 5 × 3 around the pixel of the center pixel (r, c), 2 at position (r + i, c + j)
Corresponding to the image value a (r + i, c + j)

(Equation 31) There is a method of performing a look-up table process by preparing a table corresponding to the address "address" calculated in advance in advance. Of course, the noise removal method may not use a table,
The size of the template is not limited to 5 × 3.

By removing such noises, small isolated points and the like in the binary image are almost completely removed, and at the same time, the divided curve portions are connected to recover one curve. FIG. 16 shows this state.
In the figure, the black points represent pixels that are 1 and the portions without black points represent pixel groups that are 0. FIG.
6A shows a binary image before the noise removal processing, and FIG.
FIG. 6B shows the processed image.

Next, in S5, the thinning processing is performed on the binary image from which the noise component has been removed. This is because the curve component does not become a 4-connected or 8-connected binary curve only by the above-described noise removal method. Since many methods have been proposed for thinning a binary image, a detailed description will not be given here, but the simplest method is realized by repeatedly applying a template type window to the image. (For example, see Reference 3, pp.168-173). By repeatedly performing the template processing for thinning in this manner, the curve component is thinned, and the process proceeds to the next step.

Next, in S6, the curve components in the direction of approximately 0 ° are traced with respect to the thinned curve components. This tracking utilizes the fact that the tracking direction is limited to a certain angle. For example, in the above example, it is sufficient to track a curve only in the direction of approximately 0 °. Hereinafter, curve tracking will be described.

(1) Calculation of connection order of curved pixels First, an image is scanned using a 3 × 3 window, and the connection order of each pixel of the thinned binary image with the vicinity of each pixel is stored as an image. . This concatenated order corresponds to the number of pixels having a value of 1 among its eight adjacent pixels when the central target pixel is 1 in a 3 × 3 window. In order to explain the method, two terms, namely, an end point of a curve and a branch point are defined. The end point means a start point or an end point having no connection other than the curve, and represents a point where the curve is connected only in one direction.
On the other hand, a branch point means a point where a curve branches. For example, if the pixel of interest is the end point of the curve, the degree of connection is 1, and the point on the curve that is neither an end point nor a branch point is
The coupling order is expressed as 2. On the other hand, the branch point of the curve is represented as a number whose degree is 3 or more. When the value of the image of interest is 0, the connected degree is 0. In this way, the connected order image is composed of an image having the same size as the original thinned image, in which the value corresponding to each pixel is an integer.

(2) Next, while scanning this connected-order image, an end point or a branch point of the curve component is searched for. When an end point or a branch point is found, adjacent pixels connected in an eight connection relation are tracked from that point. This tracking continues until the next branch point or end point is reached. In tracking the curve component one pixel at a time, the connected degree of the pixel at the present time and the connected degree of the next connected pixel are reduced by one. When one curve component is tracked in this way, that curve component becomes
It is checked whether the curve has a direction of 0 °.

There are various methods for this check. For example, as one method, if the straight line connecting the start point and the end point of the curve is within the directional angle permitted by 0 °, the curve is recognized as a curve in that direction, otherwise, it is recognized as a curve. Instead, the curve components are discarded. When the curve component is recognized as a curve component permitted at 0 °, the curve component corresponding to the angle is described based on a data structure as shown below. And
Registered as a curve component.

[0094]

(Equation 32) Here, c_id is an integer representing the serial number of the curve, and o_id
id indicates an index k indicating the direction of the curve. The characteristic of the curve is expressed by at least one of the start points of the curve if it is an end point, and 0 if both of them are branch points. On the other hand, n is the number of pixels constituting the curve and is a number including the start point and the end point. The starting point and the ending point of the curve are the end points or branch points used when tracking the curve. Finally, the curve-constituting pixel group represents a pixel group that has passed through the tracking of the curve. Here the first pixel is equal to the start point and the last pixel is equal to the end point.

After the registration of one curve element is completed in this way, the end point or the branch point of another curve is searched for in the connected order image, and the similar curve tracking and registration are executed. This process is performed until all the curves have been searched, that is, until 1 disappears from the connected order image. At this time, it is not necessary to check the entire image when checking whether or not the connected order image includes 1. First, before extracting a curve, the sum of all the connected degrees is obtained and stored in a certain counter s. When the curve is traced, the connected degree is reduced by one. At this time, the value of the counter s is reduced by one. And every time the registration of one curve is completed,
The value of this counter is checked, and if the value becomes 0, the extraction of the curve is completed.

After the extraction of the image curve segment corresponding to a certain model curve segment is completed, the value of the index k is increased by 1 (S7).

Next, in S8, the index k is compared with the total number N of the model curve segments. If the value of k becomes larger than N, the above processing is terminated. If k does not exceed N, the process returns to step 2 and the same process is repeated for the next model curve segment.

In S9, through S1 to S8, image curve components of all model curve segments are extracted. All the extracted image curve components are described by the method shown in Expression (35). In this step, the curve components extracted in each direction are integrated so as to fit into one image.

The curve component represented by the equation (35) was obtained in the rotated coordinate system. In this step, first, all the curve components obtained in the rotated coordinate system are returned to the display of the curve components in the coordinate system of the original image, and then integrated to generate a new curve component.

For this reason, first of all, the size of the 2
One value image (curve storage image) is prepared, and a value of 0 is assigned to all pixels as an initial value.

With respect to the curve group represented by the equation (35),
First, the coordinates (r, c) of the original image are calculated by the coordinate transformation of Expression (30). When the coordinates (r, c) are sufficiently close to the integer grid points, 1 is assigned to the pixel corresponding to (r, c). If the coordinates (r, c) are not close to integer grid points, they are assigned to two or four grid points 1 in the vicinity. To determine whether or not it is close to an integer lattice point, the coordinates (r, c) and the Euclidean distance to the lattice point are calculated, and if the value is smaller than a certain threshold value, it is determined that it is sufficiently close.

The above processing is performed on all the curve components extracted from S2 to S8. Then, the curve components in all directions are expressed as image data in the curve storage image.

The curve storing image is subjected to thinning processing again. The reason why the thinning is performed is that there is a possibility that some of the curves obtained in the images in different angular directions may overlap, and at the same time, some of the curves may include an error of about one pixel. is there. By performing thinning in this manner, gaps between pixels of a curve extracted in another angle direction are filled, so that a more accurate curve is generated.

Now, after all the curve components are represented in the binary image by this operation, the curve tracing is performed again. This operation is almost the same as the method described in S6. The difference is the following two points. First, S9 is different from the first embodiment in that the angle is not limited after the curve is traced. Furthermore, in relation to the tracking of a curve, using the status indicating the characteristics of the curve, a very short curve component and at least one of the start and end points of the curve component that is not a branch point but an end point is a noise. The point is that it is removed as a component.

This corresponds to removal of a mustache component which often occurs when performing curve extraction. Thus, the curve is given by equation (35)
Can be completely described by Of course, when an image curve segment is extracted in a limited area by the above-described method, only one segment is not necessarily extracted. In such a case, the model feature-image feature correspondence search module 15 of FIG. 1 will select the most appropriate image curve segment. Here, the model feature-image feature correspondence search module 15 will not be described.

As described in the present embodiment, it is possible to efficiently extract a curve component appearing in an image of an object modeled in advance while removing noise components as much as possible.

As described above, two types of features (linear component and curved component)
The method for extracting features described in this embodiment is not limited to such features. When expressing an object as a model, it goes without saying that the above method can be applied to features that can be expressed using parameters. For example, if a feature can be represented as a pattern with a certain spatial extent, select a parameter corresponding to the pattern and represent the image projected in the camera image with that pattern By obtaining a parameter and calculating the degree of uncertainty regarding the parameter, efficient extraction of the pattern feature can be realized. Further, by transmitting the uncertainty to the parameters of the general Hough transform as described in the above-mentioned reference 1, it is also possible to efficiently extract features corresponding to the uncertainty.

Hereinafter, a third embodiment will be described. In the above-described embodiment, the target model has been described only when the shape of the model is accurately known. In the present embodiment, even when an uncertain element is included in the target model itself, a method for efficiently extracting features included in the model from the image is described. The shape of an object may not always be known exactly. It is also conceivable that the shape slightly changes over time.

In such a case, it can be considered that the parameter u that describes the model feature described above can be expressed only stochastically. This is the model feature parameter u
May be represented by the mean value of the and the covariance matrix. This is the average value u for the model feature parameter u.
This means that the bar and the covariance Σu have been added. Also in this case, by extracting the uncertainty of the image feature parameter v in which the model feature parameter is expressed in the camera image, the image feature is efficiently extracted in the camera image. be able to. Here are the steps:

(1) In the model expression of the target model storage module 11, the parameter u of each model feature is the average value u.
It is expressed in the form of a bar and a covariance matrix Σu, and sent to the image feature prediction module 13.

(2) Object position / posture prediction module 1
In step 2, parameter values indicating the position and orientation of the origin of the local coordinate system of the object are predicted. Since this method is the same as that of the first embodiment, the description is omitted here.

(3) In the image feature prediction module 13,
The parameter p of the object position / posture obtained by the object position / posture prediction module 12, the average value p bar, and the covariance matrix Σp are used. That is, a predicted image in the camera image of the object is generated using the average value p bar.
The image feature parameter v corresponding to the model feature parameter u is determined, and its uncertainty is calculated as follows. A relational expression f (u, u) between three parameters p, u, v
v, p) = 0 is determined, and an average value v bar of v is calculated using the values of the average values p bar and u bar.

The method is obtained by solving the equation of f (u bar, v bar, p bar) = 0. Next, u
Estimate the covariance matrix of. The method is f (u, v,
By taking variation around the mean of p) = 0.
In particular,

[Equation 33] From the fact that f (u bar, v bar, p bar) = 0,

(Equation 34) From which the variation δv is represented by the variations δu and δp:

(Equation 35) This is simply expressed using the matrices F and G as follows.

[0114]

[Equation 36] Assuming that u and p are statistically independent, the covariance matrix Σv for v can be calculated as follows:

[0115]

(37) Assuming that u and p are not independent, when the operator taking the expected value is E [ ^* ],

(38) It can be calculated in the form As described above, the uncertainty of the model feature parameter u can be calculated from the average value of the model feature parameter u and the covariance matrix. Further, using these statistics, the u
Can be limited.

FIG. 17 is a schematic diagram showing propagation of uncertainty. FIG. 17A shows the position of the origin of the target object.
The posture uncertainty is represented by an ellipsoid, and the position uncertainty of each vertex in the object local coordinate system with respect to the origin is also represented by an ellipsoid. These two types of uncertainties are merged and propagated to the camera image plane. The degree of uncertainty of each image feature point created by the image feature prediction module 13 is represented by a frame ellipse in FIG.

(4) In the image feature extraction module 14,
An image feature is extracted in an area where the image feature parameter v corresponding to the model feature parameter u is limited. The extraction method has been described in the first embodiment and the second embodiment, and will not be described here.

As described in the present embodiment, even if the positions and parameters of the features defining the object model are uncertain, the degree of uncertainty is described statistically.
The image of the feature appearing in the image can be efficiently extracted.

In the above three embodiments, the object has been described as a three-dimensional object, but the present invention is not limited to the feature extraction of a three-dimensional object. Of course, the present invention can be applied to a case where a two-dimensional object is extracted. In that case, a parameter indicating the degree of uncertainty of the position and orientation of the target object
The parameter p is changed from a six-dimensional vector to a three-dimensional vector, and the feature parameter u of the object in the local coordinate system is appropriately changed.

[0120]

As described in detail above, according to the present invention,
It is possible to extract a feature that preliminarily defines a target object from an image as efficiently as possible without being affected by noise.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating an overview of an image feature extraction device that implements a first embodiment.

FIG. 2 is a diagram showing a configuration of the data processing device of FIG. 1;

FIG. 3 is a diagram expressing a target model using surface information.

FIG. 4 is a diagram showing a local coordinate system (x ₁ , y ₁ , z ₁ ) of an object using a reference origin P ₀ and a reference attitude Q ₀ .

FIG. 5 is a diagram illustrating an output example of a predicted image.

FIG. 6 is a diagram illustrating an example of a case where an uncertainty included in parameters of the position and orientation of an object is propagated in a camera image.

FIG. 7 is a diagram for explaining a method of realizing a straight line presence restriction area.

FIG. 8 is a diagram for explaining Hough transform.

FIG. 9 is a diagram illustrating an uncertainty in a Hough space.

FIG. 10 is a diagram for explaining a method of dividing a curve component into a plurality of curve components.

FIG. 11 is a diagram showing rotation of coordinates.

FIG. 12 shows a curve segment P from a search area of P ₁ and P _2.
It is a diagram for explaining a method of generating a search area of ₁ P _2.

FIG. 13 is a flowchart showing processing of a feature extraction module.

FIG. 14 is a diagram illustrating an example of a window-type linear filter.

FIG. 15 is a diagram for explaining template processing.

FIG. 16 is a diagram showing images before and after noise removal processing.

FIG. 17 is a schematic diagram of propagation of uncertainty.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... TV camera, 2 ... A / D converter, 3 ... Image memory,
4 Data processing device 11 Object model storage module 12 Object position / posture prediction module 13 Image feature prediction module 14 Image feature extraction module 15 Model feature-image feature correspondence search module
16 ... Object position / posture estimation module.

Continuation of the front page (56) References Akio Kosaka et al. , Fast Vision-gui ded Mobile Robot N avigation Using Mo del-based Reasonin g & Prediction of Uncertainties, Proc eedings of the 1992 IEEE / RSJ Internati onal Conference on Intelligent Robot s and Systems, Vol. 3, 1992 July 7, Pp. 2177-2186 (58) Field surveyed (Int. Cl. ⁷ , DB name) G06T ⁷ /00-7/60 G06T 1/00 JICST file (JOIS)

Claims (1)

(57) [Claims] An apparatus for extracting, from an image, an object whose shape is known deterministically or stochastically in advance, stochastically predicts possible values of parameters defining the position and orientation of the object. Calculating means for calculating the degree of uncertainty; and propagating the degree of uncertainty of the parameter to an image plane to limit an area in the image where a feature defining the object may exist or to limit the area. Means for limiting the existence area in the parameter space created by the parameter represented in the image, and extraction for extracting the feature in the image area or the parameter space limited by the limitation means Means for extracting an image feature.