JP2000088554A - Search method for feature point of object, and memory media with record of process program thereof and search device for feature point - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable enhancing accuracy of data of a shape feature point and a shape model by respectively searching each image when images from a plurality of view points are given. SOLUTION: A shape model S101, which describes data of a face surrounded with sides including a scene (an attention area) from an image being shot the scene with a camera, is made, and three-dimensional point (S102) of a top of an object constituting the scene from the shape model made is measured. A search area (S104) is determined from the three dimensional data of the object top measured and a position and attitude data of the camera. A search (S107) of a feature point from the nearest image with the area (S105) and the scene image (S106) is done and a candidate (S108) of the feature point is extracted. A side or face adjacent to the feature point from the shape model is searched (S109), and side or face data is extracted (S110). A side or face of the image (S111) extracted from side or face data (S110) and the candidate of the feature point is searched, thereby obtains, the most reliable feature point.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of searching for a feature point of an object by a computer, a recording medium storing a processing program for the method, and a device for searching for a feature point of an object. The present invention relates to a technology that is effective when applied to a technology for searching for a geometric feature point corresponding to a scene image from a scene image (image of a region of interest).

[0002]

2. Description of the Related Art In order to understand a scene image, a shape model (polygon) having information on three-dimensional coordinate values of vertices of an object constituting a scene and sides indicating a connection relationship between points and a surface surrounded by the sides is known. Model) is used.

If the shape model of the scene and the position and orientation of the camera are given accurately, any three
The points of the image corresponding to the dimension points can be calculated geometrically. An image feature point closest to this point is defined as a shape feature point.

[0004] For example, a conventional method for searching for a geometric feature is as follows.
As shown in FIG. 10, a shape model that describes information on a side including a scene and a surface surrounded by the side is created from an image obtained by capturing a scene with a camera (S101). Measure the three-dimensional point of the vertex (S1
02). The position and orientation of the camera are measured (S10
3). A search area is determined from the measured three-dimensional point information of the vertex of the object and the position and orientation information of the camera (S1).
04). A feature point search is performed from an image closest to the determined search area (S105) and the scene image (S106) (S107), and a most probable feature point is obtained (S108).

In order to search for a corresponding geometric feature point in a plurality of images, a part of another image (on an epipolar line) is searched from the coordinates of the feature point of one image and the highly accurate variation of the position and orientation of the camera. Repeat to do.

[0006]

However, in the above-mentioned conventional method, the image feature point closest to the center projected point is used as the shape feature point. It is difficult to measure the landscape and the like, especially in a wide area. Further, when there is an error between the shape model and the position and orientation sensors of the camera, it has been difficult to search for a feature point.

[0007] An object of the present invention is to provide an image processing apparatus which is capable of matching three-dimensional points of a shape model from an image even when a shape model having an error in the coordinate values of a polygon and a camera position and orientation and an image having the error are given. It is an object of the present invention to provide a technique capable of searching for a shape feature point to be changed.

Another object of the present invention is to provide a technique capable of improving the accuracy of a geometric feature point and the accuracy of a shape model by searching from each of images given from a plurality of viewpoints. It is in. The above and other objects and novel features of the present invention will become apparent from the description of the present specification and the accompanying drawings.

[0009]

The outline of a typical invention among the inventions disclosed in the present application will be briefly described as follows. (1) A shape model that describes information of a surface including a side and a scene (attention area) surrounded by the side is created from an image obtained by capturing a scene with a camera, and three vertexes of an object constituting the scene are created from the shape model. Measuring a dimensional point, measuring the position and orientation of the camera, determining a search area from the three-dimensional point information of the measured vertices of the object and the position information and orientation information of the camera, and determining the determined search. A feature point from an image closest to the region and the scene image is searched to extract feature point candidates, and a side or surface adjacent to the feature point is searched from the shape model to extract side information or surface information. This is a feature point search method for an object that searches for a side or face of an image from the extracted edge information or surface information and the feature point candidates to obtain the most probable feature point.

(2) The feature points searched by the object feature point search method of (1) are associated with a plurality of images photographed at different positions and postures to enhance the search accuracy of the feature points. (3) A recording medium which records a feature point search program of an object for causing a computer to execute the processing procedure of the feature point search method of the object (1).

(4) The processing procedure of the feature point searching method of the above (1) and the feature points searched by the processing procedure are associated with a plurality of images photographed at different positions and postures, and the search accuracy of the feature points is determined. Is a recording medium that records a feature point search program for an object for causing a computer to execute a procedure for increasing the feature point.

(5) A camera for photographing a scene, and means for creating a shape model for describing information on a side including a side and a scene surrounded by the side from an image including the photographed scene;
Means for measuring three-dimensional points of vertices of an object constituting a scene from the shape model; means for measuring the position and orientation of the camera; three-dimensional point information of the measured vertices of the object; Means for determining a search area from position information and orientation information; means for searching for a feature point from an image closest to the determined search area and the scene image to extract feature point candidates; Means for searching for a side or a surface adjacent to a point to extract side information or surface information, and searching for a side or a surface of an image from the extracted side information or surface information and the feature point candidate to be most likely. An object feature point search device provided with a feature point acquiring unit.

(6) There is provided means for increasing the accuracy of searching for the feature points by associating the feature points searched by the object feature point searching device of (5) with a plurality of images photographed at different positions and postures. It is an object feature point search device.

Differences between the features of the present invention and the prior art, that is, the present invention centers on the image plane a three-dimensional point obtained from the shape model based on the shape model containing the error and the position and orientation information of the image sensor. Projection is performed, and the image feature points in the vicinity are treated as candidates for geometric feature points, and the candidates are narrowed down with reference to the shape model as follows.

Image features of sides and faces adjacent to feature points image features of feature points adjacent to sides adjacent to feature points image features around geometric feature points in a plurality of images

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings. In all the drawings, components having the same function are denoted by the same reference numerals, and repeated description will be omitted.

(Embodiment 1) FIG. 2 shows Embodiment 1 of the present invention.
1 is a block diagram showing a schematic configuration of an object feature point search device.

The object feature point searching apparatus according to the first embodiment includes:
As shown in FIG. 2, a computer 100, an image input unit (hereinafter simply referred to as a camera) 200 such as a camera or a video camera, a camera position / posture measuring unit 300 for measuring the position and orientation of the camera 200, and a display And an output device 400 such as a device. The computer 100 has a central processing unit (CPU) 101 and an external storage device (external memory) 102 provided outside the central processing unit (CPU) 101.

Here, the principle of the feature point search processing of the present invention will be described by taking an example of an outdoor cityscape. In order to obtain a high-precision shape model, it is difficult to measure an outdoor landscape or the like, especially in a wide area. Conventionally, when there is an error between the shape model and the position and orientation sensor, it is difficult to search for a feature point.In the present invention, however, a region search is performed on an image for a shape feature point on the assumption that there is an error. Make it applicable to outdoor scenery, etc.

Obtaining an outline of a scene with a scan type laser range finder, and extracting geometric features (vertex, side, surface, etc.) of the range data by observing a change in curvature, and approximating with polygons. To create a shape model. When the horizontal accuracy of the range data is insufficient, the data is corrected using an available digital map.

For example, a GPS (global positioning system) is provided to a camera (image input means 200) for photographing a landscape.
m: Global position determination system) or a position sensor using D-GPS and a posture sensor (camera position / posture measuring means 300) using a gyro or the like, and camera position information based on a trigger signal output at the same time as image capturing. And posture information.

One point of the above shape model is centrally projected on the image plane of the camera based on the position information and attitude information of the camera obtained above. Generally, due to the error of the sensor and the error of the shape model, the center projected point does not coincide with the originally corresponding point of the image, and there is a corresponding point in a certain area (FIG. 3). (See: The point where the three-dimensional point is centrally projected on the image plane does not match the image feature point due to errors inherent in the shape model and the camera position and orientation.)

A circle having a constant radius is taken as a neighborhood corresponding to the estimated value of the error centered on the center-projected point, and an image feature point obtained by extracting an image in the neighborhood circle by edge detection processing is taken as a shape model. (See FIG. 4: it explains that there are three image feature points in the search area in FIG. 3).

Referring to the shape model, it is checked whether an image feature (edge detected from the image) corresponding to a side adjacent to the point of interest exists. Further, the correlation between the shade portion calculated from the shape model and the sun altitude and the brightness change of the image of the portion corresponding to the surface adjacent to the point is examined. By the above determination, the most probable shape feature point candidate is selected (see FIG. 5: FIG. 5 is a diagram showing information for narrowing down feature point candidates obtained from the shape model. The number of sides adjacent to the point is three, and the directions are left, upper right,
Below and the shading for the adjacent plane indicates that the lower right plane is darker).

With the obtained geometric feature points, an image corresponding to the surface of the shape model can be cut out from the scene image and texture-mapped to the shape model.

FIG. 1 is a flowchart showing a procedure for searching for a feature point from an image area by the apparatus for searching for a feature point of an object according to the first embodiment of the present invention.

A feature point search process from an image area by the object feature point search apparatus according to the first embodiment will be described with reference to FIG.

A shape model (S101) describing information (data) of a side including a scene (region of interest) and a surface surrounded by the side is created from an image obtained by capturing a scene with a camera (S1).
01), three-dimensional points of vertices of objects constituting a scene are measured from the created shape model (S102). The position and orientation (direction) of the camera are measured (S10
3). A search area is determined from the measured three-dimensional point information of the vertex of the object and the position and orientation information of the camera (S1).
04). A feature point search is performed from an image closest to the determined search area (S105) and the scene image (S106) (S107), and feature point candidates are extracted (S1).
08).

A side or a surface adjacent to the feature point is searched from the shape model to search and extract side information (data) or surface information (data) (S109), and the extracted side information or surface information (data) is searched. (S110), the side or the surface of the image is searched from the feature point candidates (S111), and the most probable feature point is obtained (S112).

(Algorithm for Creating Shape Model) There are several known methods for creating the shape model. A method for creating the shape model based on distance data by a scanning laser range finder will be described. Algorithms for determining characteristic points based on the curvature of a curved surface are known.

When applied to the outdoor cityscape shown in the first embodiment, the shape is mainly formed by a flat surface.
In order to obtain a general shape that can approximate the shape model in a plane, it is preferable to detect (1) a ridge or a vertex, and (2) a valley or a concave point of the shape model. It is appropriate to use the mean curvature for this purpose.

(Method of selecting three-dimensional point) The shape model is composed of points / sides / surfaces expressed in three dimensions, but is hidden by the surface among the three-dimensional points considered to be photographed in the scene image. Select all except points.

(Method of Obtaining Camera Position and Orientation at the Time of Shooting Scene Image) There are several methods for knowing the camera position and attitude at the time of shooting a scene. When applied to the outdoor cityscape described in the first embodiment, position measurement using GPS is appropriate. The posture of the camera can be either an inertial navigation system using various gyros or a G that is not a relative measurement like a gyro.
Posture survey using PS is appropriate.

(Method of Determining Search Area) It is necessary to convert a shape model into a coordinate system representing a camera position and a posture. In the first embodiment, a rectangular coordinate system such as a national coordinate system defined by the Geographical Survey Institute is appropriate.

There is a known means for using a camera's internal parameters (focal length, pixel size, distortion correction coefficient) and the position and orientation of the camera to create a shape modal to be superimposed on a captured scene image. The vicinity of the radius r of the point at which the three-dimensional point is projected on the scene image is defined as a feature point search area. r is a given parameter, which varies depending on the accuracy of the model and the accuracy of the camera position and attitude.

(Feature Point Search from Image) An edge image is obtained from a search area of a scene image by using a well-known edge extraction algorithm or the like. It is considered that an edge is generated at a vertex / side connecting the vertex / other portion where image brightness is generated. By performing morphology processing / thinning processing on the edge image, it is classified into intersection points / angles / end points / intermediate points to be feature point candidates.

(Agorithm for Creating Side / Surface Data from Shape Model) A three-dimensional point and its neighboring sides / faces are projected onto a scene image using the camera position and orientation and camera internal parameters. By associating the projected shape with the original shape information, a means for obtaining a set of points on which the side connected to the three-dimensional point is projected is prepared.

(Algorithm for edge / surface search from image)
As shown in FIG. 6, in the edge / plane search from the image, correspondence between a feature point candidate classified as an intersection or a corner point and a projected point of a three-dimensional point is represented by three scales. evaluate. (1) Evaluation Based on Distance to Intersection or Corner Point Corresponding to Projected Three-Dimensional Point As shown in FIG. 7, evaluation C1 is evaluated based on a linear distance L between points (C1 = L). (2) Evaluation based on the degree of coincidence between an intermediate point connected to a certain intersection or a corner point and a line on which a side connected to the corresponding three-dimensional point is projected. As shown in FIG. The evaluation is made by dividing the total sum DT of the distances Di on the image plane from the line by the number P of the intermediate points (C2 = DT / P). (3) Inversion / overlap / penalty of no correspondence due to the combination of intersections or corner points and three-dimensional points There are various methods for evaluating the inversion of the position. For example, the magnitudes of the x-axis coordinate values of the points are compared. To determine the forward / reverse. Evaluation is made based on the linear distance Lr between the inverted points.

Further, since the intersection or corner point may fail to be detected or may be excessively detected, the number of intersection points or corner points and the number of three-dimensional points may not be the same. Number of overlapping points W
And the number N without correspondence (Equation 1).

[0040]

(Equation 1)

The overall evaluation function C is calculated as in the following equation (2). The result is shown in FIG.

[0042]

(Equation 2)

In the equation (2), P1, P2, P3, P
31 and P32 are weighting coefficients.

The above evaluation function C is calculated for all combinations of intersection points or corner points and three-dimensional points in image units, and the intersection points or corner points obtained by the smallest combination are regarded as the most probable feature points.

(Embodiment 2) The embodiment 2 of the present invention is a method according to the embodiment 1, in which a plurality of images taken at different positions and postures at different viewpoints are given to improve the accuracy of searching for geometric feature points. It is.

When obtaining the geometric feature points of each image corresponding to a certain three-dimensional point of the shape model, the most appropriate one from the feature point candidates is not limited to one, but the accuracy is given to each candidate. Output. If an image is captured in a short time like a time-series image, images around the same point should have high correlation. The candidate points are clustered based on the correlation between the images around the candidate points, and the sum of the accuracy of the candidate points included in the cluster is calculated. Accuracy can be improved by setting candidate points included in the cluster having the largest sum as the geometric feature points. That is, when images from a plurality of viewpoints are given, the accuracy of the geometric feature point and the accuracy of the shape model can be further increased by searching from each of them.

From the set of geometric feature points with improved accuracy,
Using the “Shape from motion” technique, the three-dimensional point coordinates can be estimated in reverse, and the shape model can be corrected.

Although the present invention has been described in detail with reference to the embodiments (examples), the present invention is not limited to the above-described embodiments (examples), and various modifications may be made without departing from the scope of the invention. It goes without saying that you get it.

[0049]

As described above, according to the present invention,
By performing a high-precision search for geometric feature points from an image, the texture of the image cut by the geometric feature points can be made to correspond to the shape model, or the accuracy of the given polygon model or camera position and orientation can be increased. be able to. Further, when images from a plurality of viewpoints are given, the accuracy of the geometric feature point and the accuracy of the shape model can be further increased by searching from each of them.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a processing procedure for searching for a feature point from an image area by an object feature point search device according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a schematic configuration of an object feature point searching device according to the first embodiment.

FIG. 3 is a diagram showing that a point at which a three-dimensional point is centrally projected on an image plane does not match an image feature point due to an error inherent in the shape model and the camera position and orientation according to the first embodiment. Is shown.

FIG. 4 is a diagram for explaining that three image feature points exist in a search area in FIG. 3;

FIG. 5 is a diagram showing information for narrowing down feature point candidates obtained from the shape model according to the first embodiment.

FIG. 6 is a diagram for explaining an evaluation scale in an algorithm for searching for an edge / plane from an image according to the first embodiment.

FIG. 7 is a diagram for explaining evaluation based on a distance to a point where a three-dimensional point corresponding to an intersection or a corner point according to the first embodiment is projected.

FIG. 8 is a diagram for describing evaluation based on the degree of coincidence between an intermediate point connected to a certain intersection or a corner point and a line on which a side connected to a corresponding three-dimensional point is projected according to the first embodiment.

FIG. 9 is a diagram illustrating a result of calculating an overall evaluation function C of the first embodiment.

FIG. 10 is a diagram for explaining a conventional method of searching for a geometric feature point.

[Explanation of symbols]

100 computer 100, 101 central processing unit (CPU), 102 external memory (external storage device)
200: image input means, 300: camera position / posture measuring means, 400: output device.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Toshiaki Sugimura 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo Japan Telegraph and Telephone Corporation (72) Inventor Isao Miyagawa 3- 192-1 Nishi-Shinjuku, Shinjuku-ku, Tokyo No. Nippon Telegraph and Telephone Corporation (72) Inventor Naoko Uemoto 3-19-2 Nishi-Shinjuku, Shinjuku-ku, Tokyo F-term (reference) 2F065 AA04 AA11 AA12 BB05 CC00 EE00 FF04 FF09 FF21 FF63 FF65 FF67 JJ03 JJ19 JJ26 QQ13 QQ21 QQ27 QQ31 QQ32 QQ38 QQ41 SS02 SS13 5B057 CF01 DA07 DB03 DC05 DC07 DC16

Claims (6)

[Claims] 1. A shape model describing information on a side including a side and a scene surrounded by the side is created from an image obtained by capturing a scene with a camera, and three-dimensional points of vertices of an object constituting the scene are created from the shape model. Is measured, the position and orientation of the camera are measured, and the measured three-dimensional point information of the vertex of the object;
A search area is determined from the position information and the posture information of the camera, a feature point is searched from an image closest to the determined search area and the scene image to extract feature point candidates, and a feature point is extracted from the shape model. A side or a surface adjacent to is searched to extract side information or surface information, and a side or a surface of an image is searched from the extracted side information or surface information and the candidate for the feature point to find a most probable feature point. An object feature point search method characterized by acquiring. 2. An object characterized in that the feature points searched by the object feature point search method according to claim 1 are associated with a plurality of images photographed at different positions and postures to improve the feature point search accuracy. Feature point search method. 3. A procedure for creating a shape model that describes information of a side including a side and a scene surrounded by the side from an image obtained by capturing a scene with a camera, and a step of determining a vertex of an object constituting the scene from the shape model. A procedure for measuring a dimension point, a procedure for measuring the position and orientation of the camera, a procedure for determining a search area from the three-dimensional point information of the measured vertices of the object, and the location information and orientation information of the camera. Searching for a feature point from an image closest to the determined search area and the scene image to extract feature point candidates; and searching for a side or face adjacent to the feature point from the shape model to obtain side information. Alternatively, the computer is caused to execute a procedure of extracting surface information, and a procedure of searching for an edge or a surface of an image from the extracted side information or surface information and the feature point candidate to obtain a most probable feature point. Recording medium storing a feature point search program for an object. 4. A feature point search method according to claim 1, wherein the feature points searched by the process steps are associated with a plurality of images photographed at different positions and postures to improve feature point search accuracy. A recording medium storing a feature point search program for an object for causing a computer to execute the procedure. 5. A camera for photographing a scene, means for creating a shape model describing information on a side including a side and a scene surrounded by sides from an image including the shot scene, and a scene from the shape model Means for measuring the three-dimensional point of the vertex of the object, means for measuring the position and orientation of the camera, three-dimensional point information of the measured vertex of the object,
Means for determining a search area from the position information and attitude information of the camera; means for searching for a feature point from an image closest to the determined search area and the scene image to extract feature point candidates; Means for searching for a side or a face adjacent to a feature point from a model to extract side information or face information; and searching for a side or a face of an image from the extracted side information or face information and the feature point candidate. An object feature point search device comprising means for acquiring a most probable feature point. 6. A means for improving feature point search accuracy by associating feature points searched by the object feature point search device with a plurality of images photographed at different positions and orientations. A feature point search device for an object to be a feature.