JP2003256806A - Omniazimuth image type feature point retrieval processing method and device, and its program and recording medium with its program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly retrieve feature points on time series images by narrowing down the retrieval range of the feature points. <P>SOLUTION: Omniazimuth images photographed by an image input device 10 are obtained by an image obtaining part 21, and the feature points of the initial image are calculated by a feature point calculating part 22, and the orbit curve on the omniazimuth pictures of the feature points is calculated by an orbit data calculating part 25 on the basis of the position/attitude information of the omniazimuth images at the time of photographing, and the retrieval range of the feature points is narrowed down to points on the calculated orbit curve, and small area sub-images in the peripheries of the points on the orbit curve as centers are extracted, and collated with sub-images including the feature points on the initial image as the start points of the retrieval so that the positions of the feature points of each time series image can be decided. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an omnidirectional image type feature point search processing method and apparatus for searching a plurality of images for an arbitrary feature point in space from time series image data acquired by an image input device or the like. The program and the recording medium of the program are related to the fields of computer vision, image measurement, and surveying.

[0002] For example, an omnidirectional camera capable of photographing a wide space is effective as a photographing means for various buildings, and a multi-view omnidirectional image photographed by the omnidirectional camera is used to effectively construct a building shape. Can be measured. In measuring the shape of a building using such an omnidirectional image, feature point search is important.

[0003]

2. Description of the Related Art When measuring an object to be photographed using multi-viewpoint images, it is necessary to extract feature points common to a plurality of images. Conventionally, when a camera with a fixed angle of view is used, the image may be interrupted due to frame-in or frame-out, which makes it difficult to track the complete feature points. Due to this imperfect tracking of feature points, it was not possible to secure a sufficient baseline length, and it was difficult to obtain and restore highly accurate shapes and structures of the imaging target.

FIG. 8 shows an example photographed by a camera having a finite angle of view,
FIG. 9 is a diagram showing an example of frame-in and frame-out in the feature point search.

A camera 120 with a finite angle of view allows an object 120 to be photographed.
When the image is captured, as shown in FIG. 8, in the image 110 acquired from the camera 100, only a part 121 of the object may be imaged, and a specific feature point of the object 120 is image 1
May not appear in 10. Therefore, if a time-series image is acquired by shooting while moving the camera 100 with a finite angle of view, a feature point P1 may appear in the middle of the time-series image as shown in FIG. 9A, or as shown in FIG. 9B. Since the feature point P2 may disappear from the middle of the time series image, the feature point may not be tracked between images. Such frame-in and frame-out are factors that make it difficult to recognize the shape, structure, etc. of the object to be imaged with high accuracy.

On the other hand, an omnidirectional camera or an omnidirectional camera capable of picking up images in all directions at once has been developed.
It became possible to acquire the entire landscape at once. By taking advantage of this to shoot an object, the problems such as frame-in and frame-out can be alleviated. However, the variation of the feature points on the omnidirectional image greatly depends on the shooting position and the distance to the feature points, and thus there is a problem that the search range of the feature points becomes wide.

[0007]

SUMMARY OF THE INVENTION The present invention solves the problem that the search range becomes wide when searching for feature points in an omnidirectional image, resulting in a large processing cost such as time and machine cost. By providing a means for narrowing down the search range of feature points on an omnidirectional image, it is an object to speed up the feature point search and reduce the processing cost of the feature point search.

[0008]

SUMMARY OF THE INVENTION The present invention uses an omnidirectional camera characterized by a wide space projection.
Using the position information (external parameters) of the camera itself, internal parameters (focal length, etc.) peculiar to the omnidirectional camera, and the optical projection of the omnidirectional camera, the target point to be acquired is specified from the omnidirectional image taken. The main feature is to narrow down the range on the locus curve of. That is,
In a time-series image taken by an omnidirectional camera, the trajectory curve of the feature point on the omnidirectional image is calculated based on the position / orientation information of the omnidirectional image at the time of shooting, and the feature point is calculated on the trajectory curve. By narrowing down the search range of points, extracting a sub-image of a small area around the point on the locus curve as a center, and collating with a sub-image including the feature point on the initial image that is the starting point of the search, The position of the feature point in each time series image is determined.

Specifically, according to the present invention, when searching for a feature point in a time-series image captured and acquired using an omnidirectional camera, camera position / orientation information is acquired and synchronized with the acquired time. A feature point is extracted from the first omnidirectional image photographed by taking the image, and the image coordinate value of the feature point is obtained.

Further, another omnidirectional image is photographed at a position different from the position where the first omnidirectional image was photographed, and the position / orientation information of the camera when the image was photographed is acquired.

Next, for each feature point, position / orientation information when the first omnidirectional image is taken, position / orientation information when another omnidirectional image is taken, and omnidirectional camera. The characteristic point locus passing through the image coordinate position of the characteristic point is predicted using the unique internal parameter.

A sub-image of a small area including each feature point in the first omnidirectional image is extracted, and the position of the image coordinate value is moved along the locus for each feature point predicted in the third step. Meanwhile, the sub-image of the small area in the other omnidirectional image is extracted.

Matching is performed between the sub-image of the small area including each feature point in the first omnidirectional image and each sub-image in the small area extracted while moving along the locus relating to the feature point. Then, the image coordinate position on the locus when the correlation value between the sub-images is minimized is set as the position of the feature point in the other omnidirectional images.

As described above, by narrowing down the search range of feature points to a range on a specific locus, it is possible to efficiently search for feature points between omnidirectional images.

Further, after searching the positions of the feature points in the other omnidirectional images, the omnidirectional image is used as the first omnidirectional image, and the feature point position as a result of the search is used as a new search start position. Search for feature points in other omnidirectional images. As a result, it becomes possible to search for feature points sequentially from the time-series image of the omnidirectional image with high accuracy and trace the feature points.

Further, based on the position / orientation information when the first omnidirectional image is acquired and the position / orientation information when another omnidirectional image is acquired, the moving direction of the feature point locus or the movement of the camera. The direction is determined, and the image coordinate value of each feature point in the first omnidirectional image is set as the search start position, and the feature point image position is searched along the moving direction or the traveling direction. As a result, the search direction on the feature point locus can be limited to one direction, and the search range can be further narrowed.

Further, with the searched point as the starting position, the sub-images are extracted in a small region around the starting position, and the point that minimizes the correlation value between the sub-images is searched. As a result, even if there is some error in the position / orientation information of the camera, the corresponding position of the feature point can be accurately obtained.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described with reference to the drawings. In the following, it is assumed that the omnidirectional camera is a camera equipped with a fisheye lens (projection type: y = fθ), and the omnidirectional image is acquired while the camera is mounted on a vehicle and moved.

FIG. 1 is a diagram showing an example of a device configuration for implementing the present invention. The present system includes an image input device (camera) 10 including a fisheye lens unit 11 and an optical adjustment unit 12 as an optical system, a CPU that performs a process of searching for characteristic points of time-series image data obtained by imaging from the optical system, and Image processing device 10 and processing device 20 including a memory
And a position / orientation sensor 30 for detecting the position and orientation of the.

The processing unit 20 includes an image acquisition unit 21 for generating a time-series image based on input data from the image input unit 10, a characteristic point calculation unit 22 for calculating characteristic points in the image, and a position / orientation sensor 30. Position / orientation information acquisition unit 23 that acquires position / orientation information, synchronization control unit 24 that synchronizes the image acquired by the image acquisition unit 21 with the position / orientation information acquired by the position / orientation information acquisition unit 23, each feature A locus data calculation unit 2 that calculates locus data of a characteristic point locus corresponding to a point
5, a correlation value calculation unit 26 that calculates a correlation value between a point on the trajectory data in the image for searching a corresponding point and an original feature point, a corresponding point search unit 27 that searches for a corresponding point from the calculated correlation value,
It has a corresponding point output unit 28 which outputs the obtained corresponding points.
These processing functions are realized by a software program or the like.

As the position / orientation sensor 30, for example, GP
An S (Global Positioning System), a gyro sensor, or the like can be used. Position / posture information acquisition unit 23
Instead of using the position / orientation sensor 30, the operator may input the position / orientation information of the image input device 10 using an input device such as a keyboard.

The coordinate system set for explaining the present embodiment will be described with reference to FIGS. Figure 2 is XY
It is the figure which looked at the three-dimensional coordinate system of Z from the Z-axis direction.
FIG. 6 is a diagram showing a relationship between an object and an image acquisition position when viewed in the horizontal direction on the XY plane.

The three-dimensional coordinate value of the target feature point P is (X
_p , Y _p , Z _p ). In addition, the coordinate value of the lens center of the image input device 10 at the position A is (X _a , Y _a ,
Z _a ), and the coordinate value of the lens center of the image input device 10 at the position B is (X _b , Y _b , Z _b ).

Coordinate values (X _a , Y _a , Z _a ) of the position A,
The coordinate value (X _b , Y _b , Z _b ) of the position B can be determined from the position / orientation information acquired by the position / orientation information acquisition unit 23.

Photographed at position A by the image input device 10.
Omnidirectional image I_AAnd the omnidirectional image taken at position B
I_BAnd Let the two-dimensional coordinate system on these images be
The standard system. Omnidirectional image I of the target feature point P_ATo
Coordinate value (i_a, J_a), And the same omnidirectional image I
_BThe coordinate value of the corresponding position in_b, J_b).
L is an omnidirectional image I_AFrom image center A of (i_a, J_a)
Is the distance to. φ _a, Φ_bIs an omnidirectional image I_A, I
_BFrom the center of (i_a, J_a), (I_b, J_b)
Is the angle between the line segment and the j-axis at_a, Θ_bIs the feature point
Each line segment connecting P and positions A and B, and the XY plane
Is the angle.

Here, in order to simplify the description, position A
It is assumed that there is no rotation in the optical axis direction from the movement to the position B. If there is an optical axis rotation, the optical axis rotation value is calculated from each position information and orientation information, and coordinate conversion considering this rotation component can be easily reflected in the following method.

FIG. 4 is a processing flowchart of this embodiment. When the processing starts, the position / orientation information acquisition unit 2
3 senses the position A by the position / orientation sensor 30 and acquires the coordinate values (X _a , Y _a , Z _a ) of the image input device 10 which is an omnidirectional camera (step S1). When the sensing is unsuccessful, the coordinate value at the position A is sensed until the sensing is successful (step S).
2). At this time, the image acquisition unit 21 simultaneously acquires the omnidirectional image I _A at the position A from the image input device 10 based on the time information obtained from the synchronization control unit 24 such as a timer (step S3). The feature point calculator 22 calculates some feature points in this initial image (step S).
4). As the characteristic points, points corresponding to edge parts are set by analyzing the image. For example, a portion having a large shade difference between adjacent pixels is extracted as an edge, and a portion of the extracted edge point sequence in which the distance from the image center greatly changes is used as a feature point.

Next, the image input device 10 moves to the position B.
In this case, the position / orientation information acquisition unit 23
The coordinate value at position B (X_b，
Y_b, Z _b) Is obtained (step S5). Lost in sensing
If lost or the coordinate value of position B is the same as the coordinate value of position A
If it is the same, the coordinate value at position B (X_b, Y_b，
Z _b) Is repeated (step S6).

At the same time, the image acquisition unit 21 obtains the omnidirectional image I _B at the position B from the image input device 10 (step S
7).

Next, in step S4, the omnidirectional image I
Feature points set on _A is omnidirectional image I a search range for exploring what point corresponding to the on _B, the omnidirectional image I _A feature point trajectory based on feature points at positions A Are narrowed down by predicting (step S8). Regarding the feature point search by predicting the feature point locus, referring to FIG. 5 and FIG.
It will be described in detail later.

If the feature point locus is predicted, the omnidirectional image
I_AFeature points and omnidirectional image I_BOn the feature point locus at
Do not calculate the difference between sub-images containing each of the points
The correlation value is calculated by what (step S9). In addition, this
It is desirable that the loci of feature points have a certain width.
Yes. That is, the image coordinate value of one point of the feature point locus is
(I _x, J_x), Then (i_x± Δx, j_x
Points of ± Δy) are regarded as points on the characteristic point locus. Note that Δ
x and Δy are preset constant values.

The correlation value is calculated in step S9 while moving the point on the characteristic point locus, and the point on the omnidirectional image I _B having the minimum correlation value is searched for. If the correlation value is found that the minimum, that point as the corresponding position of the characteristic point on the omnidirectional image I _B, and outputs the search result (step S10).

It is judged whether all the feature points on the omnidirectional image I _A have been processed in steps S8 to S10 (step S11). If there is an unprocessed feature point, the next feature point is processed in step S8. ~ The process of S10 is repeated.

After that, it is determined whether or not the acquisition of the time series images from the image input device 10 is completed (step S1).
2) In case of termination, the processing is terminated. When continuing, the current position B and the image I _B are updated with the position A and the image I _A , respectively, in order to track the same feature point in the omnidirectional images sequentially acquired as the time series images (step S13), the processes of steps S5 to S12 are similarly repeated for the new position B and the omnidirectional image at the position B, and the corresponding points of the feature points are acquired one after another.

The method of searching the characteristic points by obtaining the characteristic point locus will be described in more detail. FIG. 5 is a diagram for explaining a method of calculating a locus curve of a feature point locus, FIG.
FIG. 6 is a flowchart showing a procedure for obtaining a characteristic point locus and searching for a characteristic point.

In FIG. 5, A, B, P, φ _a ,
( _Ia , _ib ), L are the same as the point, angle, and distance described in FIG. L _x (φ _x ) is the locus curve of the locus of feature points,
phi _x angle variable, (i _x, j _x) represents the point on the feature point trajectory.

[0037] Get the omnidirectional image I _A at position A (step S20), the feature point in the omnidirectional image _{I A (i a, j a} )
Is calculated (step S21). Next, the distance L from the image center of the feature point is calculated (step S22). When the coordinate values of the image center and (I, J), the distance L is, L = {(I-i a) 2 + (J-j a) 2} obtained by ^1/2 of formula.

Next, the angle φ _a is calculated (step S2).
3). The angle φ _a is calculated by the following equation.

[0039] ^{_{φ a = tan -1 {(I}} -i a) / (J-j
_a )} φ _x is initialized to 0, and the minimum correlation value C _min is initialized to a value indicating infinity (∞) (step S24).

Assuming a camera using an equidistant projection fisheye lens (y = fθ) as the fisheye lens unit 11 of the image input device 10, the omnidirectional projection formula is “y = fθ” (f is the focal length), As is clear from FIGS. 2 and 5, the locus curve L _x (φ _x ) of the feature points is as follows.

L _x (φ _x ) = f × tan ⁻¹ {(tan (L
/ F) sin φ _a ) / sin φ _x } 0 <φ _x <π That is, at any position A, the feature point P (X _p ,
Y _p, the Z _p) is the coordinates of the points to be projected onto the omnidirectional image I _A and (i _a, j _a), the omnidirectional image I on _B to be captured when translated from the position A to the position B When the coordinates of the corresponding points in the (i _b, j _b), corresponding points (i _b, j _b) is
The distance L from the image center at an angle φ _x of 0 to π
It will be projected on the curve represented by the formula of _x .

In the present embodiment, the locus curve L_x(Φ_x)
Is calculated (step S25), and the characteristic point at the position A is calculated.
Omnidirectional image I at corresponding position B_BCoordinate of the upper point
(I _b, J_b) Is φ_xValue of L and L_x(Φ_x)
In the polar coordinate system consisting of and
Search by narrowing down the search range.

Therefore, the point (i _x , j _x ) on the locus curve at the angle φ of the omnidirectional image I _B is calculated (step S2).
6), the pixel value (R _x , G _x , of the omnidirectional image I _B at that point)
B _x ) is acquired (step S27). Here, RGB values are used as pixel values.

Next, 8 of the points on the locus curve at the angle φ are
Pixel value of neighboring pixels (R_n, G_n, B _n) (N = 1, 2,
..., 8) as a sub-image of a small area for calculating the correlation value
(Step S28), the characteristic points at position A
Pixel value (R_a, G_a, B_a) And the pixel values of its 8 neighboring pixels
(R_ai, G_ai, B_ai) (I = 1, 2, ..., 8)
Sub image and sub including the point on the locus curve at position B
The difference from the image is the correlation value C_xAs calculated by
Yes (step S29). C_x= {(R_x-R_a)²+ (G_x-G_a)²+ (B_x
-B_a)²}^1/2+ {(R₁-R_a1)²+ (G₁−
G_a1)²+ (B₁-B_a1)²}^1/2+ {(R₂-R_a2)
²+ (G₂-G_a2)²+ (B₂-B_a2)²}^1/2+
{(R₃-R_a3)²+ (G₃-G_a3)²+ (B₃−
B_a3)²}^1/2+ {(R_Four-R_a4)²+ (G_Four-G_a4)
²+ (B_Four-B_a4)²}^1/2+ {(R_Five-R_a5)²+
(G_Five-G_a5)²+ (B_Five-B_a5)²}^1/2+ {(R₆
-R_a6)²+ (G₆-G_a6)²+ (B₆-B_a6)²}
^1/2+ {(R₇-R_a7)²+ (G₇-G_a7)²+ (B₇
-B_a7)²}^1/2+ {(R₈-R_a8)²+ (G₈−
G_a8)²+ (B₈-B_a8)²}^1/2 Here, a 9-pixel sub-image is used as the sub-image to be collated.
Although used, the size of the sub-image is not limited to 9 pixels,
You may use the sub-image of size.

[0045] If the calculated correlation values C _x, it compares the C _min and magnitude (step S30), the correlation value C _x is smaller than C _min, changing the C _min of the value of C _x (step S31) .

The above steps S26 to S31 are repeated while changing the angle φ from 0 to π by Δφ (steps S32 and S33). If processing is performed until the angle φ becomes π, a point (i _x , j _x ) having coordinates such that C _{min at} that time is a point (i
_a , j _a ) corresponding point (i _b , j) in the omnidirectional image I _B
b ) and output the result (step S3)
4).

By repeating the above processing for the number of characteristic points calculated at the position A, the corresponding point at the position B of the characteristic point at the position A can be obtained.

In the above processing, the locus curve L _x (φ
_x ) The angle φ when searching for a point above was changed from 0 to π, but the moving direction of the point in the feature point locus or the progress of the image input device due to the change in position when the omnidirectional image was acquired. The direction is determined, and the feature point image position is searched along the moving direction or the advancing direction with the angle φ _a of the point indicating the feature point in the omnidirectional image I _A at the position A on the feature point trajectory as the search start position. Therefore, the search range is 0 <φ <
Instead of π, it can be further narrowed to the range of 0 <φ <φ _a or φ _a <φ <π.

As the internal parameter peculiar to the image input apparatus, the focal length f of the lens is used to calculate the locus curve, but in addition to this, the offset of the optical center, the distortion coefficient of the lens, or the like can be used. When these internal parameters are used, the acquired image may be corrected or coordinate conversion may be performed according to the values, and since the processing is a known technique, detailed description thereof is omitted here.

The above processing by the processing device 20 can be realized by a computer and a software program, and the program is stored in an appropriate recording medium such as a computer-readable portable medium memory, semiconductor memory, hard disk or the like. Then, the computer can execute the program by reading it from there. Also,
The program can be downloaded from another computer via a communication line, and can be installed and executed.

FIG. 7 shows the evaluation result of the experimental example according to the present embodiment. In order to verify the effectiveness of the present embodiment, an omnidirectional camera was mounted on a vehicle and an experiment was conducted on omnidirectional images (2024 × 2044 pixels) taken at 10 to 20 m intervals. The evaluation was performed based on the difference between the position of the feature point searched by this search method and the position of the feature point searched manually by the operator.

[0052] In FIG. 7, the horizontal axis is the difference angle between phi _a and phi _b (the value of the correlation values C _x is the smallest such phi _x), vertical axis, the search point according Search Method Is the distance between and the point searched manually. From this result, it was confirmed that the search can be performed even when the distance between the feature point and the shooting point is short, especially when the feature point has a large temporal variation. From the above, it was confirmed that this search method is effective for feature point search.

[0053]

As described above, according to the present invention,
In a time-series image taken by an omnidirectional camera, the trajectory curve of the feature point on the omnidirectional image is calculated based on the position / orientation information of the omnidirectional image at the time of shooting, and the feature point is calculated on the trajectory curve. By narrowing down the search range of points, extracting a sub-image of a small area around the point on the locus curve as a center, and collating with a sub-image including the feature point on the initial image that is the starting point of the search, By determining the position of the feature point in each time-series image, the search range can be limited when searching for the feature point in the omnidirectional image, and the processing speed and processing cost can be reduced by speeding up the search. It will be possible.

Further, since the search range is limited,
Since it is possible to prevent sub-images from accidentally matching and causing a search error in a region where there is no relation between time-series images, the search accuracy is improved.

The present invention targets all stationary objects such as ordinary houses, urban buildings, natural trees, etc.
It can be used when searching for feature points in those time-series images, and has the advantage that the measurement processing from images of these stationary objects is simpler and faster than in the past.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of a device configuration for implementing the present invention.

FIG. 2 is a diagram illustrating a coordinate system set in this embodiment.

FIG. 3 is a diagram showing a relationship between an object and an image acquisition position when viewed in a horizontal direction on an XY plane.

FIG. 4 is a processing flowchart of the present embodiment.

FIG. 5 is a diagram illustrating a method of calculating a locus curve of a characteristic point locus.

FIG. 6 is a flowchart showing a procedure for searching a characteristic point by obtaining a characteristic point locus.

FIG. 7 is a diagram showing an evaluation result of an experimental example according to the present embodiment.

FIG. 8 is a diagram showing an example captured by a camera having a finite angle of view.

FIG. 9 is a diagram showing an example of frame-in and frame-out in a feature point search.

[Explanation of symbols]

10 Image input device (camera) 11 Fisheye lens 12 Optical adjustment unit 20 Processor (CPU / Memory) 21 Image acquisition unit 22 Feature point calculator 23 Position / posture information acquisition unit 24 Synchronous control unit 25 Locus data calculation unit 26 Correlation value calculation unit 27 Corresponding point search unit 28 Corresponding point output section 30 Position / orientation sensor

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kaori Wakabayashi             2-3-1, Otemachi, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation (72) Inventor Tomohiko Arikawa             2-3-1, Otemachi, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation F-term (reference) 5B057 BA12 DA06 DB02 DB06 DB09                       DC05 DC08 DC16 DC25 DC34                 5L096 AA02 AA06 CA18 FA06 FA09                       FA34 FA62 FA66 FA67 FA69                       GA08 GA17 GA40 HA05 HA08

Claims (7)

[Claims] 1. A feature point search processing method for a time series image obtained by capturing an image in all directions at once using an image input device, wherein position / orientation information of the image input device is obtained, Different from the first step of extracting a feature point from the first omnidirectional image captured in synchronization with the acquired time and obtaining the image coordinate value of the feature point, and the position of capturing the first omnidirectional image A second step of capturing another omnidirectional image at a position and acquiring position / orientation information of the image input device when the image is captured, and the first step.
Using the position / orientation information when the omnidirectional image of the image is captured, the position / orientation information when the other omnidirectional image is captured, and the internal parameter unique to the image input device, the first step In the third step of predicting a feature point locus passing through the image coordinate position of the feature point obtained in step 3, and moving the position of the image coordinate value along the feature point locus predicted in the third step, A fourth step of extracting a sub-image of a small area in the omnidirectional image, a sub-image of a small area including the feature points in the first omnidirectional image, and the feature points obtained in the fourth step The image coordinate position on the feature point locus when the correlation value between the sub-images is minimized by performing matching with each sub-image in the extracted small area while moving along the locus. The corresponding positions of the feature points in the omnidirectional image and Fourth omnidirectional image type feature point search processing method characterized by a step that. 2. The omnidirectional image type feature point search processing method according to claim 1, wherein after searching the position of the feature point in the other omnidirectional image, the omnidirectional image is used as the first omnidirectional image. An omnidirectional image type feature point search processing method characterized in that a feature point position in the other omnidirectional image is searched using the feature point position of the searched result as a new search start position. 3. The omnidirectional image type feature point search processing method according to claim 1 or 2, wherein the position / orientation information when the first omnidirectional image is acquired and the other omnidirectional image From the position / orientation information obtained when is obtained, the moving direction in the feature point locus or the traveling direction of the image input device is determined, and the image coordinate value of the feature point in the first omnidirectional image is set as the search start position. An omnidirectional image type feature point search processing method characterized by searching a feature point image position along the moving direction or the traveling direction. 4. The omnidirectional image type feature point search processing method according to claim 1, claim 2 or claim 3, wherein the searched point is set as a start position and a sub-image is set in a small area around the start position. Omnidirectional image type feature point search processing method characterized by extracting a point that minimizes the correlation value between sub-images. 5. An apparatus for searching a characteristic point in a time series image acquired by capturing an image in all directions at once using an image input apparatus, and means for acquiring position / orientation information of the image input apparatus. And means for acquiring an omnidirectional image taken in synchronization with the time when the position / orientation information of the image input device is acquired, and a feature point extracted from the acquired first omnidirectional image, and an image of the feature point Means for calculating coordinate values, position / orientation information when the first omnidirectional image is photographed,
Using the position / orientation information when the omnidirectional image other than the first omnidirectional image is acquired and the internal parameters unique to the image input device, a characteristic point locus passing through the image coordinate position of the characteristic point is calculated. Prediction means, a sub-image of a small area including the feature points extracted from the first omnidirectional image, and image coordinates along the feature point trajectory predicted from the omnidirectional image other than the first omnidirectional image The image coordinate position on the feature point locus when the correlation value between the sub-images is minimized by performing matching with each sub-image of the extracted small area while moving the position of the value. An omnidirectional image type feature point search processing device, which is provided with means for setting a corresponding position of a feature point therein. 6. An omnidirectional image feature point search processing program for causing a computer to execute the omnidirectional image feature point search processing method according to any one of claims 1 to 4. 7. An omnidirectional image type recording a program for causing a computer to execute the omnidirectional image type feature point search processing method according to any one of claims 1 to 4. A recording medium for a feature point search processing program.