JP2007292657A - Camera motion information acquiring apparatus, camera motion information acquiring method, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera motion information acquiring apparatus for automatically finding a three-dimensional position and a posture of a camera in all images of a long image sequence taken by a time-series image inputting apparatus such as a video camera when the images are taken. <P>SOLUTION: The camera motion information acquiring apparatus is provided with a moving/observed image sequence acquiring means 11 for acquiring the image sequence observed by the moving image inputting apparatus, a moving/observed image sequence dividing means 12 for dividing the acquired moving/observed image sequence into a plurality of sub-image sequences as a plurality of the images are overlapped between the sub-image sequences, a coordinate system integrating means 13 for integrating different coordinate systems between the sub-image sequences into a common global coordinate system, and a camera motion re-estimating means 14 for re-estimating a camera motion in the common global coordinate system. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to a method and apparatus for acquiring camera motion information representing the three-dimensional position and orientation of a camera when each image of a moving observation image sequence obtained by a time-series image input device such as a video camera is captured. Is.

  Estimating camera motion information from an image sequence observed by a moving image input device (hereinafter referred to as a moving observation image sequence) is used in various fields such as 3D model restoration, object recognition, robot navigation, and mixed reality. Application to is possible. A typical method is a factorization method. A feature point generated on an arbitrary image in the moving observation image sequence is associated with all other images in the moving observation image sequence (hereinafter referred to as feature point tracking), and the shape of the object to be imaged is obtained by linear solution. And camera motion information can be restored at the same time. For example, in the following Non-Patent Document 1, camera motion information is acquired robustly with a perspective projection camera model.

The bundle adjustment related to the present invention is described in Non-Patent Document 2 below, and the quaternion is described in Non-Patent Document 3 below.
S. Christy and R.C. Horaud “Euclidean shape and motion from multiple perspectives by affine iterations”, IEEE Transactions on Pattern Analysis in 1996. B. Triggs, P.M. McLauchlan, R.M. Hartley and A.M. Fitzgibbon. Bundle Adjustment-A modern synthesis. Vision Algorithms '99, LNCS 1883, pp. 298-372, 2000. B. K. P. Horn. Closed-form solution of absolute orientation using unit quotas. Journal of the Optical Society of America A, vol. -4, pp. ~ 629--642, 1987.

  However, in the factorization method, feature points must be associated with all the images in the moving observation image sequence. In a moving observation image sequence (hereinafter referred to as a long sequence image sequence) having a long moving distance, it is impossible to estimate a camera motion because a feature point frame out occurs.

  The present invention has been made in order to solve the above-described problems of the prior art, and the object thereof is all images of a long sequence image sequence taken by a time-series image input device such as a video camera. Is to provide a camera motion information acquisition device, a camera motion information acquisition method, and a recording medium capable of automatically obtaining the three-dimensional position and orientation of the camera when the image is taken.

  In order to solve the above problems, the present invention obtains a moving observation image sequence using a time-series image input device such as a video camera, and processes this to obtain a camera at the time of capturing each image of the moving observation image sequence. A method and apparatus for acquiring camera motion information representing the three-dimensional position and orientation of the image capturing device, means for acquiring a moving observation image sequence, means for dividing the acquired movement observation image sequence into a plurality of sub-image sequences, Means for independently estimating camera motion information using each divided sub-image sequence, and camera motion independently estimated for each sub-image sequence using the estimated camera motion information of each sub-image sequence Means for integrating information into a common world coordinate system, and means for re-estimating the integrated camera motion information by optimizing the integrated camera motion information with a whole moving observation image sequence To do.

  That is, the camera motion information acquisition device according to claim 1 acquires camera motion information representing the three-dimensional position and orientation of the camera at the time of image capture from an image sequence observed by the moving image input device. A moving observation image sequence acquisition unit that acquires an image sequence observed by a moving image input device and an acquired moving observation image sequence while overlapping a plurality of images between sub image sequences. A moving observation image sequence dividing unit that divides the image into a plurality of sub image sequences; a coordinate system integrating unit that integrates coordinate systems different between the sub image sequences into a common world coordinate system; and a common world coordinate system. The camera motion information re-estimating means for re-estimating the camera motion is provided.

  Further, the camera motion information acquisition device according to claim 2 is the camera motion information acquisition device according to claim 1, wherein the moving observation image sequence dividing means uses the result of feature point tracking to sub-sequence the image sequence. It is characterized in that a determination is made, a measurement matrix is created using the result of tracking the feature points, and camera motion is estimated by a factorization method.

  The camera motion information acquisition device according to claim 3 is the camera motion information acquisition device according to claim 1, wherein the moving observation image sequence dividing unit uses the result of the feature point tracking to sub-sequence the image sequence. It is characterized in that the camera motion is estimated by determining and sequentially performing projection restoration using the feature point tracking result.

  The camera motion information acquisition device according to claim 4 is the camera motion information acquisition device according to any one of claims 1 to 3, wherein the coordinate system integration unit performs stereo processing using overlapping frames between sub-image sequences. A three-dimensional corresponding point table is created by using the three-dimensional corresponding point table, a coordinate transformation matrix between sub-image sequences is calculated using the three-dimensional corresponding point table, and the coordinate system is integrated.

  The camera motion information acquisition method according to claim 5 is a camera motion information acquisition method for acquiring camera motion information representing a three-dimensional position and orientation of a camera at the time of image capture from an image sequence observed by a moving image input device. A moving observation image sequence acquisition unit that acquires an image sequence observed by a moving image input device; and a moving observation image sequence dividing unit converts the acquired movement observation image sequence into A dividing step of dividing a plurality of images between image sequences while dividing them into a plurality of sub-image sequences, and a integrating step of integrating coordinate systems different between the sub-image sequences into a common world coordinate system. And re-estimating means of camera motion information comprises re-estimating camera motion in the common world coordinate system.

  The camera motion information acquisition method according to claim 6 is the camera motion information acquisition method according to claim 5, wherein the dividing step determines a sub-image sequence using a result of feature point tracking; Create a measurement matrix using the feature point tracking result, estimate the camera motion by a factorization method, optimize the estimated camera motion, and whether or not the division of the moving observation image sequence is finished It has a step of determining and a step of determining the number of overlapping frames.

  The camera motion information acquisition method according to claim 7 is the camera motion information acquisition method according to claim 5, wherein the dividing step determines a sub-image sequence using a result of feature point tracking; It is determined whether to perform projection restoration sequentially using the feature point tracking result to estimate camera motion, to optimize the estimated camera motion, and to divide the moving observation image sequence And a step of determining the number of overlapping frames.

  The camera motion information acquisition method according to claim 8 is the camera motion information acquisition method according to claims 5 to 7, wherein the integration step is a three-dimensional process by stereo processing using overlapping frames between sub-image sequences. The method includes a step of creating a corresponding point table, and a step of calculating a coordinate transformation matrix between the sub-image sequences using the three-dimensional corresponding point table and integrating the coordinate system.

  A recording medium according to claim 9 is a computer-readable recording medium in which a program for causing a computer to execute the camera motion information acquisition method according to any of claims 5 to 8 is recorded.

  In the above configuration, there is an overlapping portion in the divided sub-image sequence, and the relationship between the three-dimensional corresponding points is determined using this overlapping portion. For this reason, camera motion information can be obtained by converting to a world coordinate system based on one local coordinate system.

  According to the present invention, it is possible to acquire camera motion information with a unified coordinate system from a long sequence image sequence, which has been impossible in the past and is impossible to track feature points in all frames. Using the camera motion information obtained in this way, a wide-range 3D model can be restored stably and accurately using a 3D restoration method from various images such as stereo and silhouette methods. Is possible.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. FIG. 1 shows the configuration of a camera motion information acquisition apparatus according to an embodiment of the present invention. The camera motion information acquisition apparatus of the present invention includes a movement observation image sequence acquisition unit 11, a movement observation image sequence division unit 12, a coordinate system integration unit 13, and a camera motion information re-estimation unit 14.

  The moving observation image sequence acquisition unit 11 is a unit that acquires time-series image data while moving the image input device. As an example, a walk shooting with a handheld camera or a camera attached to a car can be considered.

  The moving observation image sequence dividing means 12 divides the moving observation image sequence into a plurality of sub-image sequences while overlapping a plurality of images between the sub-image sequences, and at the same time, using each sub-image sequence, It is a means for estimating the camera motion expressed in a coordinate system unique to the image sequence.

  The coordinate system integration unit 13 is a unit that integrates camera motion information estimated in each sub-image sequence into a common world coordinate system using overlapping image portions of adjacent sub-image sequences.

  The camera motion information re-estimating means 14 is a means for optimizing the camera motion information unified in the world coordinate system using all the images in the moving observation image sequence and re-estimating the camera motion information.

  Next, exemplary embodiments of the present invention will be described in detail. An object of the present invention is to acquire the three-dimensional position and orientation of a camera at the time of photographing each frame constituting a moving observation image sequence from a moving observation image sequence observed by an image input device such as a moving camera.

  In the present embodiment, an example will be described in which a time-series image sequence is captured by a moving video camera as shown in FIG. 2 and the three-dimensional position and orientation of the camera at the time of capturing each image are estimated.

  The camera internal parameters of the video camera used in this embodiment are calibrated in advance.

  FIG. 3 is a flowchart showing the operation of this embodiment. When the process is started, the movement observation image sequence acquisition unit 11 acquires the movement observation image sequence N observed while moving the camera (step S201).

  Next, the moving observation image sequence dividing means 12 divides the moving observation image sequence N into K sub-image sequences while overlapping the adjacent sub-image sequences as shown in FIG. 4 (step S202). Step S202 of this division process is executed along the flowchart shown in FIG.

  In FIG. 5, first, the process is started by setting n representing the serial number of the sub-image sequence to n = 1.

  Next, the image constituting the sub image sequence #n is determined (step S301).

  The number of images constituting each sub-image sequence requires a larger number of images from the viewpoint of stable camera motion estimation. However, if the number of images is too large, feature points cannot be tracked due to disappearance of feature points. . Therefore, it is desirable to determine the number of images in each sub-image sequence as the number of images that can stably perform feature point tracking. For example, it can be performed by the following procedure.

  The feature point tracking is to obtain the image coordinate value of the feature point generated in an arbitrary frame in the image sequence and the corresponding image coordinate value in all other images. Feature point tracking tracks feature points detected in a certain image in both the time-series direction and the time-series reverse direction.

  First, in order to select an image for detecting feature points, selection is performed in order from the second image in time series, feature points are detected in each image, and the first image in the sub-image sequence in the time series reverse direction. Trace feature points. When the detected feature point fails to be tracked for a certain threshold or more, the selection of a new image is stopped, and the image selected before that is adopted as an image for detecting the feature point.

  Next, the same feature point is tracked in the time series direction, and the image immediately before the time series of the image that failed to be tracked for a certain threshold value or more is determined as the last image in the sub-image sequence.

  By doing so, the sub-image sequence can be determined as the maximum number of images capable of stable feature point tracking.

  Next, the camera motion when each image of the determined sub-image sequence is photographed, that is, the three-dimensional position (xi, yi, zi) and orientation (φi, θi, γi) of the camera in the i-th frame of the sub-image sequence. Obtained (step S302).

  Hereinafter, in the present embodiment, an example using a factorization method for camera motion estimation will be described. However, it is obvious that this can be realized even by using sequential projection restoration or the like.

  The factorization method is applied using the result of the feature point tracking performed in step S301. First, a measurement matrix is created from the result of tracking feature points. The measurement matrix is a matrix of F rows and N columns expressed by the following equation (1), where F is the number of frames in the sub-image sequence and N is the number of feature points.

However, (u _ij , v _ij ) represents the image coordinate value of the j-th feature point in the i-th frame.

  In the factorization method, the three-dimensional position of each feature point and the camera motion (three-dimensional position (xi, yi shown in FIG. 4) are obtained from the measurement matrix represented by Equation (1) by, for example, singular value decomposition. , Zi) and posture (φi, θi, γi)). For example, if the method of Non-Patent Document 1 is used, the camera motion can be estimated robustly with a perspective projection model.

  Next, the estimated camera motion is optimized (step S303).

  By performing the optimization using the camera motion and the three-dimensional position of the feature point estimated in step S302 as an initial solution, a more accurate camera motion can be obtained. This can be realized, for example, by well-known bundle adjustment (Non-patent Document 2).

  Next, it is determined whether or not the division is complete (step S304). If the currently processed sub-image sequence has reached the final frame of the moving observation image sequence N, no further division is necessary, and the process ends. On the other hand, if a frame that has not yet been processed remains, the process proceeds to step S305.

  In step S305, the number of overlapping frames between adjacent sub-image sequences is determined. This overlapping frame is used to integrate different camera motions in the coordinate system estimated in each sub-image sequence. As will be described later, this integration uses a three-dimensional point group restored by stereo processing. Therefore, it is important that the three-dimensional point group is accurately restored by stereo processing using overlapping frames.

  The accuracy of the three-dimensional point group using the stereo processing is good as the distance connecting the three-dimensional positions of the camera optical center when the pair of images to be used, that is, the base line is longer, is restored. However, if the baseline is too long, stereo processing itself becomes difficult. Therefore, the number of overlapping frames is determined using the length of this baseline.

  The result of optimizing the camera motion obtained in step S303 is used. Assume that the three-dimensional position of the camera when the last frame of the sub-image sequence is captured is (Xf, Yf, Zf). In the time series, frames are selected in reverse time series from the frame before the last frame, and the baseline distance B is set to the following using the three-dimensional position (Xs, Ys, Zs) of the camera when the selected frame is photographed. (2).

  For example, a certain threshold value T is prepared, and when B exceeds T for the first time, the frame from the last frame to that frame is determined as an overlapping frame. The threshold value T can be determined as an arbitrary value empirically.

  When the overlapping frame is determined, the sub image sequence number n is set to n = n + 1 to determine the next sub image sequence, and the process returns to step S301. The moving observation image sequence N is divided by repeating the processing described above.

  As described above, after the movement observation image sequence dividing process is completed, the coordinate conversion matrix between the sub-image sequences is calculated by the coordinate system integration unit 13 and the coordinate system is integrated into a common world coordinate system.

  First, using the overlapping frame portion, a three-dimensional correspondence point table is created that represents the correspondence relationship in which the same three-dimensional point is represented by two coordinate systems derived between adjacent sub-image sequences (step S203).

  In this embodiment, stereo processing is used to create this three-dimensional corresponding point table. The reference image and comparison image used for stereo processing can be arbitrarily selected from overlapping frames. For example, the first frame and the last frame of the overlapping frame portion can be used as a reference image and a comparison image, respectively. By performing a stereo process, each pixel (u, v) of the reference image and each pixel (u ′, v ′) of the comparison image are associated to obtain corresponding point data.

  Next, the three-dimensional points are restored using the corresponding point data. Three-dimensional point restoration is performed using two camera motion data estimated from two adjacent sub-image sequences including overlapping frames for one corresponding point. That is, a three-dimensional point represented by two coordinate systems is calculated for each pixel (u, v) of the reference image.

  In the 3D point restoration, if the projection matrix P of the reference image is P and the projection matrix P ′ of the comparative image is the projection matrix P, the projection matrix P is a 3 × 4 matrix, and the estimated camera motion (3D position (x, y, z), posture (φ, θ, γ)) and camera internal matrix A can be calculated as in the following equation (3). That is,

  And each element of the A matrix is known by camera calibration.

  First, the projection matrix P of the reference image and the projection matrix P ′ of the comparison image are calculated by the equation (3).

  Next, the three-dimensional point is restored using the corresponding point data and the projection matrices P and P ′. Assuming that the image coordinates (u, v) of the reference image correspond to the image coordinates (u ′, v ′) of the comparison image, the three-dimensional point M = (X, Y, Z) is expressed by the following equation (4). Is restored. That is,

Pij and p′ij represent the elements of i rows and j columns of P and P ′, and B ⁺ is a pseudo inverse matrix of the B matrix.

  By using two camera motions, a three-dimensional point represented by two coordinate systems can be obtained. By performing this for all the pixels of the reference image, a three-dimensional corresponding point table as shown in FIG. 6 can be created.

  The above processing is performed on overlapping frames between all adjacent sub-image sequences. Next, in order to re-estimate the integrated camera motion using the obtained 3D corresponding point table, the 3D feature points obtained by the factorization method in step S201 and all frames of the moving observation image sequence Are integrated into a common coordinate system (step S204).

  The common world coordinate system can be the local coordinate system of any sub-image sequence. For example, the local coordinate system of the center sub-image sequence on the time series can be a common world coordinate system. In the following, as shown in FIG. 7, the local coordinate system of the central sub-image sequence in the time series is a common world coordinate system, and the numbers of the sub-image sequences are W and Ti are coordinate transformation matrices. If i <W, Ti Is a matrix for converting from the sub-image sequence number i to the coordinate system of the sub-image sequence number i + 1. If i ≧ W, Ti is a matrix for converting from the sub-image sequence number i + 1 to the coordinate system of the sub-image sequence number i.

  Such a matrix Ti can be calculated, for example, by a method using a quota anion described in Non-Patent Document 3, using the three-dimensional corresponding point table obtained in step S203. The matrix Ti is calculated between all adjacent sub-image columns.

  Next, using the obtained Ti, the three-dimensional points of the feature points obtained by the factorization method in each sub-image sequence in the common world coordinate system are converted into the world coordinate system. This can be calculated using the following equation (5).

However, (Xi, Yi, Zi) is the three-dimensional coordinate value of the feature point calculated by the factorization method from the sub-image sequence number i, (Xw, Yw, Zw) is (Xi, Yi, Zi) in the world coordinate system It is a three-dimensional coordinate value represented by. This conversion is performed on all three-dimensional points obtained by the factorization method in all sub-image sequences.
Similarly, the projection matrix expressed in the world coordinate system is also calculated. This can be calculated by the following equation (6).

  Here, Pi represents the projection matrix of each image of the sub-image sequence number i, and Pw represents the projection matrix converted and represented in the world coordinate system. This conversion is performed on the projection matrices of all the images in each sub-image sequence. It is possible to convert the projection matrix of the 3D point of the feature point and all images of the moving observation image sequence used to estimate the camera motion for each sub-image sequence by the above processing into a common world coordinate system. it can.

  Finally, camera motion information re-estimating means 14 re-estimates the camera motion unified in the world coordinate system using the obtained three-dimensional coordinate values of the feature points expressed in the common world coordinate system (step S205).

  The re-estimation of the camera motion can be performed by bundle adjustment as described in step S302. Using the three-dimensional coordinate values and the projection matrix of the feature points expressed in the common world coordinate system obtained in step S204 and the measurement matrix obtained by tracking the feature points in each sub-image sequence in step S202, Optimization is performed so as to minimize the maximum projection error of the three-dimensional coordinate value, and this is used as camera motion information integrated into the world coordinate system.

  In this embodiment, an example in which the factorization method is used for estimating the camera motion in the moving observation image sequence dividing unit 12 has been described. For example, it is possible to use sequential projection restoration.

  In the present embodiment, the coordinate system integration means 13 performs integration using quaternions using a three-dimensional point group restored by stereo processing. However, the present invention is not limited to this, and a simple least square approximation or the like is used. The conversion matrix may be estimated as follows.

  The camera parameters and intermediate three-dimensional coordinates are configured to be stored in a memory, for example.

  It should be noted that a part or all of the functions of each means in the apparatus shown in FIG. 1 can be configured by a computer program and that the program can be executed using the computer to realize the present invention, as shown in FIG. Needless to say, the processing procedure can be configured by a computer program, and the program can be executed by the computer. A program for realizing the function by the computer can be recorded on a computer-readable recording medium such as an FD, It can be recorded on an MO, ROM, memory card, CD, DVD, removable disk and stored or distributed. It is also possible to provide the above program through a network such as the Internet or electronic mail.

(Example)
Hereinafter, a result of processing a moving observation image acquired from one moving video camera using the above-described camera motion information acquisition apparatus will be described using a processing flow shown in FIG.

  First, when the process is started, a moving observation image sequence is acquired (step S201). FIG. 8 is a part of an example of the acquired moving observation image sequence (801 to 806). This is obtained by taking a picture while walking with a video camera in hand, and numbered in chronological order. The portion that was visible in the first frame (801) is no longer visible in the last frame (806), and it can be seen that this is a long sequence image sequence in which feature point tracking is not possible in all frames.

  In step S202, the moving observation image sequence is divided. This will be described with reference to the processing flowchart of FIG. In step S301, the sub image sequence is determined. This is determined as the number of images in which feature point tracking can be stably performed. FIG. 9 shows a part (901 to 903) of an example of the result of the feature point tracking in a certain sub-image sequence, which are arranged in chronological order taken from 901. The center image (902) is an image located in the center in the time series order of the sub-image sequence, and feature points using the Harris feature are generated in a square frame in the image. The feature points generated here are tracked in both the time-series backward direction and the forward direction, and association is performed in all frames of the sub-image sequence. It can be confirmed that there is no disappearance of feature points and that stable tracking is performed.

  In step S302, the camera motion is estimated by the factorization method using the feature point tracking result, and in step S303, this is optimized as an initial solution. FIG. 10 shows an example of the estimated camera motion and the three-dimensional position of the restored feature point. Each axis in FIG. 10 represents the X, Y, and Z axes, and represents the relative positional relationship between the three-dimensional position of the feature point and the camera position. Thereafter, the same applies to FIG. 11 and FIG.

  In step S304, it is determined whether or not the division is finished. If the currently processed sub-image sequence is included up to the last frame of the moving observation image sequence, the division ends. Otherwise, the process proceeds to the process for determining the number of duplicate frames (step S305). Here, the optimum number of overlapping frames is determined using the estimated camera motion. A three-dimensional point group can be accurately restored by stereo processing using a baseline distance between overlapping frames. When the process is completed, the number n of the sub-image sequence to be processed is set to n = n + 1, and the process returns to step S301.

  When the above processing is repeated and the moving observation image sequence dividing process (step S202) is completed, a three-dimensional corresponding point table is created (step S203). This is realized by performing stereo processing in the overlapping frame portion and restoring the three-dimensional point by two camera motions estimated from the adjacent sub-image sequence from the obtained corresponding point data.

  FIG. 11 is an example of a reconstruction result of a three-dimensional point group using stereo processing parallax images (corresponding point data) and two camera motions. It can be confirmed that the restored three-dimensional point group is separated into two because the coordinate system is different. Correspondence is required for each point of the three-dimensional point group. This process is performed between all adjacent sub-image sequences.

  Next, in step S204, the three-dimensional corresponding point table obtained is used, and the three-dimensional position of the feature point obtained for each sub-image sequence by the factorization method in step S202 and the projection matrix of all images in the moving observation image sequence are shared. Integrate into the world coordinate system.

  Finally, in step S205, the reprojection error of the three-dimensional point is minimized by bundle adjustment using the three-dimensional position, projection matrix, and feature point tracking result of the feature point integrated in the common world coordinate system obtained in step S204. So as to estimate the integrated camera motion. FIG. 12 is a diagram showing an example of camera motion integrated with a three-dimensional point group in which three-dimensional positions of feature points restored for every sub-image sequence are integrated into a common world coordinate system. According to FIG. 12, it can be confirmed that the camera motion is seamlessly restored.

  Through the processing described above, camera motion information can be estimated from a long sequence image sequence in this embodiment.

1 is a configuration diagram of a camera motion information acquisition device according to an exemplary embodiment of the present invention. It is explanatory drawing which shows the mode of the data acquisition in the camera movement information acquisition apparatus by one example of embodiment of this invention. It is a flowchart which shows the process sequence of the camera movement information acquisition method in one embodiment of this invention. It is explanatory drawing which showed the mode of the division | segmentation of the movement observation image sequence in the example of 1 embodiment of this invention. It is a flowchart which shows the process sequence of the division | segmentation of the movement observation image sequence in one embodiment of this invention. It is explanatory drawing which shows the example of the corresponding point table of the three-dimensional point represented by the different coordinate system in one example of embodiment of this invention. It is explanatory drawing which shows the method of coordinate system integration to the world coordinate system in one example embodiment of this invention. It is explanatory drawing which shows an example of the movement observation image in one Example of this invention. It is explanatory drawing which shows an example of the result of the feature point tracking in the sub image sequence of one Example of this invention. FIG. 10 is an explanatory diagram showing an example of camera motion estimated as a three-dimensional restoration result of feature points by factorization using a feature point tracking result in an embodiment of the present invention. FIG. 11 is an explanatory diagram illustrating an example of a result of restoring a three-dimensional point group using a parallax image indicating a stereo processing result and a parallax image and two camera motions in an exemplary embodiment of the present invention. In one embodiment of this invention, it is explanatory drawing which shows an example of the result of having integrated a three-dimensional point cloud and a camera motion into the world coordinate system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Movement observation image sequence acquisition means, 12 ... Movement observation image sequence division means, 13 ... Coordinate system integration means, 14 ... Camera motion information re-estimation means

Claims (9)

A camera motion information acquisition device that acquires camera motion information representing a three-dimensional position and posture of a camera at the time of image capture from an image sequence observed by a moving image input device,
Moving observation image sequence acquisition means for acquiring an image sequence observed by the moving image input device;
A moving observation image sequence dividing unit that divides the acquired movement observation image sequence into a plurality of sub image sequences while overlapping a plurality of images between the sub image sequences;
A coordinate system integrating means for integrating different coordinate systems between the sub-image sequences into a common world coordinate system;
A camera motion information acquisition apparatus comprising: camera motion information re-estimation means for re-estimating camera motion in the common world coordinate system. The camera motion information acquisition device according to claim 1,
The moving observation image sequence dividing means includes:
A camera motion information acquisition apparatus, wherein a sub-image sequence is determined using a feature point tracking result, a measurement matrix is created using the feature point tracking result, and a camera motion is estimated by a factorization method. The camera motion information acquisition device according to claim 1,
The moving observation image sequence dividing means includes:
A camera motion information acquisition apparatus characterized by determining a sub-image sequence using a result of feature point tracking and estimating a camera motion by sequentially performing projection restoration using the feature point tracking result. The camera motion information acquisition device according to claim 1,
The coordinate system integration means includes:
Create a 3D correspondence point table by stereo processing using overlapping frames between sub-image sequences, calculate a coordinate transformation matrix between sub-image sequences using the 3D correspondence point table, and integrate the coordinate system A camera motion information acquisition device. A camera motion information acquisition method for acquiring camera motion information representing a three-dimensional position and posture of a camera at the time of image capture from an image sequence observed by a moving image input device,
A moving observation image sequence acquisition means for acquiring an image sequence observed by the moving image input device; and
A dividing step for dividing the acquired moving observation image sequence into a plurality of sub image sequences while overlapping the plurality of images between the sub image sequences, and a moving observation image sequence dividing unit,
A coordinate system integrating means for integrating different coordinate systems between the sub-image sequences into a common world coordinate system;
The camera motion information re-estimating means comprises the step of re-estimating the camera motion in the common world coordinate system. The camera motion information acquisition method according to claim 5,
The dividing step includes
A step of determining a sub-image sequence using the result of feature point tracking, a step of creating a measurement matrix using the result of the feature point tracking, estimating a camera motion by a factorization method, and an estimated camera motion A camera motion information acquisition method comprising: an optimization step; a step of determining whether or not the division of the moving observation image sequence is completed; and a step of determining the number of overlapping frames. The camera motion information acquisition method according to claim 5,
The dividing step includes
A step of determining a sub-image sequence using the result of feature point tracking, a step of estimating a camera motion by sequentially performing projection restoration using the result of the feature point tracking, and an optimization of the estimated camera motion And a step of determining whether or not the division of the moving observation image sequence is completed, and a step of determining the number of overlapping frames. The camera motion information acquisition method according to claim 5,
The integration step includes a step of creating a three-dimensional corresponding point table by stereo processing using overlapping frames between sub-image sequences, and calculating a coordinate transformation matrix between the sub-image sequences using the three-dimensional corresponding point table. And a step of integrating the coordinate system. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the camera motion information acquisition method according to claim 5.