JP2006106930A - Parameter estimation device, parameter estimation method and parameter estimation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that since the restoration of the space information of the object of an external world from an image photographed by a manual operation by using a video camera results in the change of each focal distance, it is impossible to use a conventional factorization method, and it is necessary to simultaneously restore the changing focal distances for focusing. <P>SOLUTION: A factorization processing part 43 prepares matrix data from image coordinate values showing the temporal changing quantity of the featured point of each image, applies singular valve decomposition and noise removal to the matrix data thereby obtaining the matrix data showing motion information and the matrix data showing three-dimensional information, calculates a transformation matrix satisfying conditions set for stipulating motion in the component of the motion information, makes the transformation matrix act on the matrix data being the motion information thereby restoring camera viewpoint motions, and makes the inverse matrix of the transformation matrix act on the matrix data expressing the three-dimensional information thereby restoring three-dimensional information configuring an object shape, and an optical axial translational motion restoration/focal distance estimating part 44 performs the reproduction of the translational motion and the estimation of the focal distances in the optical axial direction of the camera viewpoint. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  INDUSTRIAL APPLICABILITY The present invention can be used for all time-series images such as in-vehicle images, indoor images, walking images, sea images from the ship, aerial images, etc. acquired using an image input device (for example, a camera). From the acquired time-series image, the translation movement of the camera viewpoint and the rotation around the optical axis (yaw rotation), and the three-dimensional shape of the outside world reflected in the time-series image, that is, the external shape of the subject (object) are configured. It relates to restoring 3D information and simultaneously estimating camera internal parameters.

  In the computer vision field, methods for measuring or acquiring the shape of an object from time-series image data include three-dimensional analysis methods using stereo measurement and epipolar plane analysis. According to this method, it is possible to restore the three-dimensional position information related to the spatial shape or the spatial structure and the motion related to the camera viewpoint from a plurality of time-series images in which the object is captured. However, since random noise is mixed in a time-series image taken while moving the camera, it is difficult to restore the camera motion and the object shape at the same time with high accuracy.

  On the other hand, the conventional factorization method can restore the camera motion and the object shape robustly and with high accuracy in a time-series image such as a video image in which random noise is mixed (for example, (See Patent Document 1).

  However, in this method, the internal parameters of the camera (focal length, etc.) are known, and it is necessary to calibrate the camera internal parameters by camera calibration in advance.

  On the other hand, digital still cameras, handy cameras, and the like are equipped with an autofocus function, and it is possible to obtain a focused image by adjusting the focus according to the distance from the subject. As a result, the focal length of each image captured with autofocus is finely adjusted. In the conventional factorization method, the camera motion and 3D shape are restored on the assumption that the internal parameters of the camera are fixed (the focal length does not change in the image is constant). I couldn't.

On the other hand, a technique for estimating internal parameters of a camera from an image obtained by shooting a simple shooting target (a black and white checkerboard pattern printed on paper) in a hands-free manner is also disclosed (for example, see Non-Patent Document 1). .). However, this technique cannot be applied to internal parameters that change over time.
JP 2003-271925 A Z. Zhang, “A Flexible New Technology for Camera Calibration”, IEEE Trans. Pattern Anal. & Mach. Intel, Vol. 22, no. 11, pp. 1330-1334, 2000.

  Using a measurement method that applies the principle of Shape From Motion (the idea of restoring a three-dimensional shape by the movement of the camera), it is possible to capture objects from the outside world from time-series images taken using a digital still camera or a handheld video camera. When acquiring and restoring spatial information, it is necessary to focus on each image, and commercially available cameras are equipped with an autofocus function that adjusts the focus according to the distance to the subject. An image with the focus adjusted for each time is obtained. However, since the focal length when each image is acquired by the autofocus function changes, the conventional factorization method cannot be used, and the changing focal length for focus adjustment must be restored at the same time. There's a problem.

  The present invention has been made in view of such circumstances, and an object thereof is to provide a camera parameter estimation technique that solves the above-described problems.

  In the present invention, a projection model for estimating the focal length is used.

It is assumed that the camera motion, the three-dimensional shape, and the focal length are restored based on this projection model. In Equation 1, the image coordinate value of the j-th feature point observed in the image of the i-th frame is (xij, yij), the focal length at that time is fi, and the XY translational motion is (Tx _i , Ty _i , Tz). _i ), the rotation around the optical axis is θ _i . The image coordinate values (x _ij , y _ij ) have a principal point as an origin, the principal point is known, and the horizontal and vertical scale factor ratio of the image is 1: 1.

  When the above is expressed in matrix for all frames (i = 1, 2,..., F) and all feature points (j = 1, 2,..., P),

It becomes. Here, when f _i = 1 (i = 1, 2,..., F), the right side of (Equation 2) assumes that the camera motion is a plane motion and uses the factorization method to calculate the camera motion. A three-dimensional shape can be restored. However, since the translational motion in the optical axis (Z-axis) direction is changed by the camera motion, and the focal length fi is changed by the autofocus function and is different for each frame, Tz _i (optical axis direction) on the left side of Equation 2 is changed. Translational motion) and focal length f _i must be estimated simultaneously. Therefore, the image coordinate values ((1-Tz _i / Zj) x _ij / f _i , (1-Tz _i / Zj) y _ji multiplied by the coefficient (1-Tz _i / Zj) / f _i shown on the left side of Expression 2. / F _i ) restores planar motion and 3D shape.

The reprojected coordinate values (Δu _ij , Δv _ij ) from the plane motion and the three-dimensional shape restored by the factorization method are

It is expressed. Equation 3 All feature points, when expanded to all frames,

The optical axis translation Tz _i and the focal length f _i are

It is obtained by.

  Furthermore, when the horizontal and vertical scale factor ratio of the pixel is 1: β,

Is a projection model, and α _i = f _i and β _i = βf _i are set as the focal length f _{i in} each frame. Here, when the camera motion and the three-dimensional shape are restored, reprojection coordinate values (Δu _ij , Δv _ij ) are calculated, and from Equation 7,

It becomes. Combine the above equation for all feature points,

So that

Thus, the optical axis translational movements Tz _i , α _i , and β _i in the i-th frame can be simultaneously obtained.

In the present invention, the camera motion and the three-dimensional shape are restored from the time-series image captured by the motion obtained by adding the optical axis translation to the planar motion, and the camera internal parameters are estimated as shown in FIGS. By moving the camera fixed on the flat plate with a support on the circumference of radius R, it is possible to acquire a time series image with a motion obtained by adding an optical axis translational motion to the plane motion, and feature points in the time series image By observing the image coordinate values (x _ij , y _ij ), the camera is calculated based on the projection models of Equations 2 and 7, and the calculations of Equations 5, 6, and 12 are performed in consideration of the change in focal length due to autofocus. Internal parameters can be estimated.

  According to the present invention, the camera can be used even in a situation where the autofocus function is activated from all the time-series images acquired using a camera or the like (vehicle-mounted images, sea images, aerial images, indoor images, etc. taken using a moving means). Motion and 3D shape can be restored at the same time, and camera internal parameters changed by the autofocus function can be estimated.

  In the present invention, the camera acquires time-series images by the imaging means shown in FIGS. 1 and 2, which is effective in observing the image coordinate values of the feature points with the feature point observation means with high accuracy. Thereby, the camera motion, the three-dimensional shape, and the camera internal parameters can be restored and estimated with higher accuracy. Since most of the calculations used in the present invention are composed of linear operations, implementation in a computer language is easy.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
In FIG. 1, a camera 13 is fixed to a vertical column 12 at a distance of a radius R from the center of rotation on a horizontal plane plate 11, and a subject is imaged while the camera 13 is rotated on the circumference of the radius R.

  In FIG. 2, a camera 23 is fixed to a vertically-arranged support column 22 at a distance of a radius R from a rotation center plate 21 standing vertically, and the camera 23 is rotated on the circumference of radius R. Shoot the subject.

  With these imaging means, it is not necessary to rotate the camera all around, and shooting is performed to the extent that the subject is not out of frame, and time-series images are obtained. Thereby, it is possible to pick up an image with a camera motion with a degree of freedom of 4 which has an optical axis translational motion in a plane motion. On the other hand, the present invention can also be applied to a time-series image obtained by photographing a subject while manually moving the camera. However, the camera internal parameters can be estimated with higher accuracy by obtaining the photographing means shown in FIGS. be able to.

FIG. 3 shows the positional relationship between the camera and the object in a situation where a time-series image is acquired by the photographing means with the camera motion having the translational motion in the optical axis direction in the planar motion of FIGS. The center of motion is the viewpoint, and a camera coordinate system XYZ with the viewpoint as the origin and a world coordinate system XwYwZw with the origin O are set. In this coordinate system, the camera motion is a translational motion of T _i (Tx _i , Ty _i , Tz _i ) and a yaw rotation θi around the optical axis (Z axis). It is assumed that the point P _j (X _j , Y _j , Z _j ) of the object is projected onto the image coordinate value (x _ij , y _ij ) with the projection center (principal point) as the origin on the image plane by the camera. For the sake of explanation, it is assumed that the viewpoint in the initial frame and O coincide with each other, the optical axis is parallel to the Zw axis, and θ _i is the angle formed by the X and Xw axes. There is no loss.

  FIG. 4 is a basic configuration diagram of the parameter estimation apparatus of the present invention, and this embodiment will be described with reference to FIG. The present invention restores camera motion and three-dimensional shape from a time-series image database 41 in which time-series images are accumulated, a feature point observation unit 42 that observes coordinate values of feature points in each image, and matrix data composed of image coordinate values. A factorization processing unit 43, an optical axis translational motion reconstruction / focal length estimation unit 44 that restores translational motion in the optical axis direction and estimates the focal length of camera internal parameters, and obtains the camera motion and three-dimensional shape and focal length. It consists of an iterative processing unit 45 for repeating the processing until it is received.

  In this configuration, the time series image database 41 may be in a form using a recording medium such as a hard disk, a RAID device, a CD-ROM, or a form using remote data resources via a network.

  Furthermore, FIG. 5 is a processing configuration diagram in the case of processing in real time, and the present invention does not necessarily require a storage device such as the time-series image database unit 41.

  First, in the feature point observation unit 42 in FIG. 4, an image sequence for the frame F is extracted from the time-series image database 41 as a time-series image obtained by capturing an object. Feature point tracking is performed on the extracted image series. The feature points are extracted by the following procedure as used conventionally.

In region 1 of image 1 as shown in FIG.
(1) A 2 × 2 Hessian matrix for each pixel is obtained. next,
(2) It is determined whether or not the point is a local maximum in the 3 × 3 neighborhood of each point, and points other than the local maximum are deleted (non-maxima suppression). further,
(3) The fixed values σ _l and σ _s (σ _s ≦ σ _l ) of the obtained Hessian matrix are obtained, and points where σ _s is equal to or greater than σ _p are extracted. Finally,
(4) Sort the extracted points in the order of the size of σ _s , and check whether or not a point (p _h ) higher than the point (p _l ) exists at a distance within σ _d pixels in order from the upper point. If there is, the lower point p ₁ is deleted. Further, the extracted feature points (j = 1, 2,..., P) are tracked over the image i (i = 2,..., F) by the KLT method (Kanade-Lucas-Tomasi), and image coordinate values are obtained. Observe (x _ij , Y _ij ). The image coordinate values of the features thus obtained are

2F × P matrix data (matrix data [A]) arranged in the array shown in FIG.

  Next, processing contents of the factorization processing unit 43, the optical axis translational motion restoration / focal length estimation unit 44, and the iterative processing unit 45 will be described with reference to the processing flowchart of FIG.

  First, the factorization processing unit 43 inputs matrix data (S1).

Next, the factorization processing unit 43 generates matrix decomposition data (S2). That is, the coefficient δ _ij is

From the 2F × P matrix data [A] of Equation 13 that was initialized and held as follows:

Is stored as matrix decomposition data [B].

  Next, the factorization processing unit 43 performs singular value decomposition (S3). That is, the matrix decomposition data [B] is used as input data by singular value decomposition.

Matrix decomposition into three matrices [U], [W], and [V] shown in FIG. Here, [U] is a 2F × P size matrix, [W] is a P × P size diagonal matrix, and [V] is a P × P size matrix.

  Next, the factorization processing unit 43 performs noise removal (S4). That is,

As shown in the second term, the components of each matrix after rank 4 are deleted. At the time of this deletion, the matrix [U] is taken out, the fourth to Pth columns are deleted from the matrix elements, and the matrix composed of the remaining components is held. Next, the matrix [W] is extracted, and the fourth to Pth rows and the fourth to Pth columns are deleted from the matrix elements, and the matrix composed of the remaining components is held. Further, the matrix [V] is taken out, the fourth to Pth rows are deleted from the matrix elements, and the matrix composed of the remaining components is held.

  Following denoising, from the matrix taking the square of the diagonal elements of the matrix [W] with the 4th to Pth rows and the 4th to Pth columns deleted,

The matrix [U ′] and the matrix [V ′] shown in FIG.

  Next, the factorization processing unit 43 calculates a transformation matrix (S5). That is, the held matrix [U ′] is extracted,

The coefficient concerning each element of the symmetric matrix [C] in the simultaneous conditional expressions shown in FIG. Note that the symmetric matrix [C] is

It is represented by

  These coefficient calculations are easily obtained by matrix operation, and this conditional expression is calculated for all frames. Next, each element of the 3 × 3 size matrix [C] is determined using numerical calculation such as the least square method for the simultaneous conditional expressions shown in Expressions 21 to 23 for all frames. Calculated matrix [C]

The eigenvalue decomposition is performed as shown in FIG. Here, from the square of the eigenvalue matrix and the eigenvalue matrix,

Matrix [C '] and matrix [Q] having the matrix elements as components

Calculate according to

  Next, the factorization processing unit 43 performs planar motion restoration (S6). That is, from the obtained matrix [Q] and the stored matrix [U ′],

The matrix [M ′] is calculated by the matrix operation. The matrix element (m _ix , n _ix ) or (m _iy , n _iy ) of each frame (i-th frame) is extracted from the matrix [M ′],

Is used to restore the yaw rotation θi. Next, matrix elements (T _iu , T _iv ) of each frame (i-th frame) are extracted from the matrix [M ′]. From this (T _iu , T _iv )

XY translation in Euclidean space in the i-th frame using the (Tx _i, Ty _i) is calculated.

  Next, the factorization processing unit 43 performs three-dimensional information restoration (S7). That is, from the previously held matrix [V ′] and the matrix [Q] obtained by the transformation matrix calculation,

The matrix [S ′] is obtained by performing the matrix operation shown in FIG. Next, for the elements of the matrix [S ′],

To obtain a matrix [P]. Each column vector of the matrix [P] is a three-dimensional coordinate value (X _j , Y _j , Z _j ) in the Euclidean space of the j-th feature point.

  Next, the optical axis translational motion recovery / focal length estimation unit 44 performs optical axis translational motion recovery / focal length estimation (S8). That is, from the camera motion and 3D shape restored by plane motion restoration and 3D information restoration

(Δu _ij , Δv _ij ) shown in FIG. 5 is obtained, and Δw _ij shown in Expression 3 is obtained for all frames and all feature points from the image coordinate values (x _ij , y _ij ) in each frame.

F × P matrix [D1] is prepared. Next, it consists of Z _j (j = 1, 2,..., P) obtained by three-dimensional information restoration.

Matrix [D2] is prepared.

The matrix operation of

To obtain an F × 2 matrix [D]. Further, d _i1 in the i-th row and first column and d _{i2 in} the i-th row and second column of the matrix [D] are extracted,

Is calculated over the entire frame to obtain the focal length f _i and the optical axis translation Tz _i . In this calculation, matrix calculation is performed based on Equations 5 and 6.

  Next, the iterative processing unit 45

An error ΔE obtained by the above calculation is calculated (S9). It is determined whether this error has converged below a certain value (S10). If the error has converged below a certain value, the process is terminated. If not,

Accordingly, the coefficient δ _ij is updated (S11). Further, the matrix decomposition data [B] is updated by Expression 15 in accordance with the coefficient update. This is repeated until the error ΔE converges below a certain value.

  As described above, according to the embodiment of the present invention, from the image coordinate value of the feature point in the time-series image, the motion of the camera viewpoint: the rotation around the optical axis, the translational motion in the XYZ directions, and the three-dimensional information constituting the object shape At the same time, the focal length of the camera internal parameters can be estimated.

(Embodiment 2)
FIG. 8 is a basic configuration diagram of the parameter estimation apparatus of the present invention, and this embodiment will be described with reference to FIG. In this embodiment, a time-series image database 81 in which time-series images are accumulated, a feature point observation unit 82 that observes coordinate values of feature points in each image, and camera motion and three-dimensional shape are restored from matrix data composed of image coordinate values. A factorization processing unit 83 that restores the translational motion in the optical axis direction and estimates the focal length of the camera internal parameters, and obtains the camera motion and the three-dimensional shape and focal length. And an iterative processing unit 85 for repeating the process up to and a scale factor ratio calculating unit 86 for obtaining horizontal and vertical scale factor ratios of pixels of camera internal parameters.

  In this configuration, the time series image database 81 may be in a form using a recording medium such as a hard disk, a RAID device, a CD-ROM, or a form using remote data resources via a network.

  Furthermore, FIG. 9 is a processing configuration diagram in the case of processing in real time, and this embodiment does not necessarily require a storage device such as the time-series image database unit 81.

  In the present embodiment, the processing of the optical axis translational motion restoration / focal length estimation unit 84 and the scale factor ratio calculation unit 86 is different from that of the first embodiment. Therefore, only this portion will be described in the following description.

  FIG. 10 is a processing flow of the factorization processing unit 83, the optical axis translational motion restoration / focal length estimation unit 84, and the scale factor ratio calculation unit 86 in the parameter estimation apparatus shown in FIG.

  In the first iteration,

The coefficients δ _ij and γ _ij are initialized as _follows ,

The matrix [B] is obtained. The processing in the optical axis translational motion restoration / focal length estimation unit 84 obtains (Δu _ij , Δv _ij ) shown in Expression 33 and calculates all feature points from the image coordinate values (x _ij , y _ij ) in each frame. Obtaining A _j and B _j of Equation 8, Equation 9, and Equation 10 for

2 × P matrix [D1] is prepared. In the matrix operation of Equation 36

A 2 × 2 matrix [D] in each frame is obtained. _Extract d _i1 in the first row and first column and d ′ _i1 in the second row and first column from the matrix [D].

To obtain α _i and β _i . Also, the optical axis translational motion Tz _i is derived from each element of the matrix [D]

Calculate according to These calculations are performed based on Equation 12.

The iterative processing unit 85 calculates ΔE obtained by the calculations of Expressions 39 and 40. If this error has converged below a certain value, the process is terminated. If not, set the coefficients δ _ij and γ _ij

In accordance with this update, the matrix decomposition data [B] is updated by Expression 43. This is repeated until the error ΔE converges below a certain value.

When the iterative processing is completed, the scale factor ratio calculation unit 86 obtains an average value of β _i / α _i to obtain a horizontal / vertical scale factor ratio 1: β of the pixel.

  As described above, according to the second embodiment, from the image coordinate values of the feature points in the time-series image, the motion of the camera viewpoint: the rotation around the optical axis, the translational motion in the XYZ directions, and the three-dimensional information constituting the object shape are restored. At the same time, the focal length of the camera internal parameters and the horizontal / vertical quantization ratio (scale factor ratio) of the pixels can be estimated.

  Note that the parameter estimation device described in the first and second embodiments is realized by, for example, a CPU included in a computer device that configures the parameter estimation device, and requires necessary feature point observation, factorization processing, optical axis translational motion restoration / focusing, and the like. Distance estimation, iterative processing, scale factor ratio calculation, etc. can be installed as application programs.

  In addition, the computer device stores data such as the above-described feature point observation, factorization processing, optical axis translational motion recovery / focal length estimation, iterative processing, scale factor ratio calculation, and other processing results and calculation results. You may enable it to write and read to a memory | storage device etc.

  The present invention also provides a recording medium in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or apparatus, and a program in which the CPU (MPU) of the system or apparatus is stored in the recording medium. This can also be realized by reading and executing the code. In that case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and a storage medium storing the program code, for example, a CD-ROM, a DVD-ROM, a CD-R, CD-RW, MO, HDD, etc. constitute the present invention.

The figure in case a camera images with the rotational motion of the radius R on a horizontal surface board. The figure in case a camera images with the rotation motion of radius R on a vertical plane board. The figure which shows a camera viewpoint, a three-dimensional position, and the coordinate system used by description of this invention. The basic block diagram of a parameter estimation apparatus. The basic block diagram of a parameter estimation apparatus. The figure which shows the observation of the feature point in a time series image. The processing flow figure regarding a factorization process part, an optical-axis translational movement decompression | restoration / focal distance estimation part, and an iterative process part. The basic block diagram of a parameter estimation apparatus. The basic block diagram of a parameter estimation apparatus. The processing flow figure regarding a factorization processing part, an optical axis translational movement restoration / focal distance estimation part, an iterative processing part, and a scale factor ratio calculation part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Plane version 12 ... Support | pillar 13 ... Camera 21 ... Rotation plane version 22 ... Support | pillar 23 ... Camera 41 ... Time series image database 42 ... Feature point observation part 43 ... Factorization process part 44 ... Optical axis translational movement restoration and focal length estimation 45: Iterative processing unit 51 ... Image input unit 52 ... Feature point observation unit 53 ... Factorization processing unit 54 ... Optical axis translational motion restoration / focal length estimation unit 55 ... Repetition processing unit 81 ... Time series image database 82 ... Feature point Observation unit 83 ... Factor decomposition processing unit 84 ... Optical axis translational motion restoration / focal length estimation unit 85 ... Repetition processing unit 86 ... Scale factor ratio calculation unit 91 ... Image input unit 92 ... Feature point observation unit 93 ... Factor decomposition processing unit 94 ... Optical axis translational motion recovery / focal length estimation unit 95 ... Repetition processing unit 96 ... Scale factor ratio calculation unit

Claims (7)

In the image coordinate system set in the time-series image of the object acquired by the camera means, the feature point observation means for observing the image coordinate value indicating the temporal change amount of the feature point of each image;
Factorization processing means for creating matrix data from the image coordinate values and restoring the camera viewpoint movement of the camera means and the three-dimensional shape constituting the appearance shape of the object from the matrix data;
A reprojection coordinate value that is a reprojection error of the camera viewpoint motion in the image coordinate system for each feature point is obtained from the restored camera viewpoint motion and the three-dimensional shape, and the reprojected coordinate value and the observed image coordinate value From the above, the optical axis translational motion reconstruction / focal length estimation means for reproducing the translational motion of the camera viewpoint in the optical axis direction and estimating the focal length of the camera means,
An error between the reprojection error and the coordinate value obtained by multiplying the observed image coordinate value by the coefficient composed of the camera viewpoint movement and the three-dimensional shape and the coefficient composed of the camera internal parameter is equal to or less than a predetermined value. If not, the image coordinate value multiplied by the coefficient composed of the camera viewpoint motion and the three-dimensional shape and the coefficient composed of the camera internal parameters is used as a new matrix. A parameter estimation apparatus comprising: an iterative processing unit that creates matrix data as elements and repeats the processing from the factorization processing unit. In the optical axis translational movement restoration / focal length estimation means, a reprojection coordinate value that is a reprojection error of the camera viewpoint movement in the image coordinate system for each feature point is obtained from the restored camera viewpoint movement and the three-dimensional shape, From the reprojected coordinate value and the observed image coordinate value, reproduce the translational motion in the optical axis direction of the camera viewpoint, and estimate the scale factor ratio and the focal length,
In the iterative processing means, between the reprojection error and the coordinate value obtained by multiplying the observed image coordinate value by a coefficient constituted by the camera viewpoint movement and the three-dimensional shape and a coefficient constituted by a camera internal parameter. It is determined whether the error has converged to a specified value or less, and if not converged, the image coordinates multiplied by the coefficient composed of the camera viewpoint motion and the three-dimensional shape and the coefficient composed of the camera internal parameters Create matrix data with values as new matrix elements, repeat the process from the factorization processing means, perform the iterative process until the error converges below the specified value,
The parameter estimation apparatus according to claim 1, further comprising a scale factor ratio calculating unit that calculates a scale factor ratio using the estimated focal length and the scale factor ratio when the iterative process is completed. In the factorization processing means,
Obtaining matrix data representing motion information and matrix data representing three-dimensional information by performing singular value decomposition and noise removal on the created matrix data,
In the component of the motion information, a transformation matrix that satisfies the conditions set for defining the motion is obtained, and the camera viewpoint motion is restored by applying this transformation matrix to the matrix data serving as the motion information,
The parameter estimation apparatus according to claim 1 or 2, wherein an inverse matrix of the transformation matrix is applied to matrix data representing the three-dimensional information to restore the three-dimensional information constituting the object shape. A feature point observation step of observing an image coordinate value indicating a temporal change amount of a feature point of each image in an image coordinate system set in a time-series image of the object acquired by the camera means by the feature point observation unit;
Factorization processing means creates matrix data from the image coordinate value, factorization processing step to restore the camera viewpoint movement of the camera means and the three-dimensional shape constituting the appearance shape of the object from the matrix data,
The optical axis translational movement restoration / focal length estimation means obtains a reprojection coordinate value that is a reprojection error of the camera viewpoint movement in the image coordinate system for each feature point from the restored camera viewpoint movement and the three-dimensional shape. From the projected coordinate value and the observed image coordinate value, the optical axis translational motion reconstruction / focal length estimation step for reproducing the translational motion of the camera viewpoint in the optical axis direction and estimating the focal length of the camera means;
The error between the re-projection error and the coordinate value obtained by multiplying the observed image coordinate value by the coefficient composed of the camera viewpoint movement and the three-dimensional shape, and the coefficient composed of the camera internal parameter. Is not converged to a specified value, and if not converged, the image coordinate value multiplied by the coefficient composed of the camera viewpoint motion and the three-dimensional shape and the coefficient composed of the camera internal parameters A parameter estimation method comprising: an iterative processing step of creating matrix data using as a new matrix element and repeating the processing from the factorization processing means. In the optical axis translational movement restoration / focal length estimation step, a reprojection coordinate value that is a reprojection error of the camera viewpoint movement in the image coordinate system regarding each feature point is obtained from the restored camera viewpoint movement and the three-dimensional shape, From the reprojected coordinate value and the observed image coordinate value, reproduce the translational motion in the optical axis direction of the camera viewpoint, and estimate the scale factor ratio and the focal length,
In the iterative processing step, between the reprojection error and the observed image coordinate value multiplied by a coefficient composed of the camera viewpoint movement and the three-dimensional shape and a coefficient composed of a camera internal parameter. It is determined whether the error has converged to a specified value or less, and if not converged, the image coordinates multiplied by the coefficient composed of the camera viewpoint motion and the three-dimensional shape and the coefficient composed of the camera internal parameters Create matrix data with values as new matrix elements, repeat the process from the factorization processing means, perform the iterative process until the error converges below the specified value,
The scale factor ratio calculating unit includes a scale factor ratio calculating step of calculating a scale factor ratio using the estimated focal length and scale factor ratio when the iterative process is completed. Parameter estimation method. In the factorization processing step,
Obtaining matrix data representing motion information and matrix data representing three-dimensional information by performing singular value decomposition and noise removal on the created matrix data,
In the component of the motion information, a transformation matrix that satisfies the conditions set for defining the motion is obtained, and the camera viewpoint motion is restored by applying this transformation matrix to the matrix data serving as the motion information,
The parameter estimation apparatus according to claim 4 or 5, wherein an inverse matrix of the transformation matrix is applied to matrix data representing the three-dimensional information to restore the three-dimensional information constituting the object shape. A parameter estimation program characterized in that the parameter estimation apparatus or the parameter estimation method according to any one of claims 1 to 6 is described in a computer program so as to be executable.

Images