JP2000242785A - Method and device for generating three-dimensional shape information, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate point group data representing the all-round shape of an object from an image series obtained by performing all-round photographing of the object and also to make reducible the calculation time for generating the point time data. SOLUTION: A tracking queue generating part 104 constructs a tracking queue, and a tracking queue dividing part 105 divides the tracking queue and generates plural partially overlapped partial tracking queues. A partial three- dimensional data generating part 107 generates a partial measurement queue obtained by estimating and complementing the missing value of the partial tracking queue and generates partial point group data representing a partial shape from the partial measurement queue. A partial point group integrating part 108 integrates plural partial point group data and generates all-round point group data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for estimating and restoring a three-dimensional shape of a target object from a sequence of images of the target object captured by a digital video camera or the like.

[0002]

2. Description of the Related Art Methods for inputting point cloud data representing a three-dimensional shape of an object include an active method such as laser scanning and projection of an artificial texture, and a passive method using only a camera. There is.

In the latter method, image feature points are associated (tracked) between time-series images of a target object photographed by a camera, and point group data representing the shape of the target object is obtained based on the result. calculate. One such technique is
There is a factorization method proposed by Tomasi and Kanade (
C. Tomasi and T. Kanade, Shape and Motion from Image
Streams under Orthography: A Factorization Metho
d, InternationalJournal of Computer Vision, vol.9,1
992, pp. 137-154). The method described here performs a linear formulation based on an orthographic projection model and uses singular value decomposition of a matrix that is numerically stable, so the solution is extremely stable compared to other methods. Is the feature.

A factorization method using a pseudo center projection model closer to the center projection which is an actual camera model while maintaining the linearity of the formulation has been proposed by Poelman and Kanade (CJPoelman and T. Kanade, A para
perspective factorizationmethod for shape and moti
on recovery, IEEE Transaction on PatternAnalysis
and Machine Intelligence, vol.19, no.3, pp.206-21
8).

[0005] In the factorization method, image feature points are tracked for an image sequence, and the position data of all the feature points in all the images are represented by a tracking matrix in which the columns are arranged in correspondence with the feature points and the rows are arranged in correspondence with the images. To construct. The upper half of the tracking matrix is the X coordinate of the feature point, and the lower half is the Y coordinate of the feature point. When the target object is photographed by the camera so as to make a circuit, the tracking matrix includes elements (missing values) whose position data is missing because there are feature points hidden by the target object.
The missing value is estimated from the surrounding position data and complemented. Then, by performing factorization of the tracking matrix (measurement matrix) after complementing the missing values, point group data representing the three-dimensional shape of the target object and camera posture data are simultaneously generated.

[0006]

Conventionally, in the factorization method, since the entire tracking matrix is processed collectively,
All missing values in the tracking matrix need to be supplemented by estimation. However, it is often impossible to estimate all missing values included in the tracking matrix with sufficient accuracy when the target object is photographed all around. It was almost impossible to extract with precision.

Further, even if all missing values are successfully estimated, there is a problem that a long calculation time is required to obtain point cloud data. For example, when tracking 1000 image feature points over 100 images, the size of the tracking matrix is 200 × 1000. When considering a practical application that generates high-density point cloud data over the entire circumference of an object, data of more than 1000 point clouds is often required, and the size of the tracking matrix is further increased. However, since the factorization method is a time-consuming numerical calculation method, applying the factorization method directly to the entire large matrix as described above requires enormous computation time and processing. Consumes a lot of memory. Incidentally, when the number of feature points exceeds about 400, the calculation time increases almost in proportion to the cube of the size of the tracking matrix.

Further, the factor decomposition is not unique, and there are two types of solutions, so that there is a problem that the irregularities on the surface of the target object may not be correctly restored.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to reliably generate point cloud data for restoring a three-dimensional shape over the entire circumference of a target object from a series of images of the target object taken all around. By shortening the calculation time for generating data, enabling the correct restoration of surface irregularities of the target object, and enabling optimal selection of mesh mapping images generated from point cloud data, etc. is there.

[0010]

In order to achieve the above object, according to the present invention, a tracking matrix generated from a sequence of images of a target object is divided into a plurality of partially overlapping partial tracking matrices. Then, a partial measurement matrix is generated by estimating and complementing the missing value of each partial tracking matrix.
Then, partial point cloud data of the target object is generated from each partial measurement matrix, and the generated partial point cloud data is integrated. Also, based on the camera orientation data generated from the partial measurement matrix and the shooting direction of the target object, the unevenness correction is performed on the partial point group data generated from the partial measurement matrix. Further, a texture mapping image for restoring the shape of the target object is automatically selected based on the camera posture data.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In order to facilitate understanding of the following description, an outline of a process of generating point cloud data over the entire circumference of a target object in the present invention will be described. FIG. 1 is a diagram showing the process in a simplified manner. First, while moving the camera (digital video camera or the like) in a specific direction with the target object fixed, or rotating the target object in a specific direction with the camera fixed, the entire circumference of the target object is photographed. . A tracking matrix is created by tracking feature points of a time-series image obtained by this full-circumference shooting. Next, a tracking matrix is divided to generate a plurality of partial tracking matrices partially overlapping. Next, the missing values are estimated and supplemented for each partial tracking matrix to generate a partial tracking matrix (called a partial measurement matrix) with the missing values complemented, and a factorization method is applied to this partial measurement matrix. By doing so, point group data (partial point group data) representing the partial shape of the target object is generated. Finally, by performing a process of integrating partial point cloud data generated from a plurality of partial measurement matrices, point cloud data representing the shape over the entire circumference of the target object (full circumference point cloud data)
Generate

[0012] Even if the original tracking matrix contains many missing values, and even if missing values that are difficult to estimate are included, if the appropriate division processing is performed, each tracking matrix has a missing value. Missing values,
That is, it is possible to estimate missing values required for generating each partial point cloud data. In addition, if an appropriate division process is performed and the size of each partial tracking matrix is made sufficiently smaller than the size of the original tracking matrix, the calculation time for generating the partial point cloud data from the individual partial measurement matrices is of course. That is, the calculation time for generating the all-around partial point cloud data can be significantly reduced as compared with the factorization method for the entire original tracking matrix.

Hereinafter, as one embodiment of the present invention, automatic generation of all-around point group data as described above and automatic generation of a texture mapping image for restoring a three-dimensional shape based on the all-around point group data will be described. A three-dimensional shape information generation system for performing selection will be described. FIG. 2 is a schematic configuration diagram of the three-dimensional shape information generation system.

In FIG. 2, reference numeral 100 denotes an image capturing device for capturing a time-series image of a target object taken in a specific direction all around using a digital video camera or the like. The captured time-series image data is stored in the storage device 101. The time-series image data stored in the storage device 101 is input to the two-dimensional operation unit 103 by the image input unit 102. The two-dimensional operation unit 103 performs tracking of image feature points of the input time-series image data to construct a tracking matrix, and partially performs processing of dividing the generated tracking matrix to partially generate the tracking matrix. Tracking matrix division unit 10 for generating a plurality of overlapping partial tracking matrices
5, the details of which will be described later. The processing result of the two-dimensional operation unit 103 is
6 is input. The three-dimensional operation unit 106 generates point group data representing the three-dimensional shape of the target object and selects a texture mapping image, and includes a partial three-dimensional data generation unit 107 and a partial point group integration unit 108. The details of the processing will be described later. The data generated by the three-dimensional operation unit 106 is output by the data output unit 109 and stored in the storage device 11. Hereinafter, the processing contents of each unit will be described along the processing flow.

<< Process of Tracking Matrix Generation Unit 103 >> FIG.
4 shows a processing flow of the tracking matrix generation unit 103. First, in step 200, in each frame of the time-series image, a small image region where the density change is drastic is extracted as a feature point. At this time, the feature points are ranked according to the intensity of the density change so that the feature points having a large density change have a higher rank. Then, a predetermined number of feature points are selected from the extracted feature points in descending order of rank.

Next, in step 202, the feature points selected in the previous step 200 are associated with the two focused frames of the time-series image to track the feature points. More specifically, pattern matching is performed between a feature point on one frame and a feature point on the other frame, and feature points determined to have matching patterns are associated with each other.

If the tracking or the association of some of the selected predetermined number of feature points fails (step 206, NO), the process returns to step 200, where the unselected feature points of the current two frames are selected. Of the feature points for which the tracking failed, is selected again. This selection is performed with priority given to feature points having a higher rank. Then, the process returns to step 202, and tracking of the reselected feature point is performed.

If the tracking of a predetermined number of feature points has succeeded in two frames in this way (step 206, YE
S), the process returns to step 202, and the same feature point tracking is performed again for one frame of the two frames that have been tracked, which is later in time, and the next one frame. At this time, for one frame earlier in time, a feature point that has already been successfully tracked is first selected to try to track.
A similar predetermined number of feature points are tracked for all pairs of two frames that are adjacent to each other.

If the tracking of a predetermined number of feature points has succeeded for all frames of the time-series image (step 204, YE
S), proceed to step 208 to construct a tracking matrix in which the position data on each frame of each feature point associated by the feature point tracking are arranged. This tracking matrix is stored (step 210), and the process ends.

<< Processing of Tracking Matrix Dividing Unit 104 >> FIG.
4 shows a processing flow of a tracking matrix division unit 105. First, in step 222, a tracking matrix is fetched and shaped. The tracking matrix is divided to generate a plurality of partial tracking matrices, but in order to facilitate estimation of missing values in these partial tracking matrices, in other words, a tracking matrix that generates such a partial tracking matrix In order to facilitate the division of the tracking matrix, the tracking matrix is shaped. This shaping process is performed as follows, for example. First, the feature points are divided into the following four groups, focusing on the time (frame) during which the feature points can be continuously tracked.

Group A: Feature points that have been successfully tracked continuously from the first frame to the last frame (feature points that have not become missing values over all frames). Group B: Feature points that could be tracked continuously from the first frame to a certain middle frame, but could not be tracked after the next frame after that middle frame (feature points that became missing values after the middle frame) ). Group C: Feature points that can be tracked continuously from a middle frame to a certain frame, but cannot be tracked from the next frame to the last frame. D group: feature points that can be tracked continuously from the middle frame to the last frame.

Then, the columns corresponding to the feature points of each of the groups A, B, C, and D of the tracking matrix are rearranged in the order as schematically shown in FIG. This is the shaping process. In FIG. 5, a shaded portion is an element having position data, and a blank portion is an element having no position data (missing value). Since the X coordinate of each feature point on each frame is arranged in the upper half of the tracking matrix, and the Y coordinate of each feature point on each frame is arranged in the lower half, the number of feature points is P and the number of frames is If F, the tracking matrix is a matrix of 2F rows × P columns. In FIG. 5, the X and Y coordinates are expressed in a combined form (hereinafter, the same expression method is used).

When the shaping process is completed, the process proceeds to step 222, where the tracking matrix is divided, and a plurality of partially overlapping partial tracking matrices are generated. FIG. 6 is an explanatory diagram of a partial tracking matrix generated by the division processing. FIG.
In, each shaded rectangular area represents a partial tracking matrix, and a solid black area is an overlap area between the partial tracking matrices.

Such a division process is performed based on the following conditions. (1) Each row and each column of the tracking matrix corresponds to a row and a column of at least one partial tracking matrix. (2) The size of each partial tracking matrix is 6 × 4 or more. (3) Overlapping matrix data having a size of 1 × 4 or more between the partial tracking matrices. (4) In each partial tracking matrix, four or more feature points having complete tracking data exist continuously over three or more frames.

The above condition (1) is a condition for obtaining complete point cloud data and complete camera attitude data. The condition (2) is a necessary condition for the factorization method. The condition (3) is a condition for providing common point cloud data between the point cloud data, which is necessary for integrating the partial cloud data generated from each partial tracking matrix. The condition (4) is a restriction for estimating missing values in each partial tracking matrix.

A specific example of the dividing process will be described below. FIG. 7 is an example of the tracking matrix after the shaping process. In the figure, black portions indicate elements where tracking data (position data of feature points) exist, and blank portions indicate elements where the tracking data is missing values. Is shown.
This tracking matrix is divided into, for example, seven partial tracking matrices as shown in FIG. However, in FIG. 8, for the sake of simplicity, the overlapping area of each partial tracking matrix is not shown. FIG. 9 shows partial tracking matrices generated by such division in order from the upper left to the lower right.

A plurality of partial tracking matrices are generated by such a division process. At the same time, point group data of an overlap area between the partial tracking matrices is stored as common point group data (step 226).

Next, a process of estimating a missing value (that is, a hidden feature point) for each partial tracking matrix is performed. That is, one partial tracking matrix is fetched (step 228),
It is checked whether or not the missing value is included in the partial tracking matrix (step 230). If no missing value is included, the process returns to step 228 to fetch the next partial tracking matrix. If a missing value is included in the partial tracking matrix, step 232 is executed.
To perform missing value estimation complementation processing. Such processing is repeated, and when estimation completion of missing values for all partial tracking matrices is completed (YES in step 234), each partial tracking matrix in which missing values are complemented with estimated values, or missing data from the beginning, A partial tracking matrix having no measured value is stored as a partial measurement matrix (step 236).

Next, a missing value estimation complement processing step 232
I will explain the processing content of, but before the detailed explanation,
The basic concept of missing value estimation will be described with reference to FIG.
FIG. 10 shows an example of a tracking matrix constructed by tracking feature points 1 to 7 over frames 1 to 8. In the figure, "・"
The mark indicates an element for which tracking of the feature point has succeeded and the position data has been obtained, and the "?" Mark indicates an element for which tracking of the feature point has failed and the position data has a missing value. In order to estimate such a missing value, known position data around the missing value is used. First, one missing value to be estimated is selected. Then, a sub-matrix (called an estimation matrix) including the selected missing values is created. In this estimation matrix, it is necessary that all elements other than the element to be estimated are known (position data has been obtained). For example, in FIG.
It is assumed that the missing value of the frame 5 of the feature point 6 is selected as an estimation target. In this case, for example, an estimation matrix corresponding to the inside of the frame shown in FIG. 10 is created. Missing values are estimated from this estimation matrix, and the missing values are rewritten with the estimated values. Next, for example, a missing value of the frame 6 of the feature point 6 in FIG. 10 is selected as an estimation target, and an estimation matrix for the estimation is created. For example, an estimation matrix corresponding to feature points 1 to 6 and frames 1 to 6 is created, the missing value is estimated, and the missing value in the tracking matrix is rewritten with the estimated value. Next, for example, FIG.
The missing value of the feature point 7 in the frame 2 is selected as an estimation target, an estimation matrix corresponding to the feature points 1 to 7 and the frames 2 to 6 is created, and the missing value is estimated. In this way, missing values are sequentially estimated one by one, and finally all missing values are complemented by the estimated values.

An example of the processing contents of the missing value estimation complement processing step 232 will be described with reference to the flowchart shown in FIG. Although the estimation accuracy increases as the size of the estimation matrix used for estimation increases, the calculation cost for estimating one missing value increases. Therefore, in the processing example shown here, an estimation matrix of an appropriate size is created, missing values are estimated, and if the estimation fails, the estimation is performed while gradually increasing the size of the estimation matrix to a certain size. repeat. In addition, in order to make it possible to smoothly proceed with the process of repeating the estimation calculation while gradually increasing the size of the estimation matrix, in the processing example shown here, the above-described shaping process is performed in the process. Further, when the missing values remain and the estimation matrix cannot be constructed any more, the processing of inverting the tracking matrix upside down is performed, and the estimation processing is continued.

First, the aforementioned shaping process is performed on the partial tracking matrix to be processed (step 250). It is checked whether it is possible to construct an estimation matrix for one or more missing values in the shaped partial tracking matrix (step 252). If possible, an estimation matrix for the missing value of interest is created (step 254), an estimation operation is performed (step 256), and the success or failure of the estimation is checked (step 258). If successful, replace the missing value with the estimated value. If it fails, the size of the estimation matrix is increased (step 260), and it is checked whether the size of the estimated matrix after the increase is equal to or smaller than a predetermined maximum size (step 262). If the size is equal to or smaller than the maximum size, an estimation matrix of the increased size is created and the estimation operation is performed again (step 256). If the estimation is successful,
If the size of the estimation matrix exceeds the maximum size, it is checked whether the estimation of all missing values has been completed (step 270). If there are no missing values, the process ends. If any missing values remain, the process returns to step 2502 to continue the process for the next missing value.

If the missing values remain, but it is no longer possible to construct the estimation matrix, the process proceeds from step 252 to step 264 to determine whether the partial tracking matrix has been inverted upside down. If not, the partial tracking matrix is vertically inverted (step 266), and the process returns to step 250 to continue. If missing values remain even after the partial tracking matrix has been turned upside down and the estimation matrix cannot be constructed,
The process proceeds from step 252 to step 264, and further proceeds to step 268, in which the partial tracking matrix is again inverted and returned to the original row order, and the process is terminated.

The estimation of such a missing value is performed only inside each partial tracking matrix. In the present invention,
It is not necessary to estimate all missing values in the tracking matrix, and missing values not included in any partial tracking matrix are estimated. Even if such missing values remain, all-around point group data can be finally generated.

<< Processing of Partial Three-Dimensional Data Generation Unit 107 >>
FIG. 12 shows a processing flow of the partial three-dimensional data generation unit 107. First, one partial measurement matrix after missing value estimation complementation is fetched (step 300). The partial measurement matrix is subjected to a three-dimensional data calculation process by a factor decomposition method (step 302), and the three-dimensional coordinates of the feature points (partial point group data representing a partial shape) 304 and the feature point group and the camera A relative three-dimensional motion (camera posture data) 306 is obtained.
As described above, the factorization is not unique, and there is a problem that it is impossible to distinguish the shape irregularities because there are two types of solutions. Therefore, in step 310, the camera posture data 306 is compared with the imaging direction 308 of the target object, and for feature points that do not match, the operation of inverting the z-axis coordinate value (depth information) of the partial point group data, that is, the unevenness inversion operation is performed. Do. In this way, the partial point group data 312 with the unevenness corrected is obtained and stored (step 312). In step 314, the camera posture data 306
, The image having the shooting direction (the optical axis direction of the camera) closest to the average direction of the vectors of the partial point cloud data 312 is selected as the texture mapping image of the mesh generated by the partial point cloud data. The index 316 of the texture image is stored.

If an unprocessed partial measurement matrix remains (step 318, NO), the process returns to step 300, and processing for another partial measurement matrix is performed. Until all the partial measurement matrices have been processed (step 318, YE
S), the process is repeated.

<< Processing of Partial Point Cloud Integration Unit 108 >> FIG.
9 shows a processing flow of the partial point group integration unit 108. First,
In step 350, the partial point cloud data generated from two adjacent partial measurement matrices and the common point cloud data are fetched. In the next step 352, the two partial point cloud data are integrated by rotational transformation or affine transformation using the common point cloud data as the association information between the two partial point cloud data. Then, returning to step 350,
The partial point cloud data adjacent to the integrated point cloud data up to now and the common point cloud data are fetched and further integrated in step 352. By repeating the same integration process until the integration of all the partial point cloud data is completed (step 354, YES), all the circumference point cloud data representing the entire circumference shape of the target object is generated and stored (step 356). The process ends. The all-around point group data is finally output by the data output unit 109 together with the texture mapping image index, and stored in the storage device 110.

The series of processes described above (or the functions of the components of the three-dimensional shape information generating system) are described in, for example, FIG.
4, a CPU 400 and a memory 40 as shown in a simplified manner.
1. On a computer in which an auxiliary storage device 402 such as a hard disk, a media drive 404 for reading / writing a recording medium 403 such as a floppy disk, an I / O interface 405 with an external input / output device, and the like are connected by a system bus 406. And can be realized by software. The program 407 for that purpose is read from a recording medium 403 such as a floppy disk on which the program 407 is recorded via the medium drive 404, temporarily stored in the auxiliary storage device 402, and loaded into the memory 401 when the processing is executed. The time-series image data and the shooting direction data are stored in, for example, the I / O interface 405.
Is captured via

According to the processing method of the present invention described above, even when the number of feature points is large, the calculation time can be greatly reduced as compared with the conventional factorization method which collectively processes a tracking matrix. Experimental data is shown in FIG. 15 to clarify the effect of shortening the calculation time. In this experiment, Sun (SU
N) Company ULTRA60 (CPU:
(UltraSPARC_II, memory: 384 MB), and the processing was executed by software. The number of feature points of the tracking matrix is 1921. The division of the tracking matrix was performed such that one-fourth of the feature points of the partial tracking matrix were used as a common point group. "Calculation time" shown in FIG.
Is the CPU time in terms of CPU 100%. Also,
"Division number" is the number of divisions of the tracking matrix (the number of partial tracking matrices)
Where the number of divisions = 1 corresponds to a case where division is not performed, that is, a case where a conventional factorization method is used. "Partial point cloud score"
Is the number of feature points of each partial tracking matrix, and the number in parentheses on the right side is the number of common point groups. The "execution time" is the calculation time (seconds) required from the division of the tracking matrix to the generation of each partial point cloud data, and the time required for the integration processing of the partial point cloud data (very short, on the order of milliseconds) Not included.
“Singular value decomposition” is the calculation time (seconds) for singular value decomposition, and the number in parentheses to the right indicates the ratio (%) of the calculation time for singular value decomposition to “execution time”. I have. Most of the calculation time in the factorization method is the calculation time for singular value decomposition.

Referring to FIG. 15, the effect of shortening the time is obvious. When the number of divisions = 1 (corresponding to the case of the conventional factorization method), a calculation time of 2 hours or more is required.
In the case of 6, the calculation time is reduced to about 6 minutes. When the number of divisions is 20, the calculation time is further greatly reduced, and the entire circumference point group data can be generated in a short time of about 90 seconds.

[0040]

As is apparent from the above description, according to the present invention, even if the tracking matrix contains missing values which are difficult to estimate if the entire tracking matrix is processed collectively. ,
Point cloud data required for restoring the three-dimensional shape of the target object can be reliably generated. Surrounding point group data can also be easily generated. The calculation time for generating high-density point cloud data can be greatly reduced, and the amount of memory consumed for processing can be reduced. It is possible to generate point cloud data capable of accurately restoring the shape of the target object including its surface irregularities. It is possible to obtain an effect of automatically selecting an optimal image for texture mapping of a mesh generated from point cloud data.

[Brief description of the drawings]

FIG. 1 is a diagram schematically illustrating a process from full-scale imaging of a target object to generation of full-circumference point group data.

FIG. 2 is a schematic configuration diagram of a three-dimensional shape information generation system according to the present invention.

FIG. 3 is a flowchart illustrating a processing flow of a tracking matrix generation unit.

FIG. 4 is a flowchart illustrating a processing flow of a tracking matrix division unit.

FIG. 5 is an explanatory diagram of a tracking matrix shaping process.

FIG. 6 is an explanatory diagram of division of a tracking matrix into partial tracking matrices.

FIG. 7 is a diagram illustrating an example of a tracking matrix after a shaping process.

FIG. 8 is a diagram showing an example of dividing the tracking matrix shown in FIG. 7 into partial tracking matrices.

FIG. 9 is a diagram showing seven partial tracking matrices generated from the tracking matrix shown in FIG. 7;

FIG. 10 is an explanatory diagram of a method of estimating a missing value.

FIG. 11 is a flowchart illustrating an example of a missing value estimation complementing process.

FIG. 12 is a flowchart illustrating a processing flow of a partial three-dimensional data generation unit.

FIG. 13 is a flowchart illustrating a processing flow of a partial point cloud integration unit.

FIG. 14 is a block diagram illustrating an example of a computer for realizing the present invention with software.

FIG. 15 is a diagram showing experimental data for clarifying the effect of shortening the calculation time according to the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 image capturing device 101 storage device 102 image input device 103 two-dimensional operation unit 104 tracking matrix construction unit 105 tracking matrix division unit 106 three-dimensional operation unit 107 partial three-dimensional data generation unit 108 partial point group integration unit 109 data output unit 110 storage Apparatus 403 Recording medium

Claims (9)

[Claims] A step of generating a tracking matrix of image feature points from an image sequence of an image of a target object; a step of dividing the tracking matrix to generate a plurality of partially overlapping partial tracking matrices; Estimating missing values and generating a supplementary partial measurement matrix, generating partial point cloud data from the partial measurement matrix, and integrating partial point cloud data generated from the plurality of partial measurement matrices. A three-dimensional shape information generating method. 2. A method of generating a tracking matrix of image feature points from an image sequence obtained by photographing a target object, a step of dividing the tracking matrix to generate a plurality of partially overlapped partial tracking matrices, Generating a partial measurement matrix supplemented by estimating missing values, generating partial point cloud data and camera posture data from the partial measurement matrix, and a partial measurement matrix based on the camera posture data and the imaging direction of the target object. A three-dimensional shape information generating method, comprising the steps of: performing unevenness correction on partial point cloud data generated from the data; and integrating the partial point cloud data after unevenness correction generated from a plurality of partial measurement matrices. . 3. A step of generating a tracking matrix of image feature points from an image sequence obtained by photographing a target object; a step of dividing the tracking matrix to generate a plurality of partially overlapping partial tracking matrices; Estimating missing values and generating a supplementary partial measurement matrix, generating partial point cloud data and camera attitude data from the partial measurement matrix, integrating partial point cloud data generated from the plurality of partial measurement matrices 3. A method for generating three-dimensional shape information, comprising the steps of: selecting a texture mapping image based on camera posture data. 4. The three-dimensional shape information generating method according to claim 1, wherein in the step of generating the partial tracking matrix, the tracking matrix is divided after being shaped. 5. A computer-readable recording medium having recorded thereon a program for causing a computer to execute each processing step of the method for generating three-dimensional shape information according to claim 1, 2, 3, or 4. 6. A means for inputting an image sequence obtained by capturing an image of a target object, a means for generating a tracking matrix of image feature points from the input image sequence, and a plurality of partial tracings in which the tracking matrix is divided and partially overlapped Means for generating a matrix, means for estimating missing values of the partial tracking matrix to generate a partial measurement matrix supplemented, means for generating partial point cloud data from the partial measurement matrix, parts generated from a plurality of partial measurement matrices A three-dimensional shape information generating device comprising means for integrating point cloud data. 7. A means for inputting an image sequence obtained by capturing an image of a target object, a means for generating a tracking matrix of image feature points from the input image sequence, and a plurality of partial trackings obtained by dividing the tracking matrix and partially overlapping each other. Means for generating a matrix, means for estimating missing values of the partial tracking matrix to generate a supplemented partial measurement matrix, means for generating partial point cloud data and camera attitude data from the partial measurement matrix, and from a plurality of partial measurement matrices A three-dimensional shape information generating apparatus comprising: means for integrating generated partial point cloud data; and means for selecting a texture mapping image based on camera posture data. 8. The method according to claim 6, wherein the means for generating partial point cloud data corrects unevenness of the partial point cloud data based on the camera posture data and the shooting direction of the target object. A three-dimensional shape information generation device. 9. The method according to claim 6, wherein the means for generating the partial tracking matrix divides the tracking matrix after shaping it.
Or the three-dimensional shape information generating device according to 8.