JP2003067726A - Solid model generation system and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the reliability of positional relation data (calibration data) on a camera and a turntable in a solid model generation system. SOLUTION: A positional relation between the camera and the turntable on which an object is placed is acquired (calibration), and according to the calibration data, a data processing part 14 computes three-dimensional data on the object. A reference mark is set in the background of the object, and according to a change in the position of the reference mark, the calibration data are corrected, so that reliability of the calibration data is ensured.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional modeling device or a three-dimensional model generation device for acquiring three-dimensional information of a real object, and more particularly to a calibration for obtaining a positional relationship between a rotary table on which an object is placed and a camera. Regarding

[0002]

2. Description of the Related Art Conventionally, as a three-dimensional data input device (three-dimensional scanner), (a) a device using laser light (light cutting method) (b) a device using pattern light such as slit light (coded light) Method) (c) There is an apparatus that uses an image captured by a camera (stereo method, silhouette method) or the like.

In any of these methods, when it is attempted to input all-around data of an object, the object is placed on a computer-controlled rotary table, and the input work is performed by rotating this on a certain angular step. .

For example, in the case of the stereo methods (a), (b), and (c), partial three-dimensional data can be obtained at each rotation step, but eventually a process for integrating these is required. Becomes This is actually manual work using an editing tool on the host computer (personal computer or workstation). Therefore, there is a problem that a great deal of time and skill required for editing work is required.

On the other hand, in the case of the silhouette method in (c), an image of the entire circumference of the object is obtained, and then three-dimensional data of the entire circumference of the object is obtained. At this time, the input unit (for example, a digital camera) is used. A calibration process for accurately obtaining the positional relationship between the rotary table and the rotary table is essential. Of course, (a),
Even in the stereo methods (b) and (c), the integration process can be automated if the positional relationship between the input unit (for example, a digital camera) and the turntable is obtained by a calibration process rather than manually.

[0006] Specifically, the calibration is performed by placing a panel on which a known pattern is printed on a turntable, shooting a plurality of images while rotating the turntable, and having the host computer automatically calculate the images. After that, the panel is removed, and instead, the object is input on the turntable to input the three-dimensional data.

[0007]

However, (1) the calibration data generally contains an error, and (2) the work of replacing the panel and the object occurs between the calibration process and the actual object input. , The positional relationship between the turntable and the camera may change subtly (3) The positional relationship between the turntable and the camera may change subtly due to vibration while inputting an object (4) Depending on the camera, the light may be changed after each shooting Axis misalignment occurs,
As a result, there is a large error in the integrated data (calculated three-dimensional data in the case of the silhouette method) because the calibration data contains a large error and the positional relationship between the rotating table and the camera changes. There was a problem of being included.

The present invention has been made in view of the above problems, and an object thereof is to improve the reliability of calibration data.

[0009]

In order to achieve the above object, the present invention is a device for generating three-dimensional data of an object, and means for inputting an image of the object rotated by a rotating means, And a means for calculating positional relationship data between the rotating means and the image input means, a means for calculating the positional relationship data error, and a means for generating the three-dimensional data based on the positional relationship data error. Characterize.

In this apparatus, it is preferable that the means for generating the three-dimensional data generate the three-dimensional data by correcting the input image based on the positional relationship data error.

Further, in the present apparatus, the means for generating the three-dimensional data corrects the positional relational data based on the positional relational data error, and generates the three-dimensional data using the corrected positional relational data. Is preferred.

The means for generating the three-dimensional data has means for extracting a reference mark from the input image, and the positional relationship data error can be calculated based on the extracted reference mark position.

The means for generating the three-dimensional data has means for extracting the silhouette information of the object from the input image, and calculates the positional relationship data error based on the position of the silhouette information. You can also

The means for generating the three-dimensional data includes means for extracting the first silhouette information of the object from the input image and a plurality of the plurality of the first images extracted from a plurality of the input images having different photographing angles. 1 means for generating three-dimensional shape data of the object based on the silhouette information,
Means for extracting second silhouette information based on the three-dimensional shape data, and the first silhouette information and the second silhouette information.
The positional relationship data error can also be calculated based on the difference in silhouette information.

The present invention also provides a method for generating three-dimensional data of an object. This method includes a step of placing an object on a turntable and rotating the image, a step of capturing an image of the rotating object with a camera, and an image of the positional relationship between the camera and the turntable. It is characterized by including a step of calculating, a step of calculating the positional relationship data error, and a step of generating the three-dimensional data based on the positional relationship data error.

In the method, in the step of generating the three-dimensional data, it is preferable that the input image is corrected based on the positional relationship data error.

In the method, in the step of generating the three-dimensional data, the positional relationship data is corrected based on the positional relationship data error, and the corrected positional relationship data is used to generate the three-dimensional data. Is preferred.

In the step of generating the three-dimensional data, a reference mark can be extracted from the input image, and the positional relationship data error can be calculated based on the extracted reference mark position.

In the step of generating the three-dimensional data, silhouette information of the object may be extracted from the input image and the positional relationship data error may be calculated based on the position of the extracted silhouette information. it can.

The step of generating the three-dimensional data includes the step of extracting the first silhouette information of the object from the input image and the plurality of the first images extracted from the plurality of input images having different photographing angles. The first silhouette information and the second silhouette; and a step of generating three-dimensional shape data of the object based on one silhouette information, and a step of extracting second silhouette information based on the three-dimensional shape data. The positional relationship data error can also be calculated based on the difference in information.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment> FIG. 1 shows an overall configuration of the present apparatus. This apparatus is composed of a rotating table, a background plate, a reference marker, an input unit 10 including an image capturing unit such as a digital camera, a control unit 12, and a data processing device 14.

The control unit 12 controls the rotation of the turntable and the camera.

The data processing unit 14 processes input data (for example, an image) and calculates three-dimensional data (shape and color information) of the object. In addition, here, as a calculation method of three-dimensional data, the well-known silhouette method and its extension method (for example, the document “A Portable 3D D
igitizer ”International Conference on Recent Adv
ances in 3-D Digital Imaging and Modeling, pp.197-
204 (1997)).

FIG. 1B shows a block diagram of the configuration of the data processing device 14. The data processing device 14
It has an interface I / F, a CPU, a ROM, an image memory, and a positional relationship data memory.

The image data of the object obtained by the input unit 10, that is, a plurality of image data having different photographing angles according to the rotation angle of the rotary table, which will be described later, is the interface I / F.
Is stored in the image memory via.

The CPU reads out the image stored in the image memory and executes a predetermined three-dimensional calculation. The three-dimensional calculation is performed according to the positional relationship between the camera and the object, which is stored in advance in the positional relationship data memory, more specifically, the positional relationship between the camera and the turntable. The positional relationship between the camera and the turntable is acquired by the calibration process described later. Also, C
When there is an error in the positional relationship between the camera and the turntable, the PU corrects the positional relationship data stored in the positional relationship data memory so as to eliminate the error according to a program stored in advance in the ROM.

Although the silhouette method is used in this embodiment, the present invention is not limited to this, and various three-dimensional shape reconstruction algorithms such as a stereo method may be applied.

The three-dimensional data input work and the three-dimensional data calculation work using the apparatus having the above configuration are as follows.

(1) Calibration photographing and calculation (2) Object shooting (3) Silhouette extraction (4) Voxel generation (5) Polygon generation (6) Texture acquisition These will be sequentially described below.

<Calibration Shooting and Calculation> In the calibration shooting, as shown in FIG. 2, the panel 50 on which the circle mark is arranged is placed on the rotary table 10e.
Here, it is assumed that the position of the circle mark is known in advance. Then, a plurality of images are taken while rotating the panel 50 on the turntable 10e within a certain range.
The calibration process includes (i) determining the positional relationship between the camera 10b and the panel 50 (ii) based on the result of step (i), the camera 10b
And two steps of obtaining the positional relationship between the rotary table 10e.

As a method for easily obtaining the positional relationship between the camera 10b and the panel 50, an object whose shape is known (reference object)
Is shown to the camera 10b, and the position of the camera 10b is obtained using Hough transform. A circle mark is drawn on the surface of the reference object that also serves as the panel 50, and the three-dimensional position of each is known.

Intuitively, the Hough transform is
This is a method of obtaining the target parameter by the majority rule based on the voting process to the parameter space based on the global parameter value candidates calculated based on the local information. Originally, it was proposed in the problem of finding a straight line from an edge image. Here, for each point in the edge image, if it is assumed to be part of a straight line, then for every possible straight line the distance from the origin ρ
And voting processing is performed in a parameter space formed by an angle θ formed with the y axis. Finally, the peak value (ρ, θ) in the parameter space becomes a straight line to be obtained.

In calibration, this Hough
Based on the idea of conversion, coordinate conversion parameters (x,
y, z, α, β, γ) is calculated. The coordinate conversion parameters are conversion parameters of the coordinate system of the reference object and the coordinate system of each camera.

Here, it is assumed that the internal parameters of the camera such as the focal length and the circular pattern position for calibration are known. The outline of acquisition of coordinate conversion parameters by Hough conversion is as follows.

(1) A reference object on which a known reference pattern (circular pattern) is arranged is photographed.

(2) A reference pattern is extracted from the image.

(3) Using three arbitrary reference patterns extracted from the image and three reference pattern position data registered in advance in the system, coordinate conversion parameters (translation components x, y, z, and rotation components) are used. Calculate α, β, γ). Do this for all combinations and vote in the 6-dimensional Hough space.

(4) The peak value is extracted from the Hough space to obtain the coordinate conversion parameter. Through the above steps, conversion parameters for the reference object coordinate system and the camera coordinate system are obtained.

The accuracy of the conversion parameter obtained by the above method depends on the Hough spatial resolution. In general, it is not accurate enough to be directly applied to modeling. Therefore, it is preferable to use the above parameters as initial values and further obtain final coordinate conversion parameters by the nonlinear least squares method (Levenberg-Marquardt method).

Also, step 3 in Hough transformation
The combination processing can be reduced by color-coding the circle marks, and as a result, the processing speed can be increased.

As described above, the positional relationship between the camera and the panel can be obtained. Next, the positional relationship between the camera and the turntable is obtained. Here, when observing the movement of the panel viewed from the camera, that is, the movement of the origin of the panel coordinate system, attention is paid to the point that a circular movement about the rotation axis of the turntable is performed. That is, an arbitrary point on the panel is selected and a circle in the three-dimensional space is fitted to the locus of the movement. The center axis of this circle corresponds to the rotation axis. Since the positional relationship between the rotary base and the camera is nothing but the positional relationship between the rotary shaft and the camera, the positional relationship between the rotary base and the camera is obtained as described above.

<Object Shooting> In object shooting, the panel used in the calibration shooting is removed from the rotary table, the object is placed on the rotary table instead, and the rotary table is specified in steps (for example, every 10 degrees). While turning, input the omnidirectional image of the object. Here, the background plate is appropriately selected according to the color of the object. This is necessary in the process of silhouette extraction described below. For example, in the case of an object that does not contain blue, the background plate may be a blue background plate, for example. Then, the three-dimensional data calculation process is performed.

<Silhouette Extraction> For the silhouette extraction processing, for example, the well-known chroma key method can be used. This is a technique often used in broadcasting stations and the like, and is a method for synthesizing a plurality of videos to create a new video.

The images obtained by the camera are images of the background plate and the target object. In the case of a target object that does not include blue, for example, a blue background plate can be used as the background plate. In this case, if the blue part is removed from the image, only the part of the target object can be cut out.

FIG. 3 schematically shows the silhouette extraction processing. In the image A1 obtained by photographing the target object 100 on the turntable 10e, the target object 100 and the blue background are present as shown in (A). By removing only this blue background, it is possible to extract only the image of the target object 100 as shown in FIG.
Blue is black, other than blue is represented by white).

Of course, instead of extracting the silhouette using chroma key processing, for example, Japanese Patent Application No. 9-23261.
As disclosed in No. 7, the silhouette may be extracted by the difference processing between the background image and the object image.

<Voxel Generation> Since the field of view of the camera is limited in advance, it is possible for the measurer to set the possible area in advance. This possible area is defined as a voxel (voxe
l) Express in space. Here, the voxel space is a representation of a three-dimensional space as a collection of fine solids (voxels) so that a two-dimensional image is represented by a set of surface elements (pixels). Most simply, the three-dimensional shape can be reconstructed as follows. That is, each voxel forming the voxel space is projected onto one silhouette image. At this time, if the voxel is projected outside the object silhouette, it can be determined that the voxel is clearly not the target object.

This processing is performed for all silhouettes, and for all silhouette images, only the voxels projected inside the object silhouette are extracted, and this is determined as the target object 100.

Intuitively, this process can be considered as a method in which the gypsum block 200 corresponding to the object existing region is assumed and the portion corresponding to other than the silhouette region is gradually cut. Then, the finally remaining plaster part is regarded as the three-dimensional shape of the object.

When the silhouette image has an error, a large shape error occurs in the final result in the above process. In such a case, the voting method (voting method) described below is effective. FIG. 5 conceptually shows this.

That is, instead of immediately deleting the portion outside the silhouette area, first, the points are added to the space corresponding to the silhouette area. Then, the part where the score finally exceeds a certain threshold value is regarded as the object part.

The flow of the voting process is as follows. Note that here, as described above, it is premised that the object existing region is expressed as a voxel space.

(1) Initialization: The score of each voxel is set to zero.

(2) For each voxel, the projection points on the entire silhouette image are obtained, and the number of projection points included in the silhouette area is used as the point.

(3) Voxels whose scores are equal to or greater than a certain value are regarded as an object region.

FIG. 5 schematically shows the voting method. (A), (B), and (C) are obtained by adding 1 to the projection points included in the silhouette area in the three directions. (D) is the addition of these projection points, and the total of the projection points of each voxel is shown by a number.
The voting result is indicated by, for example, numbers 10 to 16. Since the number of voxels in which the target object 100 truly exists is large, the voxels can be extracted as shown in (E) by extracting the voxels having a certain value or more, for example, 16 or more.

An important point in this processing is information about where the viewpoint of the camera exists with respect to the voxel space. If this information is incorrect, the calculation result of the three-dimensional shape will be significantly different from the actual result.
If the voxel space is defined according to the coordinates of the turntable,
The positional information of the viewpoint of the camera is nothing but the positional relationship between the turntable and the camera. By the way, the positional relationship between the turntable and the camera is fixed as described above. Therefore, as this data, calibration data registered in the data processing unit 14 in advance may be used.

<Polygon Generation> The three-dimensional shape data of the target object 100 is once obtained as a set of voxel data, but in a general shape measurement tool, it is assumed that the target object is represented by polygon data. There is also. In this case, it is necessary to convert the voxel data 300 into polygon data 400 as shown in FIG.
There are also cases where it is necessary to show the measurement shape three-dimensionally with display software (viewer). Also in this case, it is effective to convert the voxel data into polygon data.

Here, it is required to keep the number of polygons required for expression small and to maintain the accuracy of the expression shape. For example, a suitable polygon expression is possible by taking the steps shown below.

(1) Vertex setting Adjacent vertices of voxels corresponding to the object surface are connected to generate an initial polygon.

(2) The number of polygons is reduced to the target value by evaluating the shape change before and after the polygon reduction merge and merging adjacent polygons with small changes in order.

<Texture Generation> This is a step of acquiring color (texture) information of all polygons based on the omnidirectional photographed image. Specifically, after determining an input image (hereinafter referred to as a reference image) to which texture information of each polygon is given, the polygon is projected on the reference image, and the color of the projected portion is used as the texture information. It should be noted here that, for most polygons, there are multiple input images whose regions are visible. Therefore, it is necessary to determine an appropriate reference image for each polygon. Points to be considered in determining an appropriate reference image are (1) selecting a reference image with a large amount of texture information (2) improving the continuity of the image on the polygon boundary line.

From the viewpoint of (1), it is desirable that the polygon projection area on the reference image is large. On the other hand, (2)
From the viewpoint of, it is desirable that the reference images of adjacent polygons are the same, and even if they are different, it is desirable that the shooting angle difference is small. Since these required points are often in a trade-off relationship, the evaluation function of each consideration point is determined, and the reference image is determined so that the sum function thereof becomes maximum. Incidentally, such a method is disclosed in Japanese Patent Application No. 9-234829. Through the above processing, the three-dimensional data of the target object can be acquired.

By the way, in the above processing, it is assumed that the positional relationship between the rotary base and the camera remains unchanged from the start of the calibration photographing to the end of the object photographing.

However, in reality, a panel-object replacement work occurs between the calibration process and the actual object input, and at that time, the positional relationship between the turntable and the camera may be slightly changed. Also, during input of an object, the positional relationship between the turntable and the camera may change subtly due to vibration, etc. Furthermore, some cameras may shift the optical axis at each shooting, resulting in a large error in the calibration data. May be included, or the positional relationship between the turntable and the camera may change, and thus the obtained calibration data itself may include an error.

If the three-dimensional shape calculation is performed in this state, the generated three-dimensional data will include a large error.

On the other hand, the calibration data containing an error has a very small difference from the true calibration data, and therefore a reference mark whose position is fixed is introduced into the rotary table, and at the time of data input. The correction processing can be performed by capturing an image including the reference mark together with the target object.

Now, as a premise, it is assumed that the positional relationship between the background plate and the turntable remains unchanged from the start of the calibration shooting to the end of the object shooting. Then, it is assumed that the reference mark is arranged on the background plate.

Reference marks are illustrated in FIGS. 7 and 8. In FIG. 7, the reference mark 16 is attached to the same base as the turntable 10e, and the reference mark 16 is rotated and “raised” from the base (arrow in the figure) to arrange the reference mark on the background plate 10d. This is an example. Also, FIG. 8 shows the background plate 1.
In this example, the reference mark is formed on 0d itself. Although one reference mark may be provided, a plurality of reference marks are preferable. Also,
The reference mark 16 is a color that can be easily distinguished from the background,
It is suitable to be arranged near the edge of the input image. The reason is that there is usually an object around the center of the input image,
This is because it is easy to hide the mark, so you should avoid arranging around the center. In addition, when it exists in the surroundings, silhouette parts other than the object will appear in the silhouette extraction, but in general, in the voxel generation described above, even if silhouette noise exists, the distance between the noise and the object is This is because the farther away they are, the smaller the adverse effects will be. It is desirable to set a window surrounding these reference mark positions (two-dimensional) in the input image. In the above, the device in which the camera and the turntable are integrated has been described, but they may be configured separately.

Next, the outline of the calibration data correction process using the reference mark 16 will be described below. Note that, for easy understanding, in addition to correction of the calibration data itself, image correction (parallel movement and enlargement / reduction) is also treated as “calibration data correction”.

(1) Calibration shooting (2) Object image input (3) Calibration shooting image and reference marks existing in the setting window are extracted from the input image. FIG. 9 illustrates a window 500 set in the input image and a reference mark 600 in the window 500.

(4) One of the reference marks obtained in step (3) is determined as a "reference reference mark".

(5) An object other than the "reference reference mark" in step (4) is set as an "other reference mark", and this is compared with the reference reference mark to find the difference in position. FIG. 10 illustrates the standard reference mark 600 and the other reference mark 610, and shows the difference between the positions of the two marks.

(6) Correction processing is performed based on the position difference obtained in step (5).

The following two types of methods can be considered for the correction processing.

(A) Two-dimensional correction process (b) Correction of calibration data itself The two-dimensional correction process is a process for directly correcting the input image. Originally, if the positional relationship between the camera and the turntable (the positional relationship between the background plate and the turntable is unchanged) is unchanged, the above-mentioned standard reference mark position and all other reference mark positions should be unchanged. If they are different, the positions of the camera and the turntable are slightly deviated. Therefore, the image is translated so that the positional relationship between the camera and the turntable does not change, that is, the reference reference mark position and all other reference mark positions match. Then, three-dimensional data calculation processing is performed based on the image subjected to the parallel movement processing.

The difference between the position of the other reference mark and the position of the standard reference mark does not always reflect the parallel movement error of the camera position, but actually reflects the rotational movement error of the camera. Often But,
Since the difference is almost negligible on the image, in this method, the error caused by either of them is treated as a translation error.

On the other hand, the correction of the calibration data itself is to change the calibration data itself, assuming that the difference between the position of the other reference mark and the position of the standard reference mark is caused by the rotational movement error of the camera. Here, the positions of the standard reference mark and the other reference marks need to match with each other in the image captured by the calibration. This is because the calculated calibration data itself is meaningless. Under this condition, the correction amount of the calibration data is calculated from the other reference mark positions in the object input image. Specifically, when the focal length f is standardized to 1.0, assuming that the position of the actual reference mark on the CCD of the camera (digital camera) is Xs and the position of the standard reference mark is Xr, FIG. As shown in

## EQU1 ## θ-θ '= arctan (Xr) -arcta
n (Xs) is an approximate value of the rotation angle error. After correcting the calibration data by this angle, the three-dimensional data calculation process may be performed.

Further, by disposing a plurality of reference marks, it is possible to correct the error caused by the forward and backward movement of the camera. If there is no forward / backward movement of the camera, the distance between the reference marks is constant. However, if the distance between the plurality of reference marks becomes large, for example, it is determined that the position of the camera is close to the turntable side. Similar to the above, this correction is also performed by (a) correcting the input image so as to match the registered calibration data by image reduction processing, and (b) obtaining the moving distance of the camera and correcting the calibration data itself. It can be corrected depending on the method.

FIG. 12 schematically shows the method (b). The distance between the reference marker 16 and the camera when the position of the camera (focal position) changes from P to P ′ is L ′, the distance on the image between the reference marks 16 is W ′, which is registered in the data processing device 14. If the reference distances are L and W respectively,

## EQU00002 ## The moving distance can be calculated by L '= W.L / W', and the calibration data can be corrected using this.

The positional relationship between the reference mark 16 and the rotary table also changes so that the positional relationship between the camera and the rotary table changes subtly. However, with respect to the error that affects the three-dimensional calculation, generally, the parallel movement amount is extremely small, and in reality, the rotation error of the camera is large. Therefore, it can be considered that the slight deviation of the position of the reference mark 16 has almost no effect.

The position of the actual reference mark in the correction process using the reference mark 16 may be detected for all input images or for a specific image.

In particular, if it is possible to assume that the position of the camera does not change during the input work, it is preferable to actually detect the position of the reference mark only for a specific image from the viewpoint of shortening the calculation time. .

<Second Embodiment> The method for correcting the calibration data by using the reference mark 16 has been described above. However, the calibration data correction can be performed without using the reference mark 16.

Now, as shown in FIG. 13, the target object 10
It is considered that 0 is placed on the rotary table 10e and the data of the entire circumference is photographed at regular steps. Here, a specific point of the object makes a three-dimensional circular motion. Therefore, an elliptical orbit is drawn on the input image. Furthermore, after extracting the silhouette of the object, it can be considered that the movement of the center of gravity on the image also draws an elliptical orbit. Then, the center of the ellipse (denoted by Ce) lies on the straight line Lc obtained by projecting the rotation axis of the rotary table 10e on the camera image as shown in FIG. 13B.

The center Ce of this ellipse can be obtained, for example, as the average position of all the centroids of the silhouette of the object.

Therefore, after the silhouette extraction processing, Ce is calculated as described above, and is compared with the projected straight line Lc of the rotation axis obtained from the calibration data registered in the memory. If Ce is above Lc, calibration data correction is unnecessary. On the other hand, as shown in FIG. 14, if Ce deviates from Lc, two-dimensional correction or calibration data correction may be performed accordingly.

Note that this processing only corrects the camera rotation error in the direction perpendicular to the rotary table rotation axis (usually in the horizontal direction in the image). In addition to the camera rotation error, there may be an error in the direction along the rotation axis.
However, in the three-dimensional data calculation, the influence of the error in the direction perpendicular to the rotation axis is large, while the influence of the error in the direction along the rotation axis is extremely small. Therefore, practically, it is often sufficient to correct the error in the direction perpendicular to the rotation axis. However, in this case, the condition that the positional relationship between the camera and the turntable is unchanged in the calibration photographing is required.

This is because the calculated calibration data itself is meaningless.

<Third Embodiment> In the above-described second embodiment, the calibration data correction is performed by using the information of the silhouette image. However, when the extracted silhouette is noisy, such as when an unnecessary object is reflected in the background, the method of the second embodiment may not work well. In this case, calibration correction using voxel generation processing is possible.

Now, based on the registered calibration data including the error in the direction perpendicular to the rotary table rotation axis,
Consider a case where voxel generation processing is performed. In this case, due to the influence of the error, the generated three-dimensional data has a shape thinner than the actual object.

In FIG. 15, two viewpoints a and b which are 180 degrees opposite to each other are included in the case where the error is not included in the direction perpendicular to the rotation axis of the turntable 10e (A) and the case where the error is included (B). The silhouette image obtained from is shown. When an error is included in the direction perpendicular to the rotation axis, the target object 100 moves by the amount of the error at the viewpoint b, and the silhouette image is also blurred. Therefore, when the voxels of the target object 100 are extracted from the silhouette images obtained at the viewpoints a and b, the shape 800 actually obtained rather than the true shape 700 is proportional to the error of the rotation axis as shown in FIG. Becomes smaller. Therefore, by performing the following processing,
Correct the error.

(1) Voxel generation is performed based on the registered calibration data.

(2) A silhouette image (referred to as a re-silhouette image or a second silhouette image) is generated by projecting the generated voxel data on a camera image.

(3) As shown in FIG. 17, the silhouette image obtained by the silhouette extraction processing is compared with the re-silhouette image (second silhouette image) obtained in (2).

(4) In step (3), the difference between the silhouette image and the re-silhouette image (the re-silhouette image does not become larger but is the same or smaller)
Ask for.

The difference obtained in step (4) above corresponds to the error in the calibration data. Two-dimensional correction or calibration data correction may be performed by the amount of this error. Also in this case, the condition that the positional relationship between the camera and the turntable is unchanged in the calibration photographing is required. This is because the calculated calibration data itself is meaningless.

Although the embodiment of the present invention has been described above, the present invention is not limited to this, and other modifications are possible. For example, the background board 10d can be replaced with a plurality of colors. Generally, with the simple silhouette method, it is difficult to input an object having the same color as the background color. However, as shown in Japanese Patent Application No. 11-322098, by inputting using a plurality of background plates having different hues, it is possible to input an object of any color.
Further, by adding white as a background color, (1) silhouette extraction for creating shape data is performed based on an image using a colored (for example, blue) background plate (2) texture acquisition processing Can be selectively used based on an image using a white background plate.

When an image is input using a colored background plate, the background color may be reflected in the peripheral portion of the object depending on the material of the object. When texture acquisition processing is performed based on such an image, the quality of texture information of the finally obtained three-dimensional data may be insufficient. On the other hand, if texture acquisition is performed based on a white background image,
It is possible to minimize the adverse effect of the background color reflected in the peripheral portion of the object as described above.

[0101]

As described above, according to the present invention,
It is possible to improve the reliability of the calibration data and generate a highly accurate three-dimensional model.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram of an embodiment.

FIG. 2 is an explanatory diagram of calibration.

FIG. 3 is an explanatory diagram of silhouette extraction.

FIG. 4 is an explanatory diagram of voxel generation.

FIG. 5 is an explanatory diagram of a voting method.

FIG. 6 is an explanatory diagram of conversion from voxel data to polygon data.

FIG. 7 is an explanatory diagram of a reference mark.

FIG. 8 is an explanatory diagram of another reference mark.

FIG. 9 is an explanatory diagram of an input image when a reference mark is used.

FIG. 10 is an explanatory diagram of a positional relationship between a standard mark and a reference mark.

FIG. 11 is an explanatory diagram (1) of calibration data correction based on a rotation error of the digital camera.

FIG. 12 is an explanatory diagram of calibration data correction based on a front-back movement error of the digital camera.

FIG. 13 is an explanatory diagram (part 2) of calibration data correction based on a rotation error of the digital camera.

FIG. 14 is an explanatory diagram showing the relationship between the center of the ellipse in FIG. 13 and the rotation axis of the turntable.

FIG. 15 is an explanatory diagram (part 3) of calibration data correction based on rotation error of a digital camera using voxel generation processing.

16 is an explanatory diagram of a difference between the true shape and the actual shape in FIG.

FIG. 17 is an explanatory diagram of a difference between a silhouette image and a re-silhouette image.

[Explanation of symbols]

10 input unit, 12 control unit, 14 data processing unit.

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 2F065 AA04 AA53 BB05 FF04 JJ03                       JJ19 JJ26 MM04 PP13 QQ00                       QQ18 QQ24 QQ27 QQ31 QQ42                 5B057 AA20 BA02 BA21 DB03 DB06                       DB09 DC13 DC16

Claims (12)

[Claims] 1. An apparatus for generating three-dimensional data of an object, said means for inputting an image of an object rotated by a rotating means, and means for calculating positional relationship data between said rotating means and said image input means. And a means for calculating the positional relationship data error, and a means for generating the three-dimensional data based on the positional relationship data error. 2. The apparatus according to claim 1, wherein the means for generating the three-dimensional data generates the three-dimensional data by correcting an input image based on the positional relationship data error. Stereo model generator. 3. The apparatus according to claim 1, wherein the means for generating the three-dimensional data corrects the positional relationship data based on the positional relationship data error, and uses the corrected positional relationship data to perform the cubic A three-dimensional model generation device characterized by generating original data. 4. The apparatus according to claim 2, wherein the means for generating the three-dimensional data includes means for extracting a reference mark from the input image, and the extracted reference A stereo model generation device, wherein the positional relationship data error is calculated based on a mark position. 5. The apparatus according to claim 2, wherein the means for generating the three-dimensional data has a means for extracting silhouette information of the object from the input image, A stereo model generation device, characterized in that the positional relationship data error is calculated based on the position of the silhouette information. 6. The apparatus according to claim 2, wherein the means for generating the three-dimensional data includes means for extracting first silhouette information of the object from the input image, Means for generating three-dimensional shape data of the object based on the plurality of first silhouette information extracted from different plurality of the input images, and means for extracting second silhouette information based on the three-dimensional shape data And the positional relationship data error is calculated based on the difference between the first silhouette information and the second silhouette information. 7. A method for generating three-dimensional data of an object, comprising placing the object on a turntable and rotating the object, and capturing the image of the rotating object with a camera. A step of calculating positional relationship data between the camera and the turntable, a step of calculating the positional relationship data error, and a step of generating the three-dimensional data based on the positional relationship data error. A method for generating a stereo model, characterized by having. 8. The stereo model generation method according to claim 7, wherein in the step of generating the three-dimensional data, the input image is corrected based on the positional relationship data error. 9. The method according to claim 7, wherein the step of generating the three-dimensional data corrects the positional relationship data based on the positional relationship data error.
A three-dimensional model generation method comprising generating the three-dimensional data by using the corrected positional relationship data. 10. The method according to claim 8, wherein in the step of generating the three-dimensional data, a reference mark is extracted from the input image, and the reference mark is extracted based on the extracted reference mark position. A stereo model generation method, characterized in that a positional relationship data error is calculated. 11. The method according to claim 8, wherein in the step of generating the three-dimensional data, silhouette information of the object is extracted from the input image, and the silhouette information of the extracted silhouette information is extracted. A stereo model generation method comprising calculating the positional relationship data error based on a position. 12. The method according to claim 8, wherein the step of generating the three-dimensional data includes a step of extracting first silhouette information of the object from the input image, Generating three-dimensional shape data of the object based on the plurality of first silhouette information extracted from different input images, and extracting second silhouette information based on the three-dimensional shape data And the positional relationship data error is calculated based on the difference between the first silhouette information and the second silhouette information.