JP2002163679A - Device and method for generating three-dimensional object model - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a three-dimensional object model generating method applicable to an object having a hole region and a plurality of objects and small in calculation load. SOLUTION: This generating method is provided with a stop area formation step for forming circles of a prescribed expansion radius which are mutually connected around each data point constituting a distance image and defining an area formed by such mutually connected circles as a step area, an initial closed surface formation step for forming a single initial closed surface enclosing the stop area outside the stop area, and a closed surface contraction step for contracting the initial closed surface until each component point constituting the initial closed surface is located within the stop area, and makes a closed surface obtained in the closed surface contraction step to be a three-dimensional object model.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional object model generating apparatus and a three-dimensional object model generating method.

[0002]

2. Description of the Related Art In recent years, a study has been carried out to measure the entire circumference of an object surface by integrating distance images acquired from a plurality of viewpoint directions (H. Zha, K. Morooka, and T. Hasegawa, "Activ Modeling").
of 3-D Objects: Planning on the Next Best Pose (NB
P) for Acquiring Range Images, "Proc.Int.Conf.on R
ecent Adbances in 3-D Digital Imaging and Modelin
g, pp. 68-75, 1997).

However, the data obtained in this way is merely a set of discrete coordinate points, and the amount of data is enormous.
In order to use this data for object recognition and graphics, it is necessary to convert it to a compact full-circle description.
This is called object modeling.

In such object modeling, an active contour model (Snakes) (M. Kass, A. Witkin, and D. Terzopoulos, "Snakes"
kes: Active Contour Models, "Int'l J. of Computer Vis
ion, pp. 321-331, 1998) and deformable surfaces (H. Delingette, M. Hebert, and
K. Ikeuchi, "Shape Representation and Image Segmenta
tion U sing Deformable Surfaces, "Proc.IEEE Conf.C
VPR, pp. 467-472, 1991; T. McInerney and D. Terzopoulo
s, "A Finite Element Model for 3D Shape Reconstruc
tion and Nonrigid Motion Tracking, "Proc.Int.Conf.o
n Computer Vision, pp. 518-523, 1993, K. Hara, H. Zha, an
d T.Hasegawa, "Regularization Based3-D Object Mod
eling from Multiple Range Images, "Proc.Int.Conf.o
n Pattern Recognition, Vol. 2, pp. 964-968, 1998) is widely used. This deformable surface is based on the principle of energy minimization, and has the advantage that a smooth and closed surface can always be obtained.

[0005] However, this deformable surface cannot adapt to changes in phase such as splitting and integration in the process of deforming the surface, and it is difficult to perform phase-adaptive modeling on an object having a hole or a scene including a plurality of objects. Problems have been pointed out.

In order to solve this problem, a contour surface method (Level
Set Method) (JASethian, LevelSet Methods and Fas
t Marching Methods, Cambrige University Press, 199
9. S. Osher and JASethian, "Fronts Propageting wit
h Curbature-Dependent Speed: Algorithms Based on
Hamilton-Jacobi Formulation, "J. Of Computational Ph
ysics, Vol. 79, pp. 12-49, 1988, Miichi Giga and Ungo Chen, chasing a moving surface, Nihon Hyoronsha, 1996) (hereinafter referred to as “contour surface modeling method”). ) (R.Mal
ladi, JASethian, and BCVemuri, "Shape Modeling wi
th Front Propagation: A Level Set Approach, "IEEE Tr
ans.PAMI, Vol.17, No.2, pp.158-175, 1995) has been proposed, and phase adaptive modeling has become possible. This contour surface method is based on surface evolution (surface evolu
In this method, a curved surface is regarded as a zero contour surface of a certain auxiliary function in a four-dimensional space to which one dimension is added, and an equation relating to the auxiliary function is analyzed. This makes it possible to track a surface after a point that cannot be differentiated due to division or the like, and to avoid self-intersection of the surface.

[0007]

However, the above-mentioned contour surface modeling method involves repetition of distance conversion and addition of dimensions, and is not efficient in terms of calculation amount and storage capacity.

Here, the contour surface modeling method will be outlined, and the problems of the contour surface modeling method will be described.

First, an outline of the contour surface method will be described in two dimensions for simplicity. Consider a case where a certain closed curve is deformed at time t. Let γ (t) be this curve and derive the equation of motion of γ (t) for the given initial closed curve γ (t = 0). Here, a component F in the outward normal direction of the velocity vector at each point of γ (t) is introduced. This F is called a growth rate and is an amount determined for each point of γ (t).

The basic idea of the contour surface method is that a curve γ (t) moving with time is converted into a higher-order auxiliary function z = φ (x, y,
It is to regard as zero contour of t) and embed this.
Now, an auxiliary function for a point X = (x, y) on a two-dimensional plane is defined as φ (X, t = 0) = ± d (1). Here, d is the shortest distance from X to the initial closed curve γ (t = 0). The right side has a positive sign when the point X is outside γ (t = 0), and a negative sign when the point X is inside.
Therefore, the initial value of φ (X, t) is γ (t = 0) = {X | φ (X, t = 0) = 0} (2) φ (X, t = 0): R ² → R is selected. In the following, the equation of motion for φ (X, t) will be derived.

If X (t) is a point on the curve γ (t) that moves with the passage of time t, obviously X (t = 0) is the initial curve γ
(t = O). That is, the zero contour of φ is γ (t)
The necessary and sufficient condition for becoming equal to φ (X (t), t) = 0 (3). Differentiating both sides of Equation (3) with t gives φ _t + ∇φ (X (t), t) · X ′ (t) = 0 (4). At this time, F is calculated by using the product of the moving speed X ′ (t) of X (t) and the outward unit vector n = γφ / | ∇φ |
Since it can be written as X ′ (t) · n, the equation of motion (4) for φ can also be written as φ _t + F | ∇φ | = 0 (5). Equation (5) is an equation derived by Osher and Sethian and is called a contour surface equation (Leve1 Set Equation) (S. Osher and JASethian, "Fronts Propaget
ing with Curbature-Dependent Speed: Algorithms Base
d on Hamilton- Jacobi Formulation, "J. of Computatio
nal Physics, Vol. 79, pp. 12-49, 1988.). The contour surface method
In this method, the contour surface equation is solved under certain initial conditions, and the zero contour of the obtained solution φ (X, t) is regarded as γ (t).

An advantage of the contour surface method is that even if a singular point occurs in the process of deforming γ (t), it is possible to track the curved surface thereafter. To explain this, FIG.
(c) and (d) show a state before and after a single smooth initial closed curve γ (0) is separated into two closed curves. When this curve separates, a non-differentiable point called a singular point occurs, at which point F cannot be determined and tracking after splitting becomes impossible. On the other hand, the auxiliary function z = φ (x,
y, t) always remains a smooth function, and γ is φ (x, y, t)
Tracking after separation is also possible by considering it as a zero contour line.

The procedure for tracking a curved surface is exactly the same as the above procedure. In this case, the auxiliary function u = Φ in a four-dimensional space is used.
By solving the contour surface equation relating to (x, y, z, t), a zero contour surface of the obtained solution is obtained.

Next, for the sake of simplicity, an outline of the contour surface modeling method will be described two-dimensionally in the same manner as described above. Contour surface modeling method (R. Mallad
i, JASethian, and BCVemuri, "Sha pe Modeling with
Front Propagation: A Level Set Approach, "IEEE Tr
ans.PAMI, Vol.17, No.2, pp.158-175, 1995) tracks closed curves that evolve in shape according to the equation of motion F = ± 1-εκ (6) using the contour surface method. There is an advantage that a plurality of contours can be extracted.

Here, κ is the average curvature of the point P on the curve φ (x, y, t) = O in the direction of the inward unit method vector, and ε is the weighting coefficient. When the right-hand side of the equation (6) consists of only the first term ± 1, the curve shows a constant speed 1 constant growth (+1 is expansion, −1
Evolves according to shrinkage). However, uniform growth has a property that a singular point is apt to be generated in the course of curve deformation. To avoid this as much as possible, a smoothing term −εκ is added. The equation of motion consisting of only the second term is called an average curvature flow equation.

By solving equation (6) using the contour surface method, it becomes possible to adapt to changes in the phase such as splitting and merging of curves, but the movement of the curves does not stop as it is.
Therefore, the equation (6) and the filter

[0017]

(Equation 1) F = k _I (± 1−εκ) (8) The growth rate F is defined again. Here, ∇G _σ * I represents the convolution of the target image I by the Gaussian function with the standard deviation σ. At this time, k _I (x, y) becomes almost zero where the gradient of the image sharply changes like a contour point, so that the curve stops moving near the contour.

The same applies to the case where this contour surface modeling method is applied to three dimensions. In this case, three-dimensional image data in which each voxel stores density values, such as CT data, is used. Becomes

The above-described contour surface modeling method has a problem that the amount of calculation becomes enormous, especially in three dimensions.

That is, the following procedure is taken when directly performing a numerical calculation of the contour surface modeling method. First, the three-dimensional space and time are finely divided, and the differential equation (5) is converted into a difference equation. Then, each time the time is advanced by the division width, the auxiliary function is updated according to the difference equation, and the auxiliary function is corrected according to equation (1) based on the obtained zero contour surface of the auxiliary function. At this time, it is necessary to find the distance from all the elements constituting the space to the zero contour plane, which requires a huge calculation cost. For example, consider a case in which the three-dimensional space is divided into a set of small cube cells, and n cells are arranged in each axis direction. At this time, the number of cells constituting the zero contour plane is proportional to n ² and the number of cells constituting the space is n ³ , so that the computational complexity of the above processing is O (n ⁵ ).

In order to reduce this enormous amount of calculation, the distance calculation is performed only on the neighboring cells on the zero contour plane.
and algorithm (DLChopp, "Computing Minimal Surfa
cesvia Level Set Curvature Flow, "J. of Computationa
l Physics, Vol. 106, pp. 77-91, 1993.). Assuming that the number of cells near the zero contour surface is κ, the computational complexity at this time is O (κn ² ), but it can still be said that the computational complexity is still enormous.

[0022]

Therefore, in order to solve the above-mentioned problem, the present invention defines a shape evolution of a curved surface using iterative morphological operation, and a phase adaptive object modeling method using the same. Suggest. Here, morphology (Hidefumi Obata, Morphology, Corona Co., 1996) performs a logical operation between the image obtained by moving the target image according to a set of position vectors called a structuring element and the original target image. It has been mainly used in the field of two-dimensional image processing. In the present invention, such a morphology is extended to a dynamic model for generating a shape evolution of a three-dimensional surface, and the surface is shape-evolved by iterative morphological operations. If the structuring element is convex and small enough,
Morphological operations such as (di1ation) and scraping (erpsion) produce small deformations of a surface according to a differential equation
(ABArehart, L. Vincent, and BBKimia, "Math ematic
al Morphology: The Hamilton-Jacobi Connection, "Pro
c. Int. Conf. on Computer Vision, pp. 215-219, 1993).
Therefore, by changing the size of the structuring element, a solution surface of a differential equation in a contour surface modeling method can be generated at high speed, although it is pseudo.

In this method, first, data points are connected to each other by expanding each data point into a sphere, and a region where a velocity vector of a point on a curved surface becomes zero (hereinafter, a “stop region”)
That. ) Is generated. Next, a single closed surface surrounding the entire data is set as an initial value, and the scraping operation on the internal region of the closed surface is repeated until the surface stops deforming. In order to maintain the smoothness of the curved surface during the contraction / division process, a sphere whose size changes depending on the curvature is used as a structuring element. Based on the obtained curved surface, an outer boundary surface of the stop region is extracted, and the stop region is corrected by scraping the stop region along this surface. Then, the curved surface contracts again under the stop area. By correcting the obtained model by energy minimization, a highly accurate object model can be obtained.

That is, according to the present invention, there is provided an apparatus for generating a three-dimensional object model which is an image approximated to a target object, wherein the distance measuring means measures distances from a plurality of viewpoints around the target object to the target object. A distance image storage means for storing a distance image composed of a plurality of distance data measured by the same distance measurement means; and a three-dimensional object model for calculating a three-dimensional object model based on the distance image stored by the same distance image storage means. The three-dimensional object model calculating means forms an interconnecting sphere of a predetermined expansion radius around each data point constituting the range image, and is formed by the interconnecting sphere. Forming a stop area with the area as a stop area, forming an initial closed surface outside the stop area, and forming a single initial closed surface surrounding the stop area; A closed surface contraction step of contracting the initial closed surface until each of the constituent points constituting the surface is located inside the stop region, and the closed surface obtained in the closed surface contraction step is a three-dimensional object model. The present invention provides a three-dimensional object model generation device characterized by the following.

Further, according to the present invention, a three-dimensional object model which is an image approximated to a target object is generated from a distance image including a plurality of distance data obtained by measuring distances from a plurality of viewpoints around the target object to the target object. Forming a mutually connected sphere having a predetermined expansion radius around each data point constituting the distance image, and forming a stop region formed by the mutually connected spheres as a stop region. A step, an initial closed surface forming step of forming a single initial closed surface surrounding the stop region outside the stop region, and an initial closed surface until each constituent point constituting the initial closed surface is located inside the stop region. A closed surface contraction step of contracting a curved surface, wherein a closed surface obtained in the closed surface contraction step is used as a three-dimensional object model. A.

Further, in the present invention, the step of contracting the closed surface includes forming interconnecting spheres having a predetermined scraping radius around each of the constituent points constituting the closed surface, and forming the connecting spheres within the interconnecting spheres. The closed surface is shrunk sequentially by making the side surface contracted, and the scraping radius is changed according to the curvature of the closed surface at each of the constituent points constituting the closed surface. After each of the constituent points constituting the initial closed surface is located inside the stop region, the contraction step is a re-stop region in which a sphere having a predetermined expansion radius is removed from the outer peripheral surface of the stop region. Shrinking the closed surface again until each component point is located inside the re-stop area, and further, the closed surface shrinking step includes:
After each of the constituent points constituting the initial closed surface is located inside the stop region, a region where a sphere having a predetermined expansion radius is removed from the outer peripheral surface of the stop region is defined as a re-stop region, and each of the constituent points of the contracted closed surface is It is also characterized in that the closed surface is contracted again until it is located inside the re-stop region, and then the contracted closed surface is slightly contracted based on the principle of energy minimization.

The present invention also provides a computer-readable recording medium on which a program for executing the above-described method for generating a three-dimensional object model is recorded.

[0028]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments of the present invention will be specifically described below.

In order to use the shape evolution of an evolutionary surface for object modeling, it is required to prevent self-intersection from a surface in the process of shape evolution and to maintain smoothness. Although the contour surface modeling method satisfies these requirements, as described above, it is necessary to perform repetition of distance transformation and addition of dimensions, which is not practical in terms of computational efficiency. Therefore, a method for more efficiently generating a shape evolution of a curved surface satisfying the above requirements is proposed.

First, the basic operation of three-dimensional morphology will be described.

The three-dimensional morphology is developed as a set theory in a three-dimensional space, and, as in the two-dimensional case, is based on displacement, erosion, opening, and closing. .

E ³ is a three-dimensional Euclidean space, A is a set in the three-dimensional space, B is a set used for morphological operation on the set A, that is, a three-dimensional structuring element, a is an element of A, and b is an element of B Element. The element here means a position vector of a point forming a set in a three-dimensional space. At this time, the displacement of the set A by the structuring element B is

[0033]

(Equation 2) Is defined as This represents an area that B can cover when the origin of the structuring element B is moved in the three-dimensional figure A.

The scraping of the set A by the structuring element B is as follows.

[0035]

(Equation 3) Is defined as This represents an area in which the figure B can be moved within a range in which the figure B is inserted into the three-dimensional figure A and does not protrude outside the A.

Further, the opening A by the structuring element B of the set A
_B is

[0037]

(Equation 4) And the closure A ^B of the set A with the structuring element ^B is

[0038]

(Equation 5) Is defined as

Next, iterative calculation of morphology and equations of motion will be described.

Have shown that the shape evolution of a surface caused by iterative computation of morphology obeys a differential equation (ABArehart, L. Vincent, and BBK).
imia, "Mathematical Morphology: The Hamilton-Jacobi
Connection, "Proc.Int.Conf.on Computer Vision, pp.2
15-219, 1993, BBKimia, ARTannenbaum, and SWZuc
ker, "Shapes, shocks, and deformations, I: The compon
ents of shape and the reaction-diffusion space, "In
t'l J. of Computer Vision, Vol. 15, pp. 189-224, 1995).

Assuming that the size of the set B is 1, a set nB (n is a positive integer) of a size n obtained by expanding B by n times is obtained by shifting and overlapping.

[0042]

(Equation 6) Is defined as If B is convex, then nB has the same shape as B and is n times larger in size. Therefore, the shifting and overlapping of the three-dimensional figure A by the structuring element nB is equivalent to performing the shifting and overlapping by B in order of n times as in the following equation. That is,

[0043]

(Equation 7) However, in the equation (14), the combination rule regarding the shift overlap operation is used. Similarly, scraping by the structuring element nB is equivalent to performing scraping by B n times in sequence as in the following equation. That is,

[0044]

(Equation 8) However, in equation (15), a distribution rule relating to a scraping operation is used. Equations (14) and (15) are staggered using a certain convex structuring element (scraping), but the shape is the same and the size is 1 / n
This means that it is equivalent to n times the structuring element is repeatedly shifted (scraped). At this time, if n → ∞, analysis using differentiation is possible.

The morphology operation for obtaining the three-dimensional figure S ′ from the three-dimensional figure S is performed by changing the boundary surface S of S to the boundary surface S ′ of S ′.
It can be regarded as a deformation operation of the curved surface converted into. Figure S at time t
, A point p on the surface ∂S is moved by Γ (p, t, λ) in the outward normal vector direction after λ time. I do. At this time, the growth rate F is obtained by using Γ (p, t, 0) = 0.

[0046]

(Equation 9) I can write Therefore, the deformation of the curved surface caused by the offset overlapping or scraping follows the partial differential equation represented by Expression (16). It is known that the amount of differential deformation β (p, t) is the maximum value of the projection of B onto the normal of の S at p (AB Arehart, L. Vincent, and BBKimia, “Mathema
tical Morphology: The Hamilton-Jacobi Connection, "
Proc.Int.Conf.on Computer Vision, pp. 215-219,199
3). For example, when using the structuring element of a sphere B with a radius of 1,
The equation of motion (16) is as follows: F = ± 1 (17) This is equivalent to the equation of motion of constant growth in which the curved surface grows outside (shifted over) or inside (scraping) at a constant normal speed.

The partial differential equation (16) is derived on the assumption that the curved surface is smooth on p. If p is undifferentiable, that is, if it is a singular point, equation (16) can be solved using the contour surface method.However, it is known that this solution is equivalent to the deformation of the curved surface by shifting and overlapping (scraping). (BBKimia, ARTannenbaum, and SW Zucker, "Sh
apes, shocks, and deformations, I: The components of
shape and the reaction-diffusion space, "Int'l J.of
Computer Vision, Vol.15, pp.189-224,1995).

The curved surface according to equation (17) tends to lose its smoothness during the deformation process (FIG. 2 (a)), and is not suitable for application to object modeling. Therefore, it is necessary to generate a shape evolution of a curved surface that maintains smoothness, such as a solution surface of an equation obtained by expanding Equation (6) into three dimensions. Therefore, a mechanism for changing the size of the structuring element depending on how the curved surface bends when shifting and overlapping (scraping) along the curved surface is introduced.

Here, a scraping operation mainly using a spherical structuring element is applied to object modeling. Therefore, the radius λ of the sphere used for scraping is changed as follows: λ = λ ₀ + ε′K (18) Here, λ _O and ε ′ are certain positive constants. K = K (p, t) is introduced instead of the three-dimensional curvature susceptible to noise.
Represents the degree of convexity in the, using the volume ratio of the nearby outer region and the inner region,

[0050]

(Equation 10) Is defined as Here, α = α (p, t) and β = β (p,
t) represents the number of cells inside the curved surface and the number of cells outside the curved surface, respectively, in a region surrounded by a spherical surface having a radius ρ centered on p (FIG. 3).

As a result, the curved surface contracts while maintaining smoothness (FIG. 2B). However, the shape evolution of this curved surface is not necessarily the same as that following the equation of the average curvature flow as in equation (6). Therefore, this is called a pseudo-curvature flow in the sense that it is similar to the above-described characteristic of the curvature flow. Note that shifting in the case of overlapping the formula (18) with λ = λ ₀ -
ε'K may be used.

When the evolution of the shape of a curved surface is used for object modeling using discrete data such as a distance image as an input, there is a problem how to stop the movement of the curved surface at a position on the surface of the object. On the other hand, a region where the velocity vector of a point on a curved surface is zero is defined, and the object model with the correct phase is roughly obtained using the shape evolution of the curved surface by pseudo-curvature flow, and the obtained model is used for energy minimization. There is a method of making correction based on this. A phase-adaptive object modeling method using both the shape evolution of the curved surface and the conventional deformable curved surface will be described below.

The present method comprises two stages: initial estimation of the model and correction of the estimated value. In the initial estimation of the model,
Generate a continuous area by shifting and overlapping a set of data points (Fig. 4 (a)) constituting a range image with a sphere of size δ
(FIG. 4 (b)). A curved surface point in this region has a velocity vector of zero, which is called a stop region.
A single closed surface surrounding the entire range image is given as an initial value, and the contraction / division of the surface due to the pseudo-curvature flow described above is repeated until all surface points reach the stop region (FIG.
(c)). Using the curved surface thus obtained, the boundary surface of the stop region is separated into a boundary surface belonging to the inside of the object (hereinafter, internal boundary surface) and a boundary surface belonging to the outside of the object (hereinafter, external boundary surface).
The stop area is corrected by using the sphere of size Δ again and scraping the stop area along the obtained external boundary surface (FIG. 4 (d)). The surface starts shrinking again and stops at the stop area, giving an initial estimate of the model (Figure 4).
(e)).

Next, processing for correcting the estimated value is performed. To obtain as high a description accuracy as possible, each vertex position of the object model obtained in the above initial estimation process is finely corrected based on the principle of energy minimization (FIG. 4 (f)).

As described above, according to the present invention, a three-dimensional object model which is an image approximated to a target object from a distance image including a plurality of distance data obtained by measuring distances from a plurality of viewpoints around the target object to the target object. A method for generating
Forming a mutually connected sphere having a predetermined expansion radius around each data point constituting the distance image, forming a stop region as a stop region formed by the interconnected spheres, and a stop region forming step. An initial closed surface forming step of forming a single initial closed surface outwardly surrounding the stop area, and a closed surface for contracting the initial closed surface until each of the constituent points constituting the initial closed surface is located inside the stop area A contraction step;
And the closed surface obtained in the closed surface contraction step is used as a three-dimensional object model. Then, the closed surface shrinking step forms a mutually connected sphere having a predetermined scraping radius around each of the constituent points constituting the closed surface, and a closed curved surface obtained by contracting the inner surface of the mutually connected sphere. By doing so, the closed curved surface is contracted sequentially. In addition, the scraping radius is changed according to the curvature of the closed surface at each of the constituent points constituting the closed surface. In addition, the closed surface contraction step includes, after the respective constituent points constituting the initial closed surface are located inside the stop region, a region where a sphere having a predetermined expansion radius is removed from the outer peripheral surface of the stop region is set as a re-stop region, The closed surface is contracted again until each of the constituent points of the closed surface is located inside the re-stop region, and the contracted closed surface is slightly contracted based on the principle of energy minimization. It was decided to make it.

The details of each processing step will be described below.

For the given distance image, an initial model estimation process is performed using the shape evolution of the curved surface according to the pseudo curvature flow described above. First, the distance image is shifted and superimposed by a structuring element δB of a sphere having a size δ, and each data point is set to a size δ.
Inflate into a sphere. If δ is sufficiently large compared to the distance between the data points, the expanded data points are interconnected. However, if δ is too large, a correct topological model cannot be obtained, so care must be taken in setting δ. In this way, the discrete data is converted into a region X sandwiched between disjoint outer boundary surfaces and inner boundary surfaces. This X is a stop area,
Expressing this as an equation,

[0058]

[Equation 11] Can be expressed as Here, L is a set of data points.

Next, as shown in FIG. 5, a single closed curved surface surrounding the entire range image is set as an initial curved surface. As previously mentioned,
This curved surface is contracted and split while repeatedly scraping the curved surface with the structuring element λB of a variable size until it reaches the stop region. Here, the radius λ = λ (p, t) of the sphere having the center at the point p on the curved surface at the time t is obtained by modifying the equation (18).

[0060]

(Equation 12) Defined by Here, λ ₀ and ε are positive constants having the same role as that used in equation (18). In addition, K = K (p, t) is obtained by the equation (1
This is the curvature of the curved surface defined in 9). Equation (2
1) generates the shape evolution of a curved surface that maintains smoothness as described above. On the other hand, when λ ₀ + εK at a point p on the curved surface becomes negative or when p is located on the stop region X, Indicates that p is not moved. As a result, the curved surface stops in a state of intersecting the stop region so as to cover the object or the object group from outside. Here, it is calculated whether each point on the boundary surface of the stop region is outside or inside the curved surface, and the external boundary surface of the stop region is extracted based on the calculation result. Here, it should be noted that it is difficult to directly separate the boundary surface of the stop region into the external boundary surface and the internal boundary surface. afterwards,
The structuring element δB having the same size as that used in the generation of the stop region is used again, and the stop region S is formed along the outer boundary surface.
, The outer boundary surface of X is corrected. The movement of the curved surface is started again, and the movement of the curved surface stops near the object surface.

It is difficult to accurately represent the shape of a sphere in a discrete space, and the quantization error increases as the size of the sphere decreases. Therefore, fine irregularities often remain on the surface of the model obtained at the stage of initial estimation. Therefore, in order to obtain a description that is more faithful to the original shape, this surface is corrected based on the principle of energy minimization (H. Delingette, M. Hebert, a
nd K. Ikeuchi, "Shape Representation and Image Segme
ntation Using Deformable Surfaces, "Proc.IEEE Con
f. CVPR, pp. 467-472, 1991, T. McInerney and D. Terzopo
ulos, "A Finite Element Model for 3D Shape Reconst
ruction and Nonrigid Motion Tracking, "Proc.Int.Con
f.on Computer Vision, pp.518-523,1993, K.Hara, H.Zh
a, and T.Hasegawa, "Regularization Bas ed 3-D Objec
t Modeling from Multiple Range Images, "Proc.Int.C
onf.on Pattern Recognition, Vol.2, pp.964-968,199
8).

First, a quadrangular mesh enclosing the curved surface obtained in the previous stage is extracted. Here, when the mesh is too fine, the division width of the entire three-dimensional space is enlarged to generate a coarser mesh. Here, model correction processing is performed using a deformable curved surface with this quadrilateral mesh as an initial value. Since a model having a correct phase has been obtained in the previous stage, the deformable surface used here does not need phase adaptability.

When the position vector of a certain vertex of the mesh is represented by V, the energy function to be minimized is defined as E _point (V) = E _int (V) + E _est (V) (22) Here, E _int (V) is called an internal energy and corresponds to a regularization term. Here, internal energy in which a spring is set between vertices connected to each other is used. E _est (V) is called external energy, and requires proximity between the mesh and the distance data. Here, the external energy with a spring set between the vertex and the data point closest to that position is used (T. McInerney a
nd D. Terzopoulos, "A Finite Element Model for 3D S
hape Reconstruction and Nonrigid Motion Tracking, "
Proc.Int.Conf.on Computer Vision, pp.518-523, 199
3). Then, to minimize Equation (22) quickly, the Greedy algorithm (DJ Williams and M. Shah, A Fast Algorithm
m for Active Contours, Torsion, ICCV, pp.592-595, 1990)
Is used. In the Greedy algorithm, for each vertex,
The next destination is determined based on the energy value when the point has moved to the vicinity of 26 in the three-dimensional space.

The following shows the results of experiments performed on synthetic distance data and two types of actual distance data.

(Experiment 1 Experiment Using Synthetic Distance Data)
FIG. 6A shows distance data artificially created based on CAD data of a hexagon nut. The results of a modeling experiment using this data as input are shown. This experiment was performed to verify whether this method can be applied to sparse data and whether it is more effective than the phase-adaptive modeling method using the contour surface method.

The stop area initialized from this data
Fig. 6 (b) shows the results obtained by contracting and splitting the initial closed surface for (δ = 8), and Fig. 6 (c) shows the results obtained by modifying the stop region based on these results. Shown respectively. Thereafter, the vertices are slightly moved in the estimation value correction process, and the finally obtained object model is shown in FIG. 6 (d). Columns 3 to 5 from the right of Table 1 show the number of patches (N) at each stage and the calculation time (Tsec.) (Pentium II 300 MHz) required up to that point. However, since the shape of the object is relatively simple, when extracting a mesh from the surface of the initial estimation result shown in FIG. Has been reduced to a modest degree.

For comparison with the conventional method, narrow-b
and algorithm (DLChopp, Computig Minimal Surf
aces via Level Set Curvature Flow, J.of Computati
onalPhysics, Vol. 106, pp. 77-91, 1993), was used for the initial estimation processing stage. FIG. 7 shows the modeling result. Fig. 7 (a) shows the initial stop area
This is the result of contracting and dividing a single initial closed surface by the contour surface method for (δ = 8), and corresponds to FIG. 6 (b). Fig. 7
(b) shows the result obtained by the contour surface modeling method for the corrected stop area, and corresponds to FIG. 6 (c). From these, it can be said that almost the same results are obtained in both methods. In columns 1 and 2 from the right in Table 1,
(a) and (b) show the number of patches and the calculation time. Figure 6 (b) and figure in Table 1
When comparing the calculation time T of FIG. 7 (a) and the calculation time T of FIG. 6 (c) and FIG. 7 (b), the present method is more effective than the contour surface modeling method.
It can be seen that the execution was nearly 10 times faster.

FIG. 8 is a sectional view showing a process of deforming a curved surface at the stage of initial estimation. FIG. 8A shows an initial curved surface. FIGS. 8B and 8C show an intermediate result and a convergence result in the process of contraction and division of the surface with respect to the initial stop region, respectively.

FIG. 9 is a sectional view showing a process of deforming a curved surface when a stop area is not set. As a result, in the shape evolution of the curved surface adopted in the present method, it is confirmed that the curved surface contracted while being smoothed by the growth rate depending on the curvature.

(Experiment 2 Modeling of Object with Hole and Border) Next, the result of modeling the object (tea cup) with hole and rim shown in FIG. 10A will be introduced. The objects targeted in this experiment are typical of those that are difficult to model. For example, if a method of generating a mesh by connecting data points in order is used, there is a possibility that data points existing outside and inside the thin edge are erroneously connected. Deformable surfaces may be able to address this problem,
This time, there is a problem that it is difficult to restore the hole area at the handle.

In order to acquire the distance data of the entire surface of the object, the object was suspended by a wire and floated, and measurement was performed from multiple viewpoints using a range finder. FIG. 10 (b) shows the distance data thus obtained. FIGS. 11 (a) and 11 (b) show the initial estimation result and the final result obtained using this data as an input.

(Experiment 3 Simultaneous Modeling of a Plurality of Objects) Finally, the results of modeling of a plurality of object scenes (two moai images and one haniwa) shown in FIG. 12A will be introduced.

To obtain input data, multi-viewpoint measurement was performed using a range finder. However, since this data includes extra data points of the table on which the object is placed, the data points were excluded by threshold processing in the height direction (z-axis direction) of the object. The input data thus obtained is shown in FIG. Next, it is necessary to generate a continuous stop region without a gap, but the bottom surface of the object is an unmeasured region, and there is a hole larger than other regions. Therefore, for data points at a height close to the bottom surface, a set surrounded by an ellipsoid that is flat with respect to the z-axis,

[0074]

(Equation 13) Is used as a structuring element, and data points in other areas are shifted and superimposed using a sphere δB of size δ as a structuring element to generate a stop area including the entire surface of the object inside. did. However, δ = 8 and a = b = 3.

FIG. 13 shows the results obtained at each stage of the method. First, FIGS. 13A to 13D show a process in which a single closed surface contracts and splits under the above-mentioned stop region. From FIG. 13 (a) to FIG. 13 (c), a single closed surface contracts.
It is divided into three closed surfaces in (d). At the same time, in FIG. 13A, it can be seen that the hole area of the arm of “Haniwa” is restored. In FIG. 13A to FIG. 13D, the lower part of the object slightly protrudes because of the flat ellipse defined by the equation (23). Correct the stop area based on the result shown in FIG.
Figure 13 (e) shows the initial estimation result obtained by contracting the surface again.
Shown in The final result is shown in FIG. The number of vertices in Fig. 13 (f) is
21144, the calculation time is 2276 sec. However, the same computer as in Experiment 1 was used.

[0076]

A phase-adaptive object modeling method using a shape evolution of a curved surface based on three-dimensional morphology has been described. The proposed method was applied to a synthetic range image of an object with a hole area and an actual range image of a scene including multiple objects, and its effectiveness was confirmed. This method does not require an enormous amount of calculations or additional dimensions for finding the shortest distance unlike the contour surface modeling method, and is an algorithm with a small calculation load.

Further, an object modeling method (RTWhitaker,
A Level-Set Approach to 3D Reconstruction from Ran
ge Data, Int l J. of Computer Vision, Vol. 29, pp. 203
-231,1998), but it is limited to the case where the discrete data is dense enough to have no gap and the positional relationship between the sensor's viewpoint direction and the object is known. This method enables modeling of sparse distance data.
Further, prior knowledge regarding the measurement direction of the distance image is not required, and for example, it is possible to correspond to distance data obtained by a contact sensor.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram showing a shape evolution of a curvature in a contour surface method.

FIG. 2 is an explanatory diagram showing the evolution of the shape of a curved surface using a variable-size structuring element.

FIG. 3 is an explanatory view showing how a curved surface is bent.

FIG. 4 is an explanatory diagram showing an outline of the present technique.

FIG. 5 is an explanatory view showing contraction and division of a curved surface.

FIG. 6 shows an experimental result for a perforated object.

FIG. 7 is an experimental result by a contour surface modeling method.

FIG. 8 is a sectional view showing a process of a phase change.

FIG. 9 is a cross-sectional view showing a process of curved surface deformation while maintaining smoothness.

FIG. 10 is a perspective view showing an object having a hole and a rim.

FIG. 11 is an experimental result for an object having a hole and a rim.

FIG. 12 is a perspective view showing a plurality of objects.

FIG. 13 shows experimental results for a plurality of objects.

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2F065 AA06 AA53 BB05 FF04 NN20 QQ13 QQ17 QQ24 QQ32 QQ45 2F112 AC06 BA05 CA08 FA03 FA45 FA50 GA01 5B080 AA08 BA00

Claims (11)

[Claims] 1. An apparatus for generating a three-dimensional object model, which is an image approximated to a target object, comprising: distance measuring means for measuring distances from a plurality of viewpoints around the target object to the target object; A distance image storage unit that stores a distance image composed of a plurality of distance data measured by the measurement unit; and a three-dimensional object model calculation unit that calculates a three-dimensional object model based on the distance image stored by the distance image storage unit. The three-dimensional object model calculating means forms an interconnecting sphere having a predetermined expansion radius around each data point constituting the range image, and forms an area formed by the interconnecting spheres as a stop area. Forming an initial closed surface outside the stop region and forming a single initial closed surface surrounding the stop region; and forming each of the initial closed surfaces A closed surface shrinking step of shrinking the initial closed surface until the point is located inside the stop area, wherein the closed surface obtained in the closed surface shrinking step is a three-dimensional object model. Original object model generation device. 2. The closed surface shrinking step includes forming interconnecting spheres having a predetermined scraping radius around each of the constituent points forming the closed surface, and shrinking the inner surface of the interconnecting spheres. The three-dimensional object model generating apparatus according to claim 1, wherein the closed surface is sequentially contracted by forming the closed surface. 3. The three-dimensional object model generating apparatus according to claim 2, wherein the scraping radius is changed according to the curvature of the closed surface at each of the constituent points constituting the closed surface. 4. The closed surface shrinking step is a step in which, after each of the constituent points constituting the initial closed surface is located inside the stop region, a region in which a sphere having a predetermined expansion radius is removed from the outer peripheral surface of the stop region is a re-stop region. The three-dimensional object model generating apparatus according to any one of claims 1 to 3, wherein the closed surface is contracted again until each of the constituent points of the contracted closed surface is located inside the re-stop region. . 5. The closed surface contraction step includes, after each of the constituent points constituting the initial closed surface is located inside the stop region, re-stop the region obtained by removing a sphere having a predetermined expansion radius from the outer peripheral surface of the stop region. The contracted closed surface is contracted again until each component point of the contracted closed surface is located inside the re-stop region, and then the contracted closed surface is slightly contracted based on the energy minimization principle. The three-dimensional object model generation device according to any one of claims 1 to 4. 6. A method for generating a three-dimensional object model that is an image approximated to a target object from a distance image including a plurality of distance data obtained by measuring distances from a plurality of viewpoints around the target object to the target object. Forming a mutually connected sphere having a predetermined expansion radius around each data point constituting the distance image, and forming a stop region as a stop region formed by the mutually connected spheres; An initial closed surface forming step of forming a single initial closed surface surrounding the stop region outside the region, and shrinking the initial closed surface until each of the constituent points constituting the initial closed surface is located inside the stop region A closed surface shrinking step, wherein the closed surface obtained in the closed surface shrinking step is used as a three-dimensional object model. 7. The closed surface shrinking step includes forming interconnecting spheres having a predetermined scraping radius around each of the constituent points constituting the closed surface, and shrinking the inner surface of the interconnecting spheres. 7. The method for generating a three-dimensional object model according to claim 6, wherein the closed surface is sequentially contracted by forming the closed surface. 8. The three-dimensional object model generating method according to claim 2, wherein the scraping radius is changed according to the curvature of the closed surface at each of the constituent points constituting the closed surface. 9. The closed surface contraction step includes, after each of the constituent points forming the initial closed surface is located inside the stop region, re-stopping the region obtained by removing a sphere having a predetermined expansion radius from the outer peripheral surface of the stop region. The method for generating a three-dimensional object model according to any one of claims 6 to 8, wherein the closed surface is contracted again until each constituent point of the contracted closed surface is located inside the re-stop region. . 10. The closed surface contraction step includes, after each of the constituent points constituting the initial closed surface is located inside the stop region, re-stop the region where a sphere having a predetermined expansion radius is removed from the outer peripheral surface of the stop region. The contracted closed surface is contracted again until each component point of the contracted closed surface is located inside the re-stop region, and then the contracted closed surface is slightly contracted based on the energy minimization principle. Claim 6-
A method for generating a three-dimensional object model according to claim 9. 11. A computer-readable recording medium on which a program for executing the method for generating a three-dimensional object model according to claim 6 is recorded.