JP5068732B2 - 3D shape generator - Google Patents

Description

  The present invention relates to a three-dimensional shape generation apparatus that generates a three-dimensional shape model from a multi-viewpoint image of a subject.

  As a method for generating a three-dimensional shape from a multi-viewpoint image of a subject, a method is known in which an approximate shape obtained by a visual volume intersection method is used as an initial shape and the obtained initial shape is corrected. As a correction method, a method of searching for corresponding points using a plurality of pairs of stereo images (for example, see Non-Patent Document 1), or by projecting initial surface vertices onto a plurality of camera images, and depending on the degree of correspondence A method for correcting the position (for example, see Non-Patent Document 2) is known. As another correction method, a method of correcting an initial shape using a maximum flow / minimum cut algorithm has attracted attention. In the maximum flow / minimum cut algorithm, the maximum flow is calculated by obtaining the minimum value of the communication capacity between two distant nodes in the network. When this algorithm is applied to image processing, it can be applied to region segmentation processing, for example, by associating nodes with pixels and replacing edges connecting adjacent nodes with correlation energy between pixels. This algorithm is characterized in that global energy can be minimized in a graph represented by nodes and edges, and is also called a graph cut.

When the maximum flow / minimum cut algorithm is applied to an image reproduction technique for generating a three-dimensional shape from a multi-viewpoint image, the vertex of the subject specified by the three-dimensional coordinates is made to correspond to the node, and the connection portion between adjacent vertices Correspond to the edge. Each node is given a cross-correlation value between camera images, and each edge is given an average value of cross-correlation values of both end nodes to form a graph. The shape of the subject is cut out by applying the maximum flow / minimum cut algorithm to the obtained graph. However, since this algorithm minimizes the global energy, the cut shape tends to be small and degenerate. In order to eliminate the problem of degeneracy, some improvement is necessary.For example, it is possible to prevent unnecessary cutting by providing a large energy node inside the initial shape, A method has been proposed in which vertices capable of obtaining highly reliable shape information such as feature points are used as constraint conditions (see, for example, Non-Patent Documents 3 and 4).
"Reconstruction method of 3D shape constrained by local shape features and its real-time video display", Eiji Journal, Vol.61, No.4, pp.471-481 (Apr.2007) "High-precision 3D shape restoration from multi-viewpoint images using elastic mesh model", Information Processing, Vol.43, No.SIG11 (CVIM5), pp.53-63, (2002) "Surface Capture for Performance-Bases Animation", IEEE Computer Graphics and Applications, Vol.27, No.3, pp.21-31, (2007) "Multi-view stereo volumetric graph-cuts", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2: 391-398, (2005)

An object of the present invention is to generate a three-dimensional shape that can stably generate a three-dimensional shape model without greatly changing the initial shape obtained by the visual volume intersection method when generating a three-dimensional shape of a subject from a plurality of camera images. To implement the device. In particular, a method for generating a three-dimensional shape model is realized by applying a maximum flow / minimum cut algorithm. In order to achieve this purpose, the following problems to be solved are assumed.
A. When the maximum flow / minimum cut algorithm is applied to image data formed by the visual volume intersection method, for example, when the subject is a person wearing a kimono, thin portions such as kimono sleeves tend to be cut off. As a countermeasure, it is assumed that a solid portion that is hard to be cut is provided in the initial shape obtained by the visual volume intersection method. However, if a solid portion that becomes a hard core is provided, there is a problem that an edge to be cut is not cut, and it is necessary to develop a setting means for the solid portion that becomes a core.
B. In principle, the visual volume intersection method cannot accurately reproduce the concave shape of the subject. On the other hand, for a convex shape that changes smoothly, a generally accurate shape can be reproduced. Therefore, when the maximum flow / minimum cut algorithm is applied to the initial shape obtained by the visual volume intersection method, it is necessary to discriminate the concave portion and the convex portion of the subject and set a constraint condition according to each part. Therefore, it is necessary to develop a method for setting the concave / convex shape discrimination condition and the constraint condition.
C. When applying the maximum flow / minimum cut algorithm, the cross-correlation value between cameras for each vertex is given as energy to the edge via the node. In this case, a calculation means capable of adaptively calculating the cross-correlation value given to the edge is required. Further, when the cross-correlation value of one vertex is obtained from a combination of cameras, a means for appropriately integrating the cross-correlation values obtained for each camera combination is also necessary.

  An object of the present invention is to solve the above-described problems in a three-dimensional shape generation apparatus that generates a three-dimensional shape of a subject by applying a maximum flow / minimum cut algorithm to a three-dimensional shape model obtained by a visual volume intersection method. Another object is to realize a three-dimensional shape generation apparatus.

A three-dimensional shape generation apparatus according to the present invention is a three-dimensional shape generation apparatus that generates a three-dimensional shape model of a subject from a multi-viewpoint camera image obtained by imaging the subject from a plurality of angular directions.
Silhouette image generation means for generating a silhouette image from a multi-viewpoint camera image;
A solid model generation unit that applies a visual volume intersection method to the generated silhouette image and generates a solid model that is an approximated three-dimensional shape model of the subject;
Silhouette constraint information generating means for generating a silhouette constraint value based on the silhouette image and the solid model;
Using the solid model, vertex visible information generating means for specifying a camera that visually images the surface of the subject for each vertex;
A cross-correlation value calculating means for calculating a cross-correlation value between camera images captured by the specified camera;
Means for weighting the calculated cross-correlation value using the generated silhouette constraint value, and outputting the weighted cross-correlation value;
For the solid model, a graph composed of nodes corresponding to surface vertices and edges connecting adjacent nodes is formed, and a maximum flow / minimum cut algorithm is applied using the weighted cross-correlation values. And a maximum flow / minimum cut calculating means for generating a three-dimensional shape model of the subject.

  In the present invention, based on a solid model obtained by applying the visual volume intersection method to a silhouette image, various solid models are generated from the solid model, and the maximum flow / minimum cut algorithm is applied to the generated solid model. Therefore, even if the maximum flow / minimum cut algorithm is applied, the problem that the thin part of the subject is cut off is solved, and a three-dimensional shape generation apparatus with excellent reproducibility is realized. Moreover, the initial shape obtained by applying the visual volume intersection method to the silhouette image is obtained by defining a constraint condition so that the shape of the convex portion is constrained (held) and the shape of the concave portion is strongly corrected. Since the cross-correlation value is weighted using the above constraint conditions, the three-dimensional shape of the subject can be reproduced without greatly damaging the initial shape obtained by applying the visual volume intersection method.

In a preferred embodiment of the three-dimensional shape generation apparatus according to the present invention, the solid model generation means includes a first solid model formed by applying a visual volume intersection method to a silhouette image, and an erosion process on the first solid model. Obtained by performing erosion processing on the first solid model, the third solid model obtained by performing dilation processing on the first solid model, and the first solid model. A fourth solid model to be generated,
The maximum flow / minimum cut calculation means applies a maximum flow / minimum cut algorithm to a cut region located between the third solid model and the fourth solid model, and a three-dimensional shape that minimizes the total energy. It is characterized by generating a model.

  In the present invention, various solid models are generated based on a solid model obtained by applying the visual volume intersection method to a silhouette image, and cross-correlation values weighted by silhouette constraint information are generated for the generated solid model. Since the maximum flow / minimum cut algorithm is applied, the thin part of the subject is reproduced well, and the three-dimensional shape of the subject can be obtained without greatly breaking the three-dimensional shape obtained by applying the visual volume intersection method. Can be reproduced well.

In this embodiment, the vertex space is set in the three-dimensional world coordinate system, and the correspondence degree calculated from the camera image and the constraint value generated from the silhouette of the subject are given to each vertex, thereby defining the correspondence degree and the constraint value. Set energy for each vertex. The average energy of adjacent vertices is defined as the energy of the edges connecting the vertices, and the vertex group having the minimum sum is obtained using the maximum flow / minimum cut algorithm. By this processing, a three-dimensional shape model having the obtained vertex group as the surface vertex is generated. The processing flow according to the present invention is roughly divided into the following processes a to f. Hereinafter, the processing steps will be described in the order of processing a to e. Note that camera parameters such as the posture, position, and focal length of each camera are known.
a. Generation of various solid models b. Generation of silhouette constraint information c. Generation of visible camera information for each vertex d. Generation of correspondence information for each vertex e. Weighting process for cross-correlation value f. 3D shape generation with maximum flow / minimum cut algorithm

  FIG. 1 is a diagram showing an example of a three-dimensional shape generation apparatus according to the present invention. N cameras are arranged so as to surround the subject, the subject is imaged from various angles by the cameras 1a to 1n, and the captured camera images are supplied to the three-dimensional shape generation device. In this example, the camera image is subjected to software processing by a computer to reproduce the three-dimensional shape of the subject. In addition, prior to execution of this example, camera parameters such as the position, posture, and focal length of each camera are measured and recorded.

  The n camera images are supplied to the silhouette image generation unit 2 to form n silhouette images. The silhouette image generation unit 2 removes background regions other than the subject from individual camera images, and generates a silhouette image obtained by cutting out only the subject region. As a method for generating a silhouette image, a difference between a camera image in which a subject is present and an image of only the background can be obtained, and can be generated by a background difference method that extracts only the subject, or can be generated by a chroma key method. .

  The generated silhouette image is supplied to a solid model generation unit 3 that generates a solid model that is a variety of three-dimensional shape models of the subject. First, the silhouette image is supplied to the visual volume intersection method unit 4. The visual volume intersection method unit 4 generates an approximate three-dimensional shape of the subject by the visual volume intersection method using the silhouette image and the camera parameters. An approximate three-dimensional shape obtained by the visual volume intersection method will be referred to as a solid model 1.

  Further, the solid model generation unit 3 includes an erosion processing unit 5, a dilation processing unit 6, and an adaptive erosion processing unit 7. The erosion processing unit 5 applies the erosion processing that is one of the morphological methods to the solid model 1 that is a three-dimensional shape model generated by the visual volume intersection method, and reduces the solid model 1 by one vertex. Three-dimensional shape data (solid model 2) is generated. Dilation processing unit 6 applies dilation processing, which is one of the morphological methods, to solid model 1 to generate three-dimensional shape data (solid model 3) expanded from solid model 1 by one vertex. To do.

The adaptive erosion processing unit 7 performs adaptive erosion processing on the solid model 1 and generates three-dimensional shape data (solid model 4) reduced according to the thickness of each part of the solid model 1. In this case, if the erosion process is simply performed, there may be a problem that a thin portion in the solid model 1 is missing. For example, when the erosion process is performed with the number of reduced vertices set to N = 2, a portion where the thickness of the solid model 1 is equal to or less than (2N + 1) vertices is lost. Therefore, in the present invention, the distance to the surface vertex of each vertex of the solid model 1 is measured, and the maximum value of the distance in the region where the vertexes are located inward by N from the vertex to be reduced is obtained. The value is reduced by erosion processing by the number divided by M (integer). An example of this adaptive erosion process is shown in FIG. In FIG. 2, N = 2 and M = 2 are shown in two dimensions. FIG. 2A shows the solid model 1 two-dimensionally. As shown in FIG. 2B, the distance from each vertex to the surface vertex is measured, and this is set as a distance value A. Next, as shown in FIG. 9C, the maximum value of the distance A is detected in the range of (N × M + 1) × (N × M + 1) around the target vertex. Next, as shown in FIG. 4D, a distance value of 1 / M (rounded off) of the maximum value is calculated. This is distance B. The size of the structural element is set to (2 × distance B + 1) for each vertex, and the erosion process is executed. The erosion process is performed using a 3 × 3 structural element in the region of distance B = 1 and using a 5 × 5 structural element in the region of distance B = 2. As shown in FIG. 2E, by executing the adaptive erosion process, a thin portion remains, and the vertex indicated by a broken line and a dot point becomes the solid model 4. The three-dimensional shape of the subject is reproduced using the four solid models 1 to 4 generated in this way.
The solid models 1 to 4 are binary data that takes a value of “1” or “0” for each vertex. In the following description, it is assumed that the vertex assigned “1” represents the subject shape, and the vertex assigned “0” represents the outside of the subject.

  Next, the silhouette constraint information generation process will be described. Here, the “silhouette constraint value” has a significance as a variable for constraining (holding) the shape of the convex portion corresponding to the true object shape faithfully with respect to the three-dimensional shape obtained by the visual volume intersection method. The convex portion is processed so that the constraint value is small and the constraint value is large for the concave portion. The silhouette image obtained from each camera image and the solid models 1 and 3 are supplied to the silhouette constraint information generation unit 10. The silhouette constraint information generation unit 10 includes a silhouette number generation unit 11 that counts the number of silhouettes, and a silhouette constraint value conversion unit 12 that generates a silhouette constraint value based on the obtained number of silhouettes.

  The silhouette image obtained from each camera image and the solid models 1 and 3 are input to the silhouette number generation unit 11. The silhouette number generation unit 11 projects the surface vertices of the solid model 3 on the silhouette image of each camera, and determines whether or not there is a silhouette edge near the projection point using the following method. In this specification, the silhouette edge means a boundary point between the silhouette of the subject and the background.

  First, if the projection point is a background pixel and there is at least one pixel constituting the silhouette image around the projection point, the vertex is determined to be the vertex at the silhouette end. For example, in the example shown in FIG. 3, the vertex A is determined as the silhouette end, and the vertex B does not correspond to the silhouette.

  The silhouette edge detection process described above is performed for all cameras (for all silhouette images), the number of cameras determined to be silhouette edges is counted, and the counted number of cameras is defined as the number of silhouettes. Then, the counted number of silhouettes is given to the surface vertex of the solid model 3 as silhouette constraint information. The generated silhouette constraint information is supplied to a weighting calculation unit 32 of the cross-correlation calculation 30 described later, and is used as a weighting factor for the cross-correlation value.

  FIG. 4 is a diagram illustrating a generation state of the solid model 1 generated by the visual volume intersection method. As shown in FIG. 4, it can be understood that the concave portion of the subject has a small number of silhouettes and the convex portion has a large number of silhouettes. In the concave shape, the number of silhouettes is small and the crossing angle of silhouettes is also small. On the other hand, in the convex shape part where the surface shape changes smoothly, the number of silhouettes is large and the intersection angle tends to be large. Therefore, the silhouette constraint value is determined in consideration of such characteristics. FIG. 5 is a diagram showing the true subject shape and the positional relationship between the solid model 1 and the solid model 3.

  As a method for correcting the silhouette constraint value, when the number of silhouettes = 2, the number of silhouettes can be set to 0 when the intersection angle between the vectors from the two cameras toward the vertex is, for example, 120 ° or less. . This is because the silhouette information is given only to the convex portion where the shape of the subject surface changes smoothly. The vertex C in FIGS. 4 and 5 has the number of silhouettes = 2, but if the intersection angle is about 120 degrees or less, the number of silhouettes is zero.

The number of silhouettes generated by the silhouette number generation unit 11 is supplied to the silhouette constraint value conversion unit 12. The number of silhouettes is given for each vertex. In order to adjust the degree of constraint, the silhouette constraint value conversion unit converts the silhouette number into the following silhouette constraint values.
T is a constant, and the number of silhouettes is controlled as follows.
IF (if silhouette> T) A = T
IF (if the number of silhouettes <T) A = number of silhouettes
That is, when the number of silhouettes exceeds the threshold T, the number of silhouettes reaches a peak at T. Furthermore, the silhouette constraint value is defined by the following formula and normalized.
Silhouette constraint value = 1-A / T

  Next, a process for generating visible camera information for identifying a camera that visually captures the surface of a subject will be described. In the cross correlation value calculation process described later, a camera image output from a camera that directly captures each vertex of a subject, that is, visually, is necessary. Therefore, in the visible camera information generation unit 20, a camera that visually captures each vertex using the solid models 1 to 3 is specified by the following method.

  In this example, the vertex where the solid model 1 is “1” and the solid model 2 is “0” is defined as the surface vertex of the solid model 1, and the camera that visually images the surface vertex is specified. A unit vector from one surface vertex to the camera is obtained, the position of the surface vertex is moved by the vertex interval along the vector, and if another vertex exists at the moved position, the camera directly images the vertex. It is determined that it is not (invisible). On the other hand, if the movement is repeated to the end of the three-dimensional vertex space and another vertex does not exist, it is determined that the camera is in a state of visually capturing the vertex (visible). This process is performed for all the cameras, and visible camera information that identifies the camera that visually captures the surface vertex is generated. By this processing, a camera that visually shoots for each surface vertex is specified.

  Next, the reference camera is specified. A single vector and an average direction vector from the surface vertex to the specified camera are obtained, and the specified camera numbers are rearranged in ascending order of the angle between the average direction vector and each camera unit vector to generate visible camera information. . In the visible camera information, the first camera is referred to as a reference camera, and the other cameras are referred to as reference cameras. The process for specifying the base camera and the reference camera will be described with reference to FIG. In FIG. 6, it is assumed that the subject is imaged by ten cameras C1 to C10. First, the surface vertex A is moved in the direction of the cameras C1 to C10, and the camera that is visibly imaged is specified. In the case shown in FIG. 6, the cameras that directly image the surface vertex A are C6, C7, C8, and C9. Next, the inner product of the vector from the vertex A toward each camera and the average direction vector is calculated, and the camera having the maximum inner product value is set as the reference camera. Therefore, in the example shown in FIG. 6, the camera C7 is a reference camera, and the remaining cameras C6, C8, and C9 are reference cameras. On the other hand, the vector toward the camera C9 makes an angle exceeding 90 ° with respect to the average direction vector, and is inappropriate for calculating the correspondence (cross-correlation). Therefore, such a camera is excluded from the calculation target of the cross correlation value by the determination based on the angle difference. Therefore, finally, the camera C7 becomes a reference camera, and the cameras C6 and C8 become reference cameras. Then, the cameras C6 to C8 are subjected to cross correlation value calculation, and are supplied to the cross correlation calculation unit 30 as visible camera information.

  Next, the cross correlation calculation process will be described. The visible camera information, the solid models 3 and 4, and the camera image are supplied to the cross correlation calculation unit 30. The cross-correlation calculation unit 30 includes a cross-correlation value calculation unit 31 that calculates a cross-correlation value between camera images, and a weighting calculation unit 32 that weights the calculated cross-correlation value with silhouette constraint information. The cross-correlation calculation unit 31 projects the surface vertex and the internal vertex of the solid model 3 onto the camera image of the camera specified by the visible camera information generation unit 20, and calculates the cross-correlation value between the camera images in the vicinity of the projection point. This is used as the degree of correspondence of each vertex.

  FIG. 7 shows the configuration of a block for calculating cross-correlation values. The calculation of the image correlation value is performed in units of blocks, and as an example, the blocks are configured by the following procedure. First, the world coordinates of vertices for which cross-correlation values are to be calculated are converted into camera coordinates of the reference camera, and W virtual vertices in the horizontal direction on the drawing and W virtual vertices in the vertical direction in the vicinity of the vertex. A plane block is provided. As shown in FIG. 7, the virtual vertex block is a plane orthogonal to the direction vector of the reference camera. The virtual vertex block is converted into world coordinates and projected onto the base camera C7 and the reference cameras C6 and C8. As a result of this projection, W × W projection point groups are formed on each camera image, and this is used as a block to calculate the cross-correlation value.

  When a plurality of reference cameras are available, the cross-correlation value between each reference camera and the reference camera or the average value of the cross-correlation values between two reference cameras can be taken. When calculating the average value, methods such as simple average and weighted average can be used. For example, averaging may be performed by giving a large weighting factor between adjacent cameras and giving a small weighting factor between cameras far away.

  The cross correlation value is calculated using the luminance signal of the camera image. On the other hand, along with the calculation of the cross-correlation value using the luminance signal, it is also possible to calculate the cross-correlation value for each of the RGB color components and perform weighted addition using the cross-correlation value obtained from the color components.

  The above-described processing is performed for vertices in which the solid model 3 is “= 1” and the solid model 4 is “= 0”, that is, vertices in a region sandwiched between the surface of the solid model 3 and the surface of the solid model 4. Calculate the cross-correlation value for each vertex.

  Next, cross correlation value weighting processing will be described. The cross correlation value calculated by the cross correlation value calculation unit 31 is supplied to the weighting calculation unit 32. The weight calculation unit 32 is also supplied with the silhouette constraint information generated by the silhouette constraint information generation unit 10. The weighting calculation unit 32 weights the input cross-correlation value with the silhouette constraint information, and supplies the weighted cross-correlation value to the maximum flow / minimum cut calculation unit 40. By weighting using the silhouette constraint information, the shape of the convex portion of the solid model 1 is constrained, and the shape of the unconstrained portion is corrected.

  Next, a process for generating a three-dimensional shape model by the maximum flow / minimum cut calculation will be described. The maximum flow / minimum cut calculation unit 40 is supplied with the solid models 3 and 4 together with the weighted cross-correlation values. The maximum flow / minimum cut calculation unit 40 generates and generates a graph composed of nodes corresponding to vertices and edges connecting adjacent nodes, using the supplied solid models 3 and 4 and cross-correlation values. A maximum flow / minimum cut algorithm is applied to the generated graph to generate a three-dimensional shape model. FIG. 8 shows an example of a graph set in the three-dimensional vertex space. An area sandwiched between the solid model 4 and the solid model 3 is a cut area, and the solid model 3 = is “1” and the solid model 4 is a vertex group defined by “= 0”.

  As shown in FIG. 8, a large weight is given to the inner edge of the solid model 4 and the outer edge of the solid model 3, and the S node and the T node are connected to the edge with a large weight, respectively. Edge weights determined from cross-correlation values weighted with silhouette constraint values are given to the cut regions in FIG. The maximum flow / minimum cut algorithm is applied to the graph generated in this way. This algorithm is an algorithm for calculating a maximum flow (maximum information transmission amount) between nodes of a communication network. In the present embodiment, the maximum flow is made to flow from the S node to the T node by regarding the edge weight as the communication capacity between the vertices. At that time, the saturation of the flow is examined for each edge. The graph is divided into two with all saturated edges as boundary surfaces. Of the two regions, a node (vertex) group connected to the T node is cut out as a three-dimensional shape. Although there are many boundary surfaces that divide the graph into two, the sum of the communication capacities at the edges of the boundary surface is minimized, and the boundary surface is called a minimum cut.

It is a diagram which shows an example of the three-dimensional shape production | generation apparatus by this invention. It is a figure which shows an example of the method of producing | generating the solid 4. FIG. It is a figure which shows the determination process of a silhouette end. It is a figure which shows the production | generation condition of the solid model 1 produced | generated by the visual volume intersection method. It is a figure which shows the positional relationship of a true to-be-photographed object shape, the solid model 1, and the solid model 3. FIG. It is a figure for demonstrating visibility determination processing. It is a figure for demonstrating the calculation method of a cross correlation value. It is the figure which shows three-dimensional vertex space as a cross section, and the figure which expands and shows a part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1a-1n Camera 2 Silhouette image generation part 3 Solid model generation part 4 Visual volume intersection method part 5 Erosion process part 6 Dilation process part 7 Adaptive erosion process part 10 Silhouette constraint information generation part 11 Silhouette number generation part 12 Silhouette constraint value conversion 20 Visible information generator 30 Cross-correlation calculator 31 Cross-correlation calculator 32 Weighting calculator 40 Maximum flow / minimum cut calculator

Claims (4)

A three-dimensional shape generation device that generates a three-dimensional shape model of a subject from a multi-viewpoint camera image obtained by imaging the subject from a plurality of angular directions,
Silhouette image generation means for generating a silhouette image from a multi-viewpoint camera image;
A solid model generation means for generating various solid models that are approximated three-dimensional shape models of a subject by applying a visual volume intersection method to the generated silhouette image;
Silhouette constraint information generating means for generating a silhouette constraint value based on the silhouette image and the solid model;
Using the solid model, a visible camera information generating unit that specifies a camera that visually images the surface of the subject for each vertex;
A cross-correlation value calculating means for calculating a cross-correlation value between camera images captured by the specified camera;
Means for weighting the calculated cross-correlation value using the generated silhouette constraint value, and outputting the weighted cross-correlation value;
For the solid model, a graph composed of nodes corresponding to surface vertices and edges connecting adjacent nodes is formed, and a maximum flow / minimum cut algorithm is applied using the weighted cross-correlation values. And a maximum flow / minimum cut calculating means for generating a three-dimensional shape model of the subject. 2. The three-dimensional shape generation apparatus according to claim 1, wherein the solid model generation unit performs an erosion process on the first solid model formed by applying the visual volume intersection method to the silhouette image and the first solid model. Obtained by performing erosion processing on the first solid model, the third solid model obtained by performing dilation processing on the first solid model, and the first solid model. A fourth solid model to be generated,
The maximum flow / minimum cut calculation means applies a maximum flow / minimum cut algorithm to a region formed between the surface of the fourth solid model and the surface of the third solid model, so that the total energy is minimum. A three-dimensional shape generation apparatus characterized in that a three-dimensional shape model is generated by obtaining a vertex group.   3. The three-dimensional shape generation apparatus according to claim 2, wherein the silhouette constraint information generation unit projects a surface vertex of the third solid model onto each silhouette image, and is a silhouette that is a boundary point between the subject silhouette and the background. An apparatus for generating a three-dimensional shape, wherein an edge is detected and a silhouette constraint value is generated using a number of cameras associated with a silhouette image forming a silhouette edge.   4. The three-dimensional shape generation apparatus according to claim 2, wherein the cross-correlation value calculation means calculates a cross-correlation value using a luminance signal of a camera image and also calculates a cross-correlation value for each color component of RGB. And a weighted addition using a cross-correlation value obtained from the color component.