JP2007017364A - Device and method for extracting silhouette, and device and method for generating three-dimensional shape data - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and a method for extracting a silhouette capable of extracting a silhouette even in the case of an object having any kind of hue, and extracting a highly accurate silhouette, and to generate three-dimensional shape data (a view volume) of the object by a view volume crossing method based on the acquired silhouette. <P>SOLUTION: In this three-dimensional shape data generation device 1 for generating three-dimensional shape data of the object by the view volume crossing method by extracting respectively a silhouette based on a plurality of background images and a plurality of acquired images acquired by being photographed from a plurality of mutually different directions, a random pattern background wherein colors are assigned at random to each of a plurality of domains is used as a background, and a diffused reflected color on an object domain on the object is estimated from other acquired images excluding one acquired image, and a silhouette deficiency caused by similarity between an object observation color and a background color is repaired. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a silhouette extraction device and a silhouette extraction method for extracting a silhouette of an object from a two-dimensional image obtained by photographing the object. The present invention also relates to a three-dimensional shape data generation device and a three-dimensional shape data generation method for generating three-dimensional shape data of the object by a view volume intersection method based on the silhouette obtained from the silhouette extraction device and the silhouette extraction method.

  Acquiring a 3D shape of an object that exists in the real world and making it processable on a computer is called 3D shape acquisition, and a large amount of 3D shape such as virtual reality is required. Automation is desired in certain fields. And this 3D shape acquisition technology is expected to be applied to various fields such as computer graphics (CG, Computer Graphics) in the electronic information storage and amusement fields represented by virtual museums. ing.

  As a three-dimensional shape acquisition method, a stereo method and a view volume intersection method are generally known. The stereo method is a method of calculating a depth by associating points on an object surface on a two-dimensional image photographed from different viewpoints, obtaining a distance to a target in the manner of triangulation. The visual volume intersection method is a technique based on silhouette constraints. Each object is photographed by a plurality of cameras having different viewpoints, and a silhouette (object projection area) is calculated from the difference between the photographed image and a background image prepared in advance. In this method, the foreground region) is extracted, and the viewing volume is obtained by obtaining the product space of the viewing cone corresponding to the silhouette. This visual volume becomes an acquired shape, and becomes a three-dimensional shape representing the object of the subject. The background image is an image obtained by photographing only the background without the subject object. A silhouette is an area representing the presence of an object on a photographed two-dimensional image, and is usually represented by binary data. In addition, an image obtained by photographing the background and the subject object is referred to as an acquired image.

FIG. 19 is a diagram for explaining the relationship between the visual volume and the object shape. As shown in FIG. 19, the silhouette constraint has an end point at the lens center of the camera C _i whose position and shooting direction are known, and passes through an arbitrary point on the silhouette R _i of the two-dimensional image shot by the camera C _i. That is, a target object O exists inscribed in a pyramidal open space composed of a set of half straight lines. Referred to as a view frustum body "in the camera C _i" this open space. In the case of two cameras C _i and C _j , the viewing volume is a product space (product region) of the viewing cone in the camera C _{i and} the viewing cone in the camera C _j .

  When comparing these stereo methods and the visual volume intersection method, the visual volume intersection method has an error because the time required for measurement is shorter than the stereo method and it is not necessary to associate points on the object surface. The obtained result (output) is not 2.5D data such as a distance image but 3D shape data, so the entire surrounding shape can be restored without bonding, and only the silhouette is required as data. There are various advantages such as being strong against noise generated above and being applicable to an object having a poor texture feature.

  In the visual volume intersection method having such various advantages compared to the stereo method, the silhouette extraction accuracy affects the accuracy of the obtained three-dimensional shape data. In the conventional simple silhouette extraction method for obtaining binary data of a silhouette by binarizing a difference value between a captured image and a background image prepared in advance using a single color background, There is an inconvenience that a highly accurate silhouette cannot be stably extracted depending on colors and lighting conditions.

For this reason, a technique for extracting silhouettes with higher accuracy has been demanded, and one technique is disclosed in, for example, Patent Document 1. The image processing device disclosed in Patent Document 1 is a device that suitably extracts a silhouette when the object O of the subject is a metal or the like and the background is reflected, and a background display device that changes the pattern. BD, a table T on which an object O whose pattern changes and a subject is placed, a photographing device CM that photographs the subject object O, the background display device BD, and the table T, and a comparison device SC that extracts a silhouette are configured. The 2D image A including the background of the pattern a and the subject object O is photographed by the photographing apparatus CM, and the two-dimensional image B including the background of the pattern b different from the pattern a and the subject object O is photographed by the photographing apparatus CM. The two-dimensional image A and the two-dimensional image B are compared for each pixel, and a portion of the two-dimensional image A that does not change is replaced with a constant value by the comparison device SC. Corresponding to A By comparing the two-dimensional image C in which only the background is captured, the pixel portion having no change is used as background data by the comparison device SC, and the background data portion is cut out from the two-dimensional image A (or two-dimensional image B) by the comparison device SC. Extract silhouettes.
JP 2001-148021 A

  By the way, when using a background of a single color, it is possible to improve the accuracy of silhouette extraction by using a background of a color different from the color of the object of the subject, but the background must be selected for each color of the object of the subject. There is an inconvenience that it is troublesome.

  Moreover, the above-mentioned Patent Document 1 does not have a specific description of the pattern, and it is unclear what the pattern is. The above-mentioned patent document 1 does not extract silhouettes by comparing an acquired image obtained by photographing the object O of the subject with a background image, but obtains two images obtained by using backgrounds having different patterns. Background data is created from the image, and a silhouette is extracted from the background data and one acquired image. Therefore, it is similar to the conventional method of extracting a silhouette by taking a difference value between two images.

  The present invention has been made in view of the above circumstances, and provides an extraction apparatus and a silhouette extraction method capable of extracting a silhouette with a predetermined accuracy regardless of the color of an object of a subject. With the goal. It is another object of the present invention to provide a silhouette extraction device and a silhouette extraction method that can improve the silhouette extraction accuracy by a method different from that of Patent Document 1. A three-dimensional shape data generation device and a three-dimensional shape data generation method for generating three-dimensional shape data (view volume) of an object by a visual volume intersection method based on silhouettes obtained from these silhouette extraction devices and silhouette extraction methods. The purpose is to provide.

  As a result of various studies, the present inventors have found that the above object is achieved by the present invention described below. That is, in one aspect according to the present invention, based on a background image acquired by capturing only the background and an acquired image acquired by capturing the background and the object of the subject, In the silhouette extracting apparatus for extracting a silhouette which is an area representing the presence of the object, the background is a random pattern background which is divided into a plurality of areas and a color is randomly assigned to each of the plurality of areas. It is characterized by.

  In another aspect of the present invention, a plurality of background images acquired by shooting only the background from a plurality of different directions, and the background and the object of the subject are respectively shot from the plurality of directions. In the silhouette extracting device that extracts a silhouette that is an area representing the presence of the object on one acquired image based on a plurality of acquired images, the diffuse reflection color in the target area on the object When estimated from other acquired images excluding one acquired image, and the color of the region on the one acquired image corresponding to the target region is closer to the estimated diffuse reflection color than a preset threshold value Repairs the silhouette by adding the region on the one acquired image corresponding to the target region to the silhouette. It is characterized in.

  In the silhouette extracting apparatus, the background is a random pattern background that is divided into a plurality of areas and a color is randomly assigned to each of the plurality of areas.

  In another aspect of the present invention, the silhouette extraction method is configured to capture a background by photographing only a background that is a random pattern background that is divided into a plurality of regions and a color is randomly assigned to each of the plurality of regions. An area representing the presence of the object on the acquired image based on the step of acquiring an image, acquiring the acquired image by photographing the object of the background and the subject, and the background image and the acquired image; And a step of extracting a certain silhouette.

  In another aspect of the present invention, the silhouette extraction method includes a step of acquiring a plurality of background images by respectively capturing only backgrounds from a plurality of different directions, and the background and the subject from the plurality of directions. A step of acquiring a plurality of acquired images by photographing each object, and extracting a silhouette that is a region representing the presence of the object on one acquired image based on the plurality of background images and the plurality of acquired images And estimating the diffuse reflection color in the target area on the object from other acquired images excluding the one acquired image, and estimating the color of the area on the one acquired image corresponding to the target area When the color is closer to the diffuse reflection color than a preset threshold value, the region on the one acquired image corresponding to the target region is By adding the serial silhouette, characterized by comprising the step of repairing the silhouette.

  In another aspect of the present invention, a plurality of background images acquired by shooting only the background from a plurality of different directions, and the background and the object of the subject are respectively shot from the plurality of directions. In the three-dimensional shape data generation device that generates the three-dimensional shape data representing the three-dimensional shape of the object by the visual volume intersection method based on the plurality of acquired images acquired by the method, the background is divided into a plurality of regions. A random pattern background in which a color is randomly assigned to each of the plurality of regions.

  According to another aspect of the present invention, a plurality of background images acquired by shooting only the background from a plurality of different directions, and the background and the object of the subject are respectively shot from the plurality of directions. A three-dimensional representation of the three-dimensional shape of the object by visual volume intersection method by extracting silhouettes that are regions representing the presence of the object on the plurality of acquired images based on the plurality of acquired images acquired by In the three-dimensional shape data generation device that generates shape data, the diffuse reflection color in the target region on the object is estimated from other acquired images except for one acquired image, and the one acquired image corresponding to the target region If the color of the area at is closer to the estimated diffuse reflection color than a preset threshold, the one acquisition corresponding to the target area By adding the area on the image silhouette corresponding to the one acquired image, characterized by repairing the silhouette corresponding to the one acquired image.

  Furthermore, in the three-dimensional shape data generation apparatus, the background is a random pattern background that is divided into a plurality of regions and a color is randomly assigned to each of the plurality of regions.

  In another aspect of the present invention, the three-dimensional shape data generation method is different from each other only in a background that is a random pattern background that is divided into a plurality of regions and a color is randomly assigned to each of the plurality of regions. Acquiring a plurality of background images by respectively capturing images from a plurality of directions, acquiring a plurality of acquired images by respectively capturing the background and the object of the subject from a plurality of different directions, and A three-dimensional shape that represents a three-dimensional shape of the object by a visual volume intersection method by extracting silhouettes that are regions representing the presence of the object on the plurality of acquired images based on a background image and the plurality of acquired images. Generating data.

  In another aspect of the present invention, a step of acquiring a plurality of background images by capturing only backgrounds from a plurality of different directions, and a step of capturing the background and the object of the subject from a plurality of different directions, respectively. Obtaining a plurality of acquired images, and extracting silhouettes that are regions representing the presence of the object on the plurality of acquired images based on the plurality of background images and the plurality of acquired images, respectively. A method of generating three-dimensional shape data representing a three-dimensional shape of the object by a visual volume intersection method, wherein a diffuse reflection color in a target region on the object is excluded from one acquired image And the color of the region on the one acquired image corresponding to the target region is the estimated diffuse reflection. If the color is closer to the preset threshold value, the region on the one acquired image corresponding to the target region is added to the silhouette corresponding to the one acquired image, thereby obtaining the one acquired image. The step of repairing the silhouette corresponding to is provided.

  In the silhouette extraction device and the silhouette extraction method configured as described above, since a random pattern background is used, silhouettes can be correctly extracted in an area of a predetermined ratio or more even for objects having various colors. Since the silhouette is repaired, it is possible to effectively repair the silhouette defect caused by the closeness between the observed color of the object and the background color, and the silhouette can be extracted with higher accuracy.

  In the three-dimensional shape data generation apparatus and the three-dimensional shape data generation method configured as described above, since a random pattern background is used, silhouettes are correctly extracted in an area of a predetermined ratio or more even for objects having various colors. 3D shape of the object can be obtained. Since the silhouette restoration is performed, it is possible to effectively repair the silhouette defect caused by the closeness between the observed color of the object and the background color, and the three-dimensional shape of the object can be obtained with higher accuracy.

Embodiments according to the present invention will be described below with reference to the drawings. In addition, the structure which attached | subjected the same code | symbol in each figure shows that it is the same structure, The description is abbreviate | omitted.
(Configuration of the embodiment)
FIG. 1 is a diagram illustrating a configuration of a three-dimensional shape data generation apparatus. FIG. 2 is a diagram illustrating a configuration of the data generation unit.

  In FIG. 1, the three-dimensional shape data generation apparatus 1 includes a photographing unit 10 that photographs a subject object O, and a data generation unit 20 that generates three-dimensional shape data from image data captured by the photographing unit 10. Composed.

  The imaging unit 10 includes, for example, an imaging device C that shoots an object O of a subject such as a digital video camera or a digital still camera and generates image data, a lighting fixture LT that illuminates the object O of the subject, and a background plate (not shown). And a BD. A plurality of imaging devices C are prepared so that the object O can be captured from a plurality of different directions and images can be acquired.

In the present embodiment, the plurality of imaging devices C _s (s = 1 to 19) capture the object O from above in different directions so that the object O can be captured from all directions (4π direction). Three imaging devices C _{1 to} C ₃ , twelve imaging devices C _{4 to} C ₁₅ that photograph the object O from the side in different directions, and four that photograph the object O from below in mutually different directions It is composed of 19 units of the base of the imaging device _C 16 _{-C 19.} The luminaire _LT t (t = _1~6), in order to substantially uniformly illuminate the object O, mutually different in the direction from the side of the six illuminating the object O of the luminaire _LT 1 to LT ₆ 6 It consists of a stand.

Then, these 19 units of the image pickup device C ₁ -C ₁₉ for fixing toward the predetermined shooting direction, and, in order to secure towards the luminaire LT ₁ to LT ₆ six in a predetermined illumination direction In this embodiment, a frame F to which the imaging devices C _{1 to} C ₁₉ and the lighting fixtures LT _{1 to} LT ₆ are attached is prepared. The frame F is made of, for example, 12 rod-shaped members FM _{1 to} FM ₁₂ made of steel and framed in a cubic shape of 900 mm × 900 mm × 900 mm, and the two rod-shaped members BM ₁ and BM ₂ are beams on a square frame constituting the upper surface. The rod-shaped members RD _{1 to} RD ₄ are respectively passed in a horizontal state substantially at the center in the _four square frames constituting the side surface.

Of the three imaging devices C _{1 to} C ₃ , the two imaging devices C ₁ and C ₂ are directed to a predetermined imaging direction with an interval between _one rod-like member BM ₁ that is passed like a beam. respectively attached to, the one imaging device C _3, and is mounted so as to face the predetermined imaging direction to the other bar-like member BM ₂ being passed as beam. The imaging device C _4, C ₅ of two of the twelve imaging device C ₄ -C ₁₅ faces toward the predetermined photographing direction spaced rod-like member RD ₁ being passed to one square frame constituting the side surface The two imaging devices C ₁₀ and C ₁₁ are respectively attached to the rod-shaped member RD ₃ that is passed over the square frame on the side surface opposite to the side surface so that the two imaging devices C ₁₀ and C ₁₁ face each other in a predetermined shooting direction. ing. Four imaging devices C _{6 to} C ₉ are attached to the rod-shaped member RD ₂ that is passed to the square frame on one of the remaining two side surfaces so as to face a predetermined photographing direction with a space therebetween. At the same time, the lighting fixture LT ₅ is attached so as to illuminate the inside of the substantially central portion of the rod-shaped member RD ₂ . Then, the four imaging devices C _{12 to} C ₁₅ are attached to the rod-shaped member RD ₄ that is passed to the square frame on the other side surface of the remaining two side surfaces so as to face each other in a predetermined shooting direction. In addition, a lighting fixture LT ₆ is attached to the substantially central portion of the rod-shaped member RD ₄ so as to illuminate the inside. Further, the lighting fixtures LT _{1 to} LT ₄ are respectively attached to the substantially central portions of the four rod-like members FM _{5 to} FM _{8 on} the side surface connecting the square frame on the upper surface and the square frame on the lower surface so as to face the inside. Yes.

In the present embodiment, the imaging device C _s and the lighting fixture LT _t are fixed by the frame F. However, the present invention is not limited to this. For example, they are suspended from the ceiling with wires or fixed with a stand. The point is that the imaging device C _s shoots the object O from a predetermined shooting direction, and the lighting fixture LT _t illuminates the object O from a predetermined lighting direction, respectively. I can do it.

  The object O as a subject is suspended using, for example, a thread ST so that the center of gravity is located at the approximate center of the observation environment.

  1 and 2, the data generation unit 20 includes an information processing unit 31, an input unit 32, an output unit 33, a storage unit 34, an interface unit 35, and a bus 36.

The input unit 32 executes various commands such as a command for starting a three-dimensional shape data generation program for generating three-dimensional shape data by the method of the present invention, and a three-dimensional shape data generation program such as a threshold described later. A device that inputs various data such as necessary data to the data generation unit 20, such as a keyboard and a mouse. The output unit 33 is a file name of the 3D model or 3D shape data of the object O based on the command or data input from the input unit 32 and the 3D shape data generated by the 3D shape data generation apparatus 1. For example, a display device such as a CRT display, LCD, organic EL display or plasma display, or a printing device such as a printer. Interface unit 35 are respectively connected to a plurality of imaging devices C _s by a communication line (not shown), an interface circuit for transmitting and receiving communication signals between a plurality of imaging devices C _s through the communication line, the communication signal processing unit 31 is a format processing data from a plurality of imaging devices C _s as well as create a communication signal in accordance with a predetermined communication protocol such as USB, for example, based on data from the information processing unit 31 Convert. Incidentally, the interface unit 35 may receive a plurality of imaging devices C _s communication signals, respectively, by wireless communication.

The storage unit 34 stores a background image data storage unit 341 that stores data of the background image B _s , an acquired image data storage unit 342 that stores data of the acquired image F _s , and data of the silhouette R _s during processing. A silhouette data storage unit 343 and a three-dimensional shape data storage unit 344 for storing three-dimensional shape data are provided, and various programs such as a three-dimensional shape data generation program, and data necessary for execution of various programs and the execution thereof Various data such as data generated in the memory is stored. The storage unit 34 is, for example, a volatile storage element such as a RAM (Random Access Memory) serving as a so-called working memory of the information processing unit 31, a ROM (Read Only Memory) or a rewritable EEPROM (Electrically Erasable Programmable Read Only Memory). And the like, and a hard disk for storing various programs and various data.

The information processing unit 31 includes, for example, a microprocessor and its peripheral circuits, and functionally stores data of a plurality of background images B _s and a plurality of acquired images F _s using a plurality of imaging devices C _s. each extract each image acquisition unit 311 to be stored in each acquired background image data storage unit 341 and acquires the image data storage unit 342, a plurality of background images B _s and a plurality of acquired images based on the F _s multiple silhouette R _s At the same time, the silhouette extracting unit 312 stores the data of the plurality of extracted silhouettes R _{s in} the silhouette data storage unit 343, and repairs the silhouette deficiency in the extracted plurality of silhouettes R _s respectively, and repairs the plurality of silhouettes R silhouette restoration unit 313 for storing the _s data on the silhouette data storage unit 343 , Three-dimensional shape for storing three-dimensional shape data the generated three-dimensional shape data storing section 344 generates the three-dimensional shape data of the hull method visual based on the plurality of silhouette R _s that these repair example voxel representation A data generation unit 314, and controls the input unit 32, the output unit 33, the storage unit 34, and the interface unit 35 according to the function according to the control program.

  The information processing unit 31, the input unit 32, the output unit 33, the storage unit 34, and the interface unit 35 are connected by a bus 36 so that data can be exchanged with each other.

  Such a data generation unit 20 can be configured by, for example, a computer, more specifically, a personal computer such as a notebook type or a desktop type.

  Note that the data generation unit 20 may further include an external storage unit 37 as indicated by a broken line as necessary. The external storage unit 37 stores data with a recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), a CD-R (Compact Disc Recordable), and a DVD-R (Digital Versatile Disc Recordable). An apparatus that performs reading and / or writing, such as a flexible disk drive, a CD-ROM drive, a CD-R drive, and a DVD-R drive.

Here, when the 3D shape data generation program or the like is not stored, it is configured to be installed in the storage unit 34 via the external storage unit 37 from the recording medium on which the 3D shape data generation program or the like is recorded. May be. Alternatively, the acquired data of the plurality of background images B _s , the data of the plurality of acquired images F _s , the generated data of the plurality of silhouettes R _s and the three-dimensional shape data are recorded on the recording medium via the external storage unit 37. You may comprise as follows.

FIG. 3 is a diagram for explaining the positional relationship among the imaging device, the object of the subject, and the background plate. Background plate BD, as shown in FIG. 3, a plurality prepared for each of a plurality of imaging devices _C a -C _e, the background plate _{BD a ~BD e,} each imaging device across the object O of the object _C a ~ It is arranged so as to face the C _e. It should be noted that a random pattern in which a color is randomly assigned to each of the plurality of divided regions RG _qr is drawn on one surface of the background plate BD. The background on which the random pattern is drawn is called a random pattern background.

FIG. 4 is a diagram illustrating an example of a random pattern background. FIG. 4A shows the entire random pattern background, and FIG. 4B shows an enlarged part of the lower right part. Since the figure cannot be displayed in color, in FIG. 4B, each color is expressed by a different texture. As shown in FIG. 4, the random pattern background is _divided into a plurality of regions RG _qr of 24 × 32, for example, and is uniformly distributed in a predetermined color space from colors observable by the imaging device C. the color selected at random are those assigned to the respective region RG _qr. In the present embodiment, the colors of the regions RG _qr are randomly selected so as to be uniformly distributed in the UV space since the U and V values of the YUV color system are used for silhouette extraction as will be described later. . Y, U, and V represent the luminance, the difference between the luminance signal and the R component, and the difference between the luminance signal and the B component, respectively.

FIG. 5 is a diagram for explaining the brightness of the random pattern background. Since the value of Y is not used for silhouette extraction, Y can be set as appropriate within the range of values that can be observed by the imaging apparatus C according to the illumination environment, but if the distribution in the UV space is biased by the value of Y, For example, when the value of Y is small and the background color is too dark, a region RG that is observed black by the imaging device C is generated as shown in FIG. Also, for example, if the value of Y is large and the background color is too bright, a region RG observed by the imaging device C in a manner that is blown out as shown in FIG. Probability this case the value of Y is not properly set, as, deviation occurs in the distribution in UV space color observed by the imaging devices C _s, silhouette defect in one of the image pickup device C _m occurs p becomes larger than the theoretical value. For this reason, the value of Y is set so as to be uniformly distributed in the UV space. An example in which the value of Y is set appropriately is shown in FIG.

  Further, when defocusing occurs, an intermediate color between adjacent regions RG is observed, and as a result, the distribution of the observed background color in the UV space is biased.

  The shape of the region RG is a square in the examples shown in FIGS. 4 and 5, but may be a triangle, another quadrangle, or a pentagon, and any shape can be used. Further, the shape of the region RG may be the same shape or a combination of a plurality of shapes.

  FIG. 6 is a diagram for explaining the size of the area of the random pattern background. If the size of the region RG is too small compared to the resolution of the imaging device C, the imaging device C will observe a uniform color as shown in FIG. On the other hand, if the size of the region RG is too large compared to the size of the object O, the background color appearing in the background portion of the object O is lost as shown in FIG. Use significance is lost. For this reason, the size of the region RG is appropriately set according to the size of the object O as the subject and the resolution of the imaging device C. An example in which the size of the region RG is appropriately set for the size of the object O and the imaging device C is shown in FIG.

Next, the operation of this embodiment will be described.
(Operation of the embodiment)
FIG. 7 is a flowchart showing an operation for generating three-dimensional shape data. FIG. 8 is a flowchart showing the detailed operation of silhouette restoration.

First, the background plate BD _s of the random pattern background is arranged at the location of the background plate BD _s as the background in the three-dimensional shape data generation device 1. Then, when an instruction to shoot a background is received from the user via the input unit 32, the image acquisition unit 311 in the information processing unit 31 of the three-dimensional shape data generation apparatus 1 has a plurality of objects in a state where the object O is not placed. taking respectively the background by the imaging device C _s, respectively obtains a plurality of background image B _s of data corresponding to a plurality of imaging devices C _s. The image acquisition unit 311 stores the acquired data of the plurality of background images B _s in association with the plurality of imaging devices C _{s in} the background image data storage unit 341 of the storage unit 34, respectively. Then, the image acquisition unit 311 outputs the effect that acquisition of the plurality of background image _{B s} is finished to the output section 33 (S11).

Next, the subject object O is placed at the subject placement location (in the observation environment) in the three-dimensional shape data generation apparatus 1. Then, when receiving via the input unit 32 instructs the user to shoot an object O, the image acquisition unit 311 captures each object O by a plurality of imaging devices C _s in this state, the plurality of imaging devices C _s corresponding plurality of data acquired image F _s to get, respectively. The image acquisition unit 311 stores the acquired data of the plurality of acquired images F _s in association with the plurality of imaging devices C _{s in} the acquired image data storage unit 342 of the storage unit 34, respectively. Then, the image acquisition unit 311 outputs the effect that acquisition of the plurality of acquired images _{F s} has been completed to the output section 33 (S12).

When instructed to generate a three-dimensional shape data from the user accepted via the input unit 32, the silhouette extraction unit 312 in the information processing unit 31 of the three-dimensional shape data generating apparatus 1, respectively corresponding to the plurality of imaging devices C _s more performed on the basis on the background image B _s and a plurality of acquired images F _s, the extraction of the plurality of silhouette R _s respectively corresponding to a plurality of imaging devices C _s, respectively (S13). In addition, when a plurality of background images B _s and a plurality of acquired images F _s respectively corresponding to the plurality of imaging devices C _s are acquired and stored, the silhouette extraction unit 312 does not wait for a user's instruction. it may be configured to perform the extraction of the silhouette R _s.

More specifically, the silhouette R _s is extracted as follows. Data of the data and the acquired image F _m of the background image B _m obtained by the photographing apparatus C _m is, R component is composed of G and B components. Here, it may be performed to extract the silhouette of R _m in the RGB color system, but from the difference image calculated by the RGB color system, a difference image is converted into YUV color system and HSV color system The person who wants can extract silhouettes with high accuracy. Therefore, the silhouette extraction unit 312 of the information processing unit 31 converts the data of the data and the acquired image _{F m} of the background image _{B m} in YUV color system or HSV color system. In the present embodiment, it converts the data of the data and the acquired image F _m of the background image B _m in YUV color system. Here, several conversion expressions for converting from the RGB color system to the YUV color system are known, but in the present embodiment, conversion is performed using Expression 1-1 to Expression 1-3.
Y = 0.299R + 0.587G + 0.114B (Formula 1-1)
U = −0.169R + −0.331G + 0.500B (Formula 1-2)
V = 0.500R-0.419G-0.081B (Formula 1-3)
The conversion from the RGB color system to the HSV color system can also be performed using a known conversion formula.

As in the three-dimensional shape data generation apparatus 1 in the present embodiment, in an illumination environment designed so that the illumination is substantially uniformly applied to the entire interior of the observation environment in which the object O of the subject is disposed, The difference in brightness value from the background observation color does not increase. For this reason, the silhouette extraction unit 312 includes the pixel values f _{m, y} (x, y), f _{m, U} (x, y), and the pixel values (x, y) of the acquired image F _m obtained by the imaging device C _m f _{m, V} (x, y) and the pixel value b _m, of the pixel (x, y) corresponding to the pixel (x, y) of the acquired image F _m in the background image B _m obtained by the imaging device C _{m The} silhouette R _m is obtained from Equation (2) from _y (x, y), b _{m, U} (x, y), b _{m, V} (x, y).
R _m = {(x, y) || f _{m, U} (x, y) −b _{m, U} (x, y) |> U _th or | f _{m, V} (x, y) −b _{m , V} (x, y) |> V _th } (Formula 2)
Here, U _th and V _th are threshold values of the background difference, and are set empirically and experimentally.

When the observed color of the object O and the background color are the same or close to each other, the difference in pixel value between corresponding pixels of the acquired image F _m and the background image B _m becomes small, and the silhouette R _m is missing. Will occur. When a single color background is used for each imaging device C _s, if _a silhouette loss occurs in one imaging device C _a , a silhouette loss occurs in another imaging device C _b . On the other hand, in the present invention, since a random pattern background is used as the background, different background colors are obtained between the imaging devices C _s , and each silhouette R _s is correctly extracted at a predetermined ratio or more even for an object O having various colors. be able to. In particular, it is suitable when an object O of various colors is used as a subject as in natural history.

Next, when each silhouette R _s is extracted by the silhouette extraction unit 312 in this way, the silhouette restoration unit 313 in the information processing unit 31 of the three-dimensional shape data generation apparatus 1 restores each extracted silhouette R _s . Each is performed (S14).

The observation color of the object O is the color of diffuse reflection light and specular reflection light. Diffuse reflected light is reflected light that is reflected in a uniform size in all directions in which the object surface can be observed when the object surface is irradiated with light. Specularly reflected light is a reflection that occurs on the surface of a mirror or polished metal, and when light is applied to the object surface, the incident direction of the light with respect to the normal vector of the object surface is equal to the reflection direction. The reflected light is strongly reflected in the total reflection direction. For this reason, when a plurality of imaging devices C _s observe the same part of the object O, the diffuse reflection color of that part is observed as the same color in each imaging device C _s .

If a silhouette deficit in a certain imaging device C _m can know the diffuse reflection color of the object O from the acquired image C _{n of} another imaging device C _n , the background color in the certain imaging device C _m and the other imaging device C can be obtained. It can be detected by comparing with the diffuse reflection color obtained from _n . Therefore, by adding the detected silhouette defect to the silhouette R _m , the silhouette R _m can be repaired, and in other words, the silhouette R _m with fewer silhouette defects can be extracted.

More specifically, the silhouette restoration is performed as follows. First, silhouette restoration unit 313, by using a plurality of silhouette R _s extracted in the processing S13, obtains a view volume by known processing procedure using the volume intersection method (e.g. three-dimensional shape of the voxel representation) (S21 ). Note that the viewing volume in the process S21 is referred to as an initial viewing volume. The information processing procedure for obtaining the view volume by the volume intersection method viewed from the plurality of silhouette R _s is an example and for example, "c Xiaojun, Tokai Shogo, Toshikazu Wada, Takashi Matsuyama, of body movements using a" PC Cluster “Real-time 3D visualization”, Information Processing Society of Japan, CVIM 121-9, 2000.3 ”.

  Next, the silhouette restoration unit 313 obtains an area (hereinafter, referred to as “recovery determination area”) including a voxel v that may have a defect (S22). Of the voxels v not included in the initial visual volume, if only the voxel v missing from the true visual volume is determined to be repaired, the erroneous voxel v will not be restored, so the initial visual volume defect is repaired. However, since the true visual volume cannot be known at this point, the area that includes the true visual volume and that does not include the extra area as much as possible is obtained, and the difference area between this and the initial visual volume is calculated. This is the return determination area.

More specifically, silhouette restoration unit 313 _obtains a recovery judgment region by obtaining the _{N allow = N th (N th} ≧ 1) the set view volume to the initial view volume of the difference area. N _{The allow} = _N the set view volume to _th are mutually among photographing direction different n-number of the imaging _device, a region included in the viewing cone of the _{(n-N allow)} more than one imaging apparatus. Defect from the true view volume is reduced when the N _th is increased but increased extra space. On the other hand, reducing the N _th more missing from the true view volume but does not increase so much extra space. Such trend in consideration, the complexity of the shape of the object O of the object, in consideration of noise or the like generated by the number and the imaging device of the imaging device for observing the object O, the threshold N _th is determined as appropriate. In this embodiment, N _th = 1.

Incidentally, in the case of the N _{allow =} N _th, probability p of the voxel v silhouette defects not included in the return determination area does not correct 'is an expression 3.

Here, p is the probability of pixels I of the voxel v is projected is background color and the same color, N'vis (e), the imaging device C voxel e is visible among the plurality of imaging devices C _e It is a number.

  Then, the silhouette restoration unit 313 gives the Manhattan distance from the visual volume surface to the voxel v included in the obtained return determination region, and returns determination voxel a for determining return in order from the smallest Manhattan distance. (S23).

Next, the silhouette restoration unit 313 determines the imaging device C _{s in} which the return determination voxel a can be seen (S24). In this determination, the presence / absence of another voxel v between the return determination voxel a and the imaging device C _m is checked. If there is no other voxel v, it is determined that the imaging device C _m can see the return determination voxel a. To do. FIG. 9 is a diagram for explaining a determination method of the imaging apparatus in which the return determination voxel can be seen. In FIG. 9, more specifically, in consideration of the case where a plurality of voxels v are projected onto the same pixel, a positive small threshold value z _th is set, and is the return determination voxel a visible from the imaging device C _m ? Whether or not the Z buffer value z _m (x, y) corresponding to the pixel I _m (x, y) onto which the return determination voxel a is projected is smaller than z _m (q _m (a)) + z _th. Judge by. q _m (a) is a vector from the lens center of the imaging device C _m to the return determination voxel a, and the magnitude of this optical axis direction component when the optical axis direction of the imaging device C _m of this vector is Z. It is. Here, if the threshold value z _th is set to a large value, it may be determined that even the voxel v on the back side of the visual volume can be seen. Therefore, the threshold value z _th is the return view formed by the set of the size of the object O and the return determination voxel a. A small threshold z _th is set in consideration of the width of the layer on the volume surface and the like. The value of the Z buffer is used in a known Z buffer method for determining which object color of a plurality of objects is reflected in each pixel value of an image and generating a projected image of the object in computer graphics. Value.

  Next, the silhouette restoration unit 313 obtains the diffuse reflection color of the return determination voxel a using Equation 4-1 and Equation 4-2 (S25).

Here, Ave _U (a) and Ave _V (a) are estimated values of the diffuse reflection colors U and V of the return determination voxel a, and Cin (a) is an imaging device in which the return determination voxel a is visible. of the set CVIS (a) of C _m, the pixel recovery judgment voxels a is projected is a set of an imaging device _{C m} contained in the silhouette _{R m.}

Next, the silhouette restoration unit 313 performs U and V dispersions Var _U (a) and Var _V (a) in the observation color of the imaging device C _m used for estimating the diffuse reflection color of the return determination voxel a in the process S25. The standard deviations Sig _U (a) and Sig _V (a) are obtained by Expressions 5-1, 5-2, 6-1, and 6-2, respectively (S26).

Next, the silhouette restoration unit 313 detects silhouette defects and repairs them (S27). Here, if the recovery judgment voxel a is part of (1) a concave, (2) a partial specular reflection is occurring, and, if a small number of the image pickup apparatus C _m which are visible (3) Since the diffuse reflection color of the return determination voxel a cannot be correctly determined, the silhouette defect is not repaired. By thus performing the repair silhouette defect can be repaired more appropriately silhouette deficiency, it is possible to obtain a more accurate silhouette R _m.

Here, when the object color observed in each part of the concave surface is different, the variances Var _U (a) and Var _V (a) become large, so whether the return determination voxel a is a concave surface part. Whether or not the variances Var _U (a) and Var _V (a) are larger than a predetermined threshold value α can be determined. The threshold value α is set empirically and experimentally.

In the imaging device C _m observing partial specular reflection occurs, we will observe a color different from the diffuse reflection color. Therefore, when the diffuse reflection color is estimated including the imaging device C _m that observes the portion where the specular reflection occurs, the estimated diffuse reflection color becomes a color due to the specular reflection. Therefore, the diffuse reflection color is estimated using only the pixel q _m (a) on which the return determination voxel a is projected and the values of U and V are close to the average value. That is, the diffuse reflection colors Ave _U (a) and Ave _V (a) are recalculated and estimated using the pixel q _m (a) on which the return determination voxel a satisfying the dispersion condition of Expression 7 is projected.

Here, β is a threshold value that allows non-specular reflection. The threshold value β is set empirically and experimentally.

Furthermore, if the number of which the imaging device C _m which appeared is small, it is impossible to eliminate the influence of the specular reflection from the diffuse reflection color to estimate. For this reason, when it can be assumed from the dispersion condition of Expression 7 that specular reflection is observed only at most N _spec imaging devices C _m with respect to the return determination voxel a, Nvis (a) is 2 × N. If it is not more than _spec + 1, no repair is performed. The threshold N _spec is set empirically and experimentally.

Then, the silhouette restoration unit 313 determines that a silhouette loss has occurred in the pixel q _m (a) that satisfies the average condition of Expression 8 using the diffuse reflection color that has been subjected to the above processing, and a silhouette loss has occurred. the return decision voxels a it is determined that the inclusion in the silhouette R _m.

The silhouette restoration unit 313 uses the silhouette R _m a restored silhouette deficient, obtaining the view volume by known processing procedure using the volume intersection method (e.g. three-dimensional shape of the voxel representation) (S28). The visual volume in this process S28 will be referred to as a repair visual volume.

  Next, the silhouette restoration unit 313 determines whether or not the silhouette restoration process is completed (S29). This determination is performed by, for example, comparing the previous repair visual volume with the current repair visual volume, and determining whether the current repair visual volume matches the previous repair visual volume within a predetermined threshold. As a result of the determination, if the current repair visual volume and the previous repair visual volume substantially coincide with each other, it is determined that the silhouette repair processing has been completed, while the current repair visual volume and the previous repair visual volume are determined. Is not substantially matched, it is determined that the silhouette restoration process has not ended.

  If it is determined as a result of the process S29 that the silhouette repair process has been completed (Yes), the silhouette repair unit 313 executes the process S15 of FIG. 7, while the silhouette repair process has been completed. If it is determined that there is no (No), the silhouette restoration unit 313 returns the process to step S21. In this case, in the process S21, the current repair visual volume is set as a new initial visual volume.

By performing the above processing on each silhouette R _s , it is possible to effectively repair a silhouette defect caused by the closeness between the observed color of the object and the background color. Therefore, it is possible to obtain each silhouette R _s good accuracy. In particular, the random pattern background according to the present invention, since the background colors corresponding to the respective imaging devices C _s for a certain portion of the object O are different can be expected, the probability that it is possible to repair the silhouette deficiency very high it is possible to obtain a more accurate each silhouette R _s to.

Returning to FIG. 7, when silhouette repair is complete, three-dimensional shape data generating portion 314 of the information processing unit 31 uses the data of the repaired silhouette R _s, a known information processing procedure using the volume intersection method For example, voxel representation three-dimensional shape data is generated. When three-dimensional shape data corresponding to the object O of the subject is generated, the three-dimensional shape data generation unit 314 attaches a file name to the generated three-dimensional shape data and stores the three-dimensional shape data storage unit of the storage unit 34. The 3D shape and the file name based on the generated 3D shape data are output to the output unit 33 (S15).

Such three-dimensional shape data generating apparatus 1 by operating the can extract the correct silhouette R _s at a predetermined ratio or more areas even for objects with different colors, and observation of the object O It is possible to effectively repair a silhouette defect caused by the closeness between the color and the background color. For this reason, the three-dimensional shape data generation apparatus 1 can generate the three-dimensional shape of the object O with higher accuracy.

Since such a three-dimensional shape data generation apparatus 1 can output the three-dimensional coordinates of the object, an arbitrary viewpoint image is generated by restoring the three-dimensional model in the virtual space from the three-dimensional coordinates of the object. You can also
(Example)
Three-dimensional shape data for various objects O was created using this three-dimensional shape data generation apparatus 1. The results are shown below.

FIG. 10 is a diagram illustrating an acquired image and a silhouette for the first object. FIG. 10 (A) shows an acquired image F _m obtained by photographing a first object using a random pattern background, FIG. 10 (B) shows a silhouette R _m before silhouette restoration, and FIG. (C) shows the silhouette _{R m} after silhouette repair. Region indicated by white shown in FIG. 10 (B) and (C) is a silhouette _{R m.} FIG. 11 is a diagram illustrating a first object and a viewing volume (three-dimensional shape) with respect to the first object. FIG. 11A shows the first object O, FIG. 11B shows the correct viewing volume, and FIG. 11C shows the viewing volume using a random pattern background and without silhouette restoration. FIG. 11 (D) shows a visual volume obtained by the visual volume intersection method using a random pattern background and _Nallow = 1, which is a visual volume obtained by the crossing method (hereinafter referred to as “normal visual volume”). The volume (hereinafter referred to as “N _allow = 1 viewing volume”) is shown, and FIG. _11E shows the viewing volume obtained by the viewing volume intersection method using a random pattern background and performing silhouette restoration. (Hereinafter referred to as “repair visual volume”).

In this embodiment, the three-dimensional shape data generation apparatus 1 extracts silhouettes in the YUV color system, and each threshold value is V _th = U _th = 10, z _th = 2, N _th = 1, N _spec = 1, β was set to 0.5. As shown in FIG. 11A, the first object O that is a subject is a dinosaur model that is mostly green. The correct visual volume is obtained by manually extracting the silhouette R _s for the acquired images F _s obtained from all the imaging devices N _s . Further, in FIGS. 11B to 11D, the marching cube method is applied to each viewing volume, and a surface patch is configured and displayed. The surface color is an average value of diffuse reflection colors obtained in the obtained visual volume.

  As can be seen by comparing FIG. 10B and FIG. 10C, silhouette loss due to the closeness between the observed color of the first object O and the background color is reduced by performing the silhouette restoration according to the present invention. Yes.

As can be seen by comparing FIG. 11C and FIG. 11E, the normal visual volume has a large defect in the back and tail, but is obtained by the three-dimensional shape generation apparatus 1 according to the present invention. The visual volume of the repaired image is closer to the correct visual volume, since the defect in this part is almost repaired. Further, as can be seen by comparing FIG. 11 (D) and FIG. 11 (E), the visual volume of the repair is an extra area that is confirmed around the back and corners of the mane as compared to the visual volume of _Nallow = 1. Is decreasing and closer to the correct visual volume.

  Table 1 shows the correct viewing volume and the results of numerical evaluation for each viewing volume. The number of additional voxels in Table 1 is the number of voxels that are not included in the correct visual volume but are included in the visual volume. The number of missing voxels (missing voxels) is the number of voxels that are included in the correct visual volume but not included in the visual volume. The number of error voxels (error voxels) is the sum of the number of additional voxels and the number of lost voxels.

As can be seen from Table 1, the repaired visual volume has a smaller number of additional voxels and a smaller number of lost voxels compared to the visual volume of _Nallow = 1. The error number of voxels is made smaller than the case in the normal viewing volume or N _{The allow = 1} the view volume.

  The results of creating the three-dimensional shape data for the second to sixth objects O using the three-dimensional shape data generation apparatus 1 are shown in FIGS.

  FIG. 12 to FIG. 16 are views showing the viewing volumes (three-dimensional shapes) for the second to sixth objects and the second to sixth objects. Each figure (A) shows each object, each figure (B) shows a normal viewing volume, and each figure (C) shows a repairing viewing volume.

  As shown in FIG. 12A, the second object O is a gray image model, and the third object O is a white image model as shown in FIG. The object O is mainly a white female model as shown in FIG. 14 (A), and the fifth object O is mainly a brown case model as shown in FIG. 15 (A). The sixth object O is mainly a brown horse model as shown in FIG.

  As can be seen from each figure (B), silhouettes can be extracted with a predetermined accuracy to obtain a normal visual volume even for objects O of various shades. As can be seen from each figure (C), a visual volume with few defects is obtained for the object O of various shades by performing the silhouette restoration.

In the above-described embodiment, the three-dimensional shape data of the object is generated from the product area of all viewing cones in n cameras having different shooting directions. You may produce | generate from the area | region included in the viewing cone of a camera of (n- _Nallow ) or more units among cameras. _Here, N allow = _{1,2,3, ···,} it is n-1.

In the aforementioned embodiment, the case is not particularly subjected to post-processing after the silhouette extraction, the object O of the subject is not projected as a small isolated points certain area following in all of the imaging device C _s is Post-processing may be performed to remove silhouettes with a small area.

In the above-described embodiment, the silhouette restoration is performed on the acquired image F _m obtained using the random pattern background. However, the silhouette restoration according to the present invention is also applied to the case where a single color background is used as in the past. Can do. As is clear from the information processing procedure for silhouette restoration described above, if there is a silhouette defect due to the presence of a unique part that is shaded in the background, the silhouette defect is caused by the noise of the imaging device. The silhouette R _s can be repaired when the mirror reflection is observed and when specular reflection is observed.

  As an example, silhouette restoration was performed for the case where the dinosaur model shown in FIG. The results are shown below.

FIG. 17 is a diagram illustrating an acquired image and a silhouette for a first object in a single color background. FIG. 17 (A) shows an acquired image F _m where the first object photographed obtained using a solid color background, FIG. 17 (B) is shown a silhouette before repair silhouette, and, FIG. 17 (C ) shows a silhouette _{R m} after silhouette repair. Region indicated by white shown in FIG. 17 (B) and (C) is a silhouette _{R m.} FIG. 18 is a diagram illustrating a viewing volume (three-dimensional shape) for the first object and the first object in a single color background. 18A shows the first object O, FIG. 18B shows the correct viewing volume, and FIG. 18C shows the normal viewing volume when a single color background is used. FIG. _18D shows a view volume of N _allow = 1 when a single color background is used, and FIG. _18E shows a repair view volume when a single color background is used.

In this example, as in the above-described embodiment, the three-dimensional shape data generation apparatus 1 extracts silhouettes in the YUV color system, and each threshold value is V _th = U _th = 10, z _th = 2 and N _th = 1, N _spec = 1, β = 0.5.

  As can be seen by comparing FIG. 17B and FIG. 17C, silhouette loss due to the closeness between the observed color of the first object O and the background color is reduced by performing the silhouette restoration according to the present invention. Yes.

As can be seen by comparing FIG. 18C and FIG. 18E, the normal visual volume has a large defect in the tail portion, but the restoration obtained by the three-dimensional shape generation apparatus 1 according to the present invention. The visual volume of this area is almost restored to the defect of this part, and is closer to the correct visual volume. Further, as can be seen by comparing FIG. 18D and FIG. _18E , the visual volume of the repair is smaller than the visual volume of N _allow = 1, and the extra area that is confirmed on the back of the mane and around the waist is reduced. It is closer to the correct visual volume.

  Table 2 shows the correct viewing volume and the results of numerical evaluation for each viewing volume.

As can be seen from Table 2, the error number of voxels in the view volume of the repair is made smaller as compared with the case in the normal viewing volume or N _{The allow = 1} the view volume.

It is a figure which shows the structure of a three-dimensional shape data generation apparatus. It is a figure which shows the structure of a data generation part. It is a figure for demonstrating the positional relationship of an imaging device, a to-be-photographed object, and a background board. It is a figure which shows an example of a random pattern background. It is a figure for demonstrating the brightness of a random pattern background. It is a figure for demonstrating the magnitude | size of the area | region of a random pattern background. It is a flowchart which shows the operation | movement which produces | generates three-dimensional shape data. It is a flowchart which shows the detailed operation | movement of a silhouette restoration. It is a figure for demonstrating the determination method of the imaging device which can see a return determination voxel. It is a figure which shows the acquired image and silhouette with respect to a 1st object. It is a figure which shows the visual volume (three-dimensional shape) with respect to a 1st object and a 1st object. It is a figure which shows the visual volume (three-dimensional shape) with respect to a 2nd object and a 2nd object. It is a figure which shows the visual volume (three-dimensional shape) with respect to a 3rd object and a 3rd object. It is a figure which shows the visual volume (three-dimensional shape) with respect to a 4th object and a 4th object. It is a figure which shows the visual volume (three-dimensional shape) with respect to a 5th object and a 5th object. It is a figure which shows the visual volume (three-dimensional shape) with respect to a 6th object and a 6th object. It is a figure which shows the acquired image and silhouette with respect to the 1st object in a single color background. It is a figure which shows the visual volume (three-dimensional shape) with respect to the 1st object in the 1st object and a single color background. It is a figure for demonstrating the relationship between a visual volume and an object shape.

Explanation of symbols

BD Background plate C Imaging device LT Lighting fixture O Object RG Background plate area 1 3D shape data generation device 10 Imaging unit 20 Data generation unit 31 Information processing unit 32 Input unit 33 Output unit 34 Storage unit 35 Interface unit 311 Image acquisition unit 312 Silhouette extraction unit 313 Silhouette restoration unit 314 3D shape data generation unit 341 Background image data storage unit 342 Acquired image data storage unit 343 Silhouette data storage unit 344 3D shape data storage unit

Claims (10)

A silhouette that is a region representing the presence of the object on the acquired image based on a background image acquired by capturing only the background and an acquired image acquired by capturing the background and the object of the subject In the silhouette extraction device that extracts
The silhouette extraction apparatus according to claim 1, wherein the background is a random pattern background that is divided into a plurality of areas and a color is randomly assigned to each of the plurality of areas. Based on a plurality of background images acquired by shooting only the background from a plurality of different directions, and a plurality of acquired images acquired by shooting the background and the object of the subject from the plurality of directions, respectively. In a silhouette extraction device that extracts a silhouette that is an area representing the presence of the object on one acquired image,
The diffuse reflection color in the target area on the object is estimated from other acquired images excluding the one acquired image, and the color of the area on the one acquired image corresponding to the target area is the estimated diffuse reflection color. When the color is closer to a preset threshold, the silhouette is restored by adding the region on the one acquired image corresponding to the target region to the silhouette. . The silhouette extraction device according to claim 2, wherein the background is a random pattern background that is divided into a plurality of areas, and a color is randomly assigned to each of the plurality of areas. Obtaining a background image by capturing only a background that is a random pattern background divided into a plurality of regions and randomly assigned to each of the plurality of regions; and
Acquiring an acquired image by photographing the background and the object of the subject;
Extracting a silhouette, which is a region representing the presence of the object on the acquired image, based on the background image and the acquired image. Acquiring a plurality of background images by shooting only the background from a plurality of different directions, respectively;
Acquiring a plurality of acquired images by respectively capturing the background and the object of the subject from the plurality of directions;
Extracting a silhouette that is an area representing the presence of the object on one acquired image based on the plurality of background images and the plurality of acquired images;
The diffuse reflection color in the target area on the object is estimated from other acquired images excluding the one acquired image, and the color of the area on the one acquired image corresponding to the target area is the estimated diffuse reflection color. A step of repairing the silhouette by adding the region on the one acquired image corresponding to the target region to the silhouette when the color is closer to a preset threshold value. To extract silhouettes. Based on a plurality of background images acquired by shooting only the background from a plurality of different directions, and a plurality of acquired images acquired by shooting the background and the object of the subject from the plurality of directions, respectively. In a three-dimensional shape data generation device that generates three-dimensional shape data representing the three-dimensional shape of the object by a visual volume intersection method,
The background is a random pattern background that is divided into a plurality of regions, and a color is randomly assigned to each of the plurality of regions. Based on a plurality of background images acquired by shooting only the background from a plurality of different directions, and a plurality of acquired images acquired by shooting the background and the object of the subject from the plurality of directions, respectively. A three-dimensional shape data generation device that extracts silhouettes that are regions representing the presence of the object on the plurality of acquired images and generates three-dimensional shape data representing the three-dimensional shape of the object by a visual volume intersection method. In
The diffuse reflection color in the target area on the object is estimated from other acquired images except for one acquired image, and the color of the area on the one acquired image corresponding to the target area is set in advance to the estimated diffuse reflection color. Corresponding to the one acquired image by adding the region on the one acquired image corresponding to the target region to the silhouette corresponding to the one acquired image when the color is closer to the set threshold value A three-dimensional shape data generation device characterized by repairing the silhouette. The three-dimensional shape data generation apparatus according to claim 7, wherein the background is a random pattern background that is divided into a plurality of regions and a color is randomly assigned to each of the plurality of regions. Obtaining a plurality of background images by photographing only a background which is a random pattern background divided into a plurality of regions and randomly assigned to each of the plurality of regions from a plurality of directions different from each other;
Acquiring a plurality of acquired images by photographing the background and the object of the subject from a plurality of different directions, respectively;
Based on the plurality of background images and the plurality of acquired images, silhouettes that are regions representing the presence of the object on the plurality of acquired images are respectively extracted, and the three-dimensional shape of the object is represented by a visual volume intersection method. And a step of generating three-dimensional shape data. A step of acquiring a plurality of background images by respectively capturing only backgrounds from a plurality of different directions, and a step of acquiring a plurality of acquired images by respectively capturing the background and the object of the subject from a plurality of different directions. And a three-dimensional shape of the object by a view volume intersection method by extracting silhouettes that are regions representing the presence of the object on the plurality of acquired images based on the plurality of background images and the plurality of acquired images. Generating three-dimensional shape data representing the three-dimensional shape data generating method,
The diffuse reflection color in the target area on the object is estimated from other acquired images except for one acquired image, and the color of the area on the one acquired image corresponding to the target area is set in advance to the estimated diffuse reflection color. Corresponding to the one acquired image by adding the region on the one acquired image corresponding to the target region to the silhouette corresponding to the one acquired image when the color is closer to the set threshold value A method for generating three-dimensional shape data, comprising: restoring the silhouette.