JP2003067775A - Texture mapping method, texture mapping processing program, and computer-readable storage medium storing the program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To sufficiently establish practical usability by allowing a high-speed processing. SOLUTION: From the center of each plane of a polygon for enclosing a three-dimensional object, the image when photographing the object is acquired (S101), and the distance histogram and distance profile from the acquired outline image to an edge are formed (S103). The image of the three-dimensional object is taken by a camera (S201), and its distance histogram and distance profile are formed (S205). The matching of the distance histogram calculated for the profile of the three-dimensional object with the distance histogram of the model data is performed (S207), and the matching of an actual image with a model data image is performed by use of the distance profiles (S209). Further, candidates are narrowed by outline matching (S211), and the position and direction of the camera are precisely determined (S213).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a texture mapping method, a texture mapping processing program, and a computer-readable recording medium recording the program. INDUSTRIAL APPLICABILITY The present invention is used for three-dimensional computer graphics and three-dimensional image generation, in particular, for taking an image of a three-dimensional object in a computer and measuring its geometrical shape and a technique for pasting textures including surface colors and patterns. Related to texture mapping technology.

[0002]

2. Description of the Related Art Generally, in order to generate three-dimensional computer graphics (CG), the surface shape of a three-dimensional object and its texture (texture is, for example, feeling of surface color, pattern, material, etc.). Need to get In order to generate a three-dimensional CG, there is a method using an optical system scanner when a three-dimensional object is actually present. Most optical scanners acquire data on the surface of an object by rotating a camera and a slit projector around the object or by fixing the camera and rotating the object. Some of such optical scanners can simultaneously acquire the shape and texture of the object surface, and can easily reproduce the three-dimensional CG data.

On the other hand, there is a relatively inexpensive optical system scanner that acquires only the shape of the surface of an object, and if this is used, it is possible to obtain a somewhat accurate surface shape. Since the surface shape acquired by such means does not include any texture, it is necessary to obtain the surface texture.

[0004]

As described above, when a three-dimensional real object (three-dimensional object) is introduced into a computer and used for computer graphics, the geometric shape and the texture including the surface color and pattern are generated. is necessary. However, a device that can both measure a geometric shape and paste a texture (mapping) is extremely expensive, and the size of the device may limit the size of a target object to be handled. Therefore, a method has been proposed in which the shape is measured by a device that measures only the geometrical shape, then the texture is acquired by a digital camera, and the image is pasted. However, it currently takes about 3 to 5 minutes per image and 30 minutes to 1 hour for about 10 images, which is not practical.

In view of the above points, the present invention is capable of performing high-speed processing (for example, about 20 seconds or less per image), and has a practicable texture mapping method, texture mapping processing program, and its program. An object of the present invention is to provide a computer-readable recording medium in which is recorded.

Another object of the present invention is to accurately map a texture image on the surface of an object based on the contour of the object given only by the shape obtained by photographing a real object from multiple directions with a digital camera. It is another object of the present invention to provide a multi-stage search method for accurately and rapidly determining the mutual positional relationship between an object and a camera.

[0007]

According to the solving means of the present invention, the processing unit, based on the three-dimensional shape model data representing the three-dimensional object, from the central viewpoint of each surface of the polyhedron enclosing the three-dimensional object, The step of acquiring the two-dimensional virtual image data for each viewpoint when the three-dimensional object is photographed, and the processing unit, from the acquired virtual image data for each viewpoint, for each predetermined angle based on the center or the center of gravity of the image. A step of creating a virtual distance profile representing the distance to the edge, the processing unit, based on the created virtual distance profile for each viewpoint, a step of creating a virtual distance histogram of a predetermined stage for virtual image data, The processing unit inputs a real image obtained by shooting a real object corresponding to the three-dimensional shape model data with the real camera, and the processing unit uses the obtained real image. From the center or the center of gravity of the image to create an actual distance profile that represents the distance to the edge for each predetermined angle; and the processing unit, based on the created actual distance profile, The step of creating a real distance histogram and the processing unit compare the virtual distance histogram and the real distance histogram for each viewpoint, extract a predetermined number of virtual distance histogram candidates with a high degree of coincidence, and select the camera related to that candidate. A distance histogram matching step of storing data including a position in a memory, the processing unit reads a virtual distance profile corresponding to the extracted virtual distance histogram from the memory, compares the virtual distance profile with an actual distance profile, Extract the matching virtual distance profile candidates and And a distance profile matching step of storing data including a camera rotation angle in a memory, the processing unit reads the extracted candidates from the memory, and based on the virtual silhouette image and the real silhouette image representing the shapes of the virtual image and the real image, A texture mapping method and a texture mapping method including a silhouette image matching step of determining a position and a rotation angle of an actual camera by selecting a matching candidate, a texture mapping processing program, and a computer-readable recording medium recording the program. Will be provided.

[0008]

BEST MODE FOR CARRYING OUT THE INVENTION 1. Overview The embodiment of the present invention is to photograph a real object from multiple directions with a digital camera and accurately map a texture image on an object surface given only a shape. For that purpose, it is necessary to accurately determine the mutual positional relationship between the object and the camera. In the embodiment of the present invention, the contour of the object is obtained from the image captured by the digital camera, and the accurate position of the camera is searched in multiple stages.

Specifically, the following processing is executed using a digital camera and a computer. (1) Based on the 3D shape model data of the 3D object,
From the center of each surface of a polyhedron (for example, a 320-hedron) that encloses a three-dimensional object, an image (outline of the object) when the object is photographed is acquired. Next, a distance histogram and a distance profile from the acquired silhouette image to the edge are created. (2) A three-dimensional object is actually photographed by a digital camera, and the distance histogram calculated for the contour of the three-dimensional object is matched with the distance histogram of the model data created in (1) above to obtain the camera position. Is fixed within a certain range. Here, as an example, the search range is limited to about 20/320 (6%). (3) Using the distance profile from the center of the contour, the actual image and the model data image are matched, and the range of the camera position is further narrowed. Here, as an example, the range is narrowed to about 20 / 7,200. (4) From the narrowed camera position candidates (here, 20 as an example), the number is narrowed to one by silhouette matching between the actual image and the model data image. (5) With respect to the last remaining one, silhouette matching is performed within a solid angle of a predetermined angle (here, 9 degrees as an example), and the position and direction of the camera are accurately determined.

By the above method, the position and direction of the camera can be adjusted. Even a general personal computer, for example, 20
The image can be pasted in less than a second, the total required time is shortened to about 3 minutes, and it can be sufficiently put into practical use. This time is determined by the computers and graphics cards available on the market, but progress in this area is remarkable and can be further reduced in the future.

2. Hardware FIG. 1 shows an example of a schematic diagram of the texture mapping device. This device consists of a personal computer 1, a camera 2
And, for example, a CG image is created for the model 3 by texture mapping.

FIG. 2 shows a block diagram of the texture mapping device. Personal computer (PC)
Reference numeral 1 denotes a processing unit 11 having a graphic card equipped with a geometry engine, an interface 12 with the camera 2 such as an IEEE 1394 interface, a memory 13 for storing various data such as intermediate results, final results, input data, output data and programs, and various data. And an output unit 15 for outputting various data including a display. The graphic card of the processing unit 11 is for obtaining a two-dimensional image when the model is photographed from the three-dimensional shape model data, for example. The camera 2 is, for example, IEEE139.
It is a digital camera or a video camera that can be connected.

FIG. 3 shows an explanatory diagram of the input / output data file. The input data file includes three-dimensional shape model data and captured image data. The three-dimensional shape model data is polygon data representing a three-dimensional object, and is represented by, for example, vertex coordinates, surface configuration data (floating point format), or the like. The captured image data is output from the camera 2 and is two-dimensional image data. On the other hand, the output data file is three-dimensional data obtained by performing texture mapping processing by pasting the texture based on the captured image data onto the three-dimensional shape model data.

FIG. 4 shows an explanatory view of the distance profile and the distance histogram data file. As for the three-dimensional shape model data as shown in FIG.
Data about the two-dimensional model image captured by the virtual camera from the viewpoint of 320 is stored. That is, the virtual distance profile and the virtual distance histogram for each viewpoint are stored. On the other hand, as shown in FIG. 4B, with respect to the two-dimensional image of the actual three-dimensional object photographed by the camera 2, the actual distance profile and the actual distance histogram for each given number are stored.

The distance profile and the histogram will be described later. The input data file, the output data file distance profile, and the distance histogram data file described above are stored in the memory 13.

3. Software FIG. 5 shows an explanatory diagram of the real optical system and the virtual optical system. FIG. 1A shows an actual optical system, which is an optical system when an actual object is photographed by an actual camera. Further, FIG. 3B shows a virtual optical system, which is an optical system when a three-dimensional model is photographed by a virtual camera on a computer. In the embodiment of the present invention, the physical scales of the real optical system and the virtual optical system are set to be the same, and two of the three-dimensional objects obtained when the virtual camera is placed in exactly the same position and direction as when shooting with the real camera. It is possible to create a textured (mapped) 3D model by replacing the 2D projection image with a 2D image obtained from a real camera and backprojecting it onto a 3D object.

The texture mapping method will be described below. FIG. 6 shows a flowchart of the texture mapping method. This method includes preprocessing and picture processing.

(1) Pre-processing (step S101) The processing unit 11 of the computer 1
It is polygon data representing a three-dimensional object from the input data file of the memory 12. Read the 3D shape model data. (Step S103) Next, when the processing unit 11 of the computer 1 uses the three-dimensional shape model data, the three-dimensional object is photographed from the center of each surface of the polyhedron (in this example, 320-hedron) enclosing the three-dimensional object. The two-dimensional image (virtual image) is obtained, and the distance profile and the distance histogram are obtained for the virtual image.

FIG. 7 is an explanatory view of 320 photographing points arranged around the object. As illustrated, the processing unit 11 arranges the viewpoint (camera position) at the center of each triangular surface of a 320-sided body obtained by dividing the regular icosahedron. The processing unit 11 uses a video card (graphic card) mounted inside the processing unit 11 of the PC 1 to acquire a two-dimensional image of the three-dimensional shape model data, and distances from the two-dimensional image (silhouette image). Profile,
A histogram is created as detailed below.

FIG. 8 shows a flowchart for creating the distance profile and the distance histogram. This corresponds to step S103.

(1-1) Extraction of Edge and Silhouette (S151) The processing unit 11 projects a three-dimensional object from 320 viewpoints based on the three-dimensional shape model data read from the memory 13 (two-dimensional). Obtained by rendering a virtual image. Furthermore, the silhouette edge is extracted from the virtual image.

(1-2) Creation of distance profile (S1
53) The processing unit 11 measures the distance from the center (center of gravity) of the virtual image to the edge of the object at every predetermined angle (here, 1 degree). Further, the processing unit 11 creates a distance profile representing the measured distance with respect to the angle and stores the distance profile in the virtual optical system in the memory 13 and the distance histogram data file.

An example is shown below. FIG. 9 is an example of a silhouette having a simple contour line. The processing unit 11 measures the distance from the center of gravity to the contour line, as indicated by the arrow in the figure.

FIG. 10 shows a distance profile created from the contour line of FIG. The vertical axis represents distance and the horizontal axis represents angle.
FIG. 11 is an example of a silhouette having a complicated contour line.
FIG. 12 shows a distance profile created from the contour line of FIG.

(1-3) Normalization (S155) Next, the processing unit 11 normalizes the distance profile created in (1-2). In the normalization method, the average value of the distance profile is set to a predetermined value (for example, 10000). FIG. 13 shows an explanatory diagram for normalization of the distance profile. The processing unit 11, as shown,
The average value of the distance profile is set to a constant value (10000).

(1-4) Creation of distance histogram (S1
57) The processing unit 11 counts the sampling points for each fixed distance (for example, 2000) in the normalized distance profile, creates a distance histogram at a predetermined stage (10 stages in this example), and stores the memory histogram in the memory 13. It is stored in the virtual optical system's distance profile and distance histogram data file. It should be noted that a predetermined distance or more may be included in the final stage (in this example, 1000 or more).
0 or more is included in the 10th element) or may be excluded.

In each step, the processing unit 11 writes or stores information on the two-dimensional image, the silhouette image, the edge, the distance to the edge, the distance profile, the normalization, and the histogram in the memory 13 as necessary. Read from 13.

(2) Picture Processing (Step S201) Next, the processing section 11 causes the camera 2
By laying a blue or green sheet on the background as needed,
The object is photographed and the data is captured via the I / F 12. The captured actual image data is appropriately stored in the input data file of the memory 13 as captured image data.

(Step S203) The processing section 11 carries out a processing for removing the background of the photographed image based on the fetched real image data.

(Step S205) Next, the processing unit 11
Is a distance profile (actual distance profile) for the actual image by extracting the edge of the object from the photographed image, and a distance histogram (actual distance histogram) for the distance profile. And store it in the distance histogram data file. These processes are similar to the above.

(Step S207) The histogram matching processor 11 reads from the memory 13 the virtual distance histogram of the virtual optical system created in the preprocessing and the actual distance histogram of the actual optical system created in step S205. The processing unit 11 compares these distance histograms with each other and takes out a match (high match). In the comparison of the histograms, the processing unit 11 does not need to consider the rotation in the z direction, so that the comparison can be performed at high speed. The processing unit 11 stores the data regarding the extracted histogram in the memory 13. For example, the viewpoint numbers of the selected candidates may be stored in an appropriate intermediate file in the memory 13, or may be stored together with the distance profile and the distance histogram. Here, for example, 320
The number of candidates will be reduced to about 20.

As a concrete method for selecting the candidates, for example, the similarity is calculated by the sum s _j of the following equation, and a predetermined number (about 20 in this example) having a high similarity is selected. In this way, the camera position is determined.

[0033]

[Equation 1]

Here, H (m) (m = 1 to 10) is a 10-level real distance histogram of the real optical system, h
_j (m) (m = 1 to 10, j = 1 to 320) is a 10-level virtual distance histogram from 320 viewpoints of the virtual optical system.

(S209) Distance Profile Matching Next, the processing unit 11 reads out the candidates from the memory 13 for the matching histograms in step S207, and compares the distance profiles. In this example, the number of distance profiles is reduced to 20, but each virtual distance profile has a 360-degree element for each degree when the predetermined angle is 1 degree.
There are 20 × 360 patterns to be compared.

FIG. 14 shows a flowchart of distance profile matching. The processing unit 11 reads out the real distance profile and the virtual distance profile candidate, and the real distance profile F (i) (i =
1. ． ． 360) and the virtual distance profile G for each predetermined angle for the three-dimensional shape model data of the virtual optical system.
_{j, k} (i) (j = 1 to 20, i, k = 1 to 360) is obtained (S301). As a matching method, the processing unit 11 compares the respective elements of the virtual distance profile that have been obtained as follows.

The processing section 11 sets S = 0 as an initial value (S302), determines i from 1 to 360, and determines the condition of the following equation (S303, S305, S309). | F (i) −G _{j, k} (i) | <100 The processing unit 11 counts up S when this condition is satisfied (S307). The processing unit 11 stores each S value in the memory 13 in association with the viewpoint number (S31
1), this is obtained for each element j, k (S31
3). The processing unit 11 orders the similarities according to the value of S obtained here. That is, if S is large, it means that the degree of matching is large.

In this way, the processing unit 11 selects a candidate and stores the data including the viewpoint number and the rotation angle in the memory 13 for the selected upper candidate. In this example, the processing unit 1
1 selects the top 20 candidates from 7,200 ways. Here, as an example, 100 is used as the comparison threshold value (this is always 1% of the average 10000 of the distance profile), but the present invention is not limited to this, and an appropriate value can be determined. In this way, the processing unit 11 determines the rotation angle of the camera 2.

(S211) Silhouette Matching Next, the processing section 11 performs silhouette matching processing on the candidates (here, 20) of the upper virtual distance profile obtained in step S209, and finally ranks them. . FIG. 15 shows a flowchart of silhouette matching. First, the processing unit 11 reads out the candidates for the virtual distance profile, the rotation angle, and the actual distance profile from the memory 13, and calculates the shapes of the virtual image and the actual image, that is,
Creating a virtual silhouette and a real silhouette (S35)
1). In the method of determining the position of the camera 2, first, the position of the virtual camera that has created the virtual distance profile is known, so this position information is used. However, since the actual shooting positions of the virtual camera and the real camera do not necessarily match, the processing unit 11 performs processing (normal processing) for bringing the position of the virtual camera closer to the actual shooting position (S353). What is needed here is the average value of the virtual distance profile of the unnormalized virtual camera image and the average value of the actual distance profile of the input image from the real camera. The processing unit 11 can estimate the distance between the actual object and the viewpoint by performing a process of bringing the average value of the virtual distance profile closer (matching) to the average value of the actual distance profile. The processing unit 11 calculates the scale-up or scale-down magnification of the silhouette used for the process as follows. Magnification = (average value of the actual distance profile of the input image) /
(Average value of virtual distance profile of virtual image)

Next, the processing unit 11 performs a rotation process on the z axis by the shift amount corresponding to the rotation angle obtained in step S209 for the rotation of the silhouette in the z axis direction (S).
355). At this time, the silhouette matching is performed by XOR between the rendered virtual image and the input real image.
(Exclusive OR) is taken, the pixels that do not match are counted, and the shift amount with a small number of pixels is determined as an appropriate value (S357).

In this way, for example, one (or a small number)
The camera position and the rotation angle are determined as candidates for the. The processing unit 11 determines the camera position, rotation angle, and
Data including the viewpoint number is stored in the memory 13 (S35).
9). (S212) Silhouette matching for camera parameters

A virtual camera is a 320-faced body that surrounds an object.
Since it is placed in the center of each side of the
The position of LA does not coincide with its center. So next
In addition, the processing unit 11 makes the best match in step S211.
Is used as a candidate for the camera position and rotation angle.
Data in detail. Here, 6 control parameters and
Then, the three movement amounts (vector) T_x, T _y, T_zAnd 3
Rotation angle R_x, R_y, R_zDetermine. For that purpose,
The processing unit 11 uses, for example, the third-order polygon as follows.
Move / rotate the original shape model data with respect to the XYZ axes
However, it repeats until it converges using the simplex method.
You

FIG. 16 shows a flowchart of silhouette matching for camera parameters. (2-1) The initial values of the movement amount and rotation amount, which are the six parameters of the camera, and the initial pixel number d are set (S32).
1). (2-2) moves, a rotational displacement amount _u t, moves the viewpoint in accordance with _{u r} (S323). (2-3) Rendering is performed to obtain a silhouette image (S325). (2-4) XOR is performed and the number D of pixels that do not match is counted (S327). (2-5) When D is smaller than the previous value, the displacement amount at that time is recorded (S329, S331). (2-6) Until (D) becomes the smallest, the above (2-
2) to (2-5) are repeated (S333). (2-7) The values of the displacement amounts u _t and u _r are halved to obtain u _t ,
The value of u _r returns to until sufficiently small (2-2), repeating the above operations (S335, S337). (2-8) As described above, the processing unit 11 determines the six camera parameters, estimates the detailed shooting position, and stores the data including the viewpoint and the camera parameter relating to the camera position in the memory 1.
3 is stored (S339).

(S215) Texture Mapping Next, a method of mapping a plurality of photographed two-dimensional images as textures on shape data will be described. First, in the texture mapping, the processing unit 11 performs the projection conversion from three-dimensional to two-dimensional for each triangular polygon forming the shape data. After matching the obtained image, the processing unit 11 first determines the visibility condition of each polygon again by using the camera parameter, based on the angle condition and the hidden condition already described. Processing unit 1
In No. 1, the area in which the end points of the visible polygon are mapped on the captured image becomes a triangular polygon, and this is cut out as a texture. This mapping uses a projection transformation from three-dimensional coordinates to two-dimensional coordinates (see Shigeo Ishii, "Introduction to 3D graphics in C language", Technical Review Company, 1990). The cut texture is mapped onto a three-dimensional polygon by projection inverse transformation. The processing unit 11 stores, in the output data file of the memory 13, data in which texture is attached to polygon data representing a three-dimensional object.

At this time, there are cases where two textures are given to one polygon (one surface is visible from two viewpoints), and in that case, the processing unit 11 performs the following blending processing. FIG. 17 shows an explanatory diagram of the blending process. In the 2 blending process, the polygon P is projected onto the _two textures Q ₁ and Q ₂ , and the projected area is A ₁ ,
If it is A ₂ , the weight W _{1 of} each texture,
W ₂ is given by the following formula.

[0046]

[Equation 2]

The blending process is the CG language OpenG.
L glblend-func. These weights become the mixture ratio alpha value of this function. If there are three or more, three may be blended, or two with a small angle between the polygon normal and the line of sight may be considered.

4. Practical Example Below, a practical example of the texture mapping process using this embodiment will be shown. FIG. 18 shows three-dimensional shape data (dinosaur).
FIG. The figure (a) is original data, and the figure (b) is the data which reduced the number of polygons.

FIG. 19 is a diagram of a silhouette image created from a photographed image of an actual example (dinosaur). By executing the compression process in this manner, a higher speed process can be achieved. FIG. 20 shows a diagram of a distance profile created from the silhouette image of FIG. 21 and FIG.
The figure of the histogram created from the distance profile of is shown. Here, a 10-step histogram is used.

FIG. 22 is a diagram of a silhouette image created from another photographed image of an actual example (dinosaur). In FIG.
The figure of the distance profile created from two silhouette images is shown. FIG. 24 shows a histogram created from the distance profile of FIG.

FIG. 25 shows a diagram of the texture mapping result. FIG. 10A is a texture-mapped image of one image, FIG. 9B is a texture-mapped image of two images, and FIG. 9C is a texture-mapped image of 12 images. In this example, the pretreatment was performed in less than 60 seconds. The 10-level histogram matching was 0.01 seconds or less, and the next distance profile matching was 0.8 seconds or less. Furthermore, the next silhouette matching was about 3 seconds, and the last camera parameter calculation was about 10 to 15 seconds. Further, the texture synthesis and display are less than 1 second, and the entire texture mapping processing can be realized in about 70 seconds. In this way, high-speed processing / real-time processing is realized by using an ordinary personal computer, and the processing time can be reduced to less than half by using a high-speed PC.

The texture mapping method or texture mapping device / system of the present invention is a texture mapping processing program for causing a computer to execute each procedure, a computer-readable recording medium recording the texture mapping processing program, and a texture mapping processing program. And a program product that can be loaded into the internal memory of a computer, a computer such as a server that includes the program, and the like.

As described above, in the image mapping process, even a workstation, which is expensive by the conventional method, can be used.
According to the present invention, the time required from one minute to one hour can be reduced to 20 seconds or less with an inexpensive personal computer, for example, one tenth or more and three to five minutes per sheet.

[0054]

As described above, the present invention is capable of performing high-speed processing (for example, about 20 seconds or less per image), and has sufficiently established practicality, a texture mapping method, a texture mapping processing program and the same. A computer-readable recording medium having a program recorded therein can be provided.

Further, according to the present invention, the texture image can be accurately mapped on the surface of the object based on the contour of the object given only the shape by photographing the real object with the digital camera from multiple directions, Further, therefore, it is possible to provide a multi-step search method for accurately and rapidly determining the mutual positional relationship between the object and the camera.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a texture mapping device.

FIG. 2 is a block configuration diagram of a texture mapping device.

FIG. 3 is an explanatory diagram of an input / output data file.

FIG. 4 is an explanatory diagram of a distance profile and a distance histogram data file.

FIG. 5 is an explanatory diagram of a real optical system and a virtual optical system.

FIG. 6 is a flowchart of a texture mapping method.

FIG. 7 is an explanatory diagram of 320 shooting points arranged around an object.

FIG. 8 is a flowchart for creating a distance profile and a distance histogram.

FIG. 9 is an example of a silhouette having a simple contour line.

10 is a distance profile created from the contour line of FIG. 9. FIG.

FIG. 11 is an example of a silhouette having a complicated contour line.

FIG. 12 is a distance profile created from the contour line of FIG.

FIG. 13 is an explanatory diagram of normalization of a distance profile.

FIG. 14 is a flowchart of distance profile matching.

FIG. 15 is a flowchart of silhouette matching.

FIG. 16 is a flowchart of silhouette matching for camera parameters.

FIG. 17 is an explanatory diagram of blending processing.

FIG. 18 is an explanatory diagram of three-dimensional shape data (dinosaur).

FIG. 19 is a diagram of a silhouette image created from a captured image of an actual example (dinosaur).

20 is a diagram of a distance profile created from the silhouette image of FIG.

21 is a diagram of a histogram created from the distance profile of FIG.

FIG. 22 is a diagram of a silhouette image created from another captured image of the actual example (dinosaur).

23 is a diagram of a distance profile created from the silhouette image of FIG. 22.

FIG. 24 is a diagram of a histogram created from the distance profile of FIG. 23.

FIG. 25 is a diagram of a texture mapping result.

[Explanation of symbols]

1 PC 2 camera 11 Processing unit 12 I / F 13 memory 14 Input section 15 Output section

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 15/00 300 G06T 15/00 300 // G01B 11/24 G01B 11/24 K F term (reference) 2F065 AA01 AA07 AA12 AA39 AA53 BB05 DD06 FF04 FF61 JJ03 JJ26 QQ24 QQ25 QQ31 QQ39 QQ42 QQ43 RR09 5B050 AA09 BA08 BA09 BA13 BA18 DA02 DA07 DA10 EA07 EA08 EA12 EA28 DC52 DC16 CD16 DB16 CD16 DB13 CD16 CD14 CD13 CD14 CD14 CD13 CD14 CD14 CD14 CD13 CD14 AA09 BA20 CA02 DA01 EA13 FA06 FA35 FA66 FA67 FA69

Claims (9)

[Claims] 1. A processing unit, based on three-dimensional shape model data representing a three-dimensional object, two-dimensional for each viewpoint when the three-dimensional object is photographed from a central viewpoint of each surface of a polyhedron enclosing the three-dimensional object. The step of acquiring virtual image data, and the processing unit creates a virtual distance profile representing the distance from the acquired virtual image data for each viewpoint to the edge for each predetermined angle with the center or center of gravity of the image as a reference. Step, the processing unit creates a virtual distance histogram at a predetermined stage for the virtual image data based on the created virtual distance profile for each viewpoint, and the processing unit uses the real camera to generate the three-dimensional shape model data. The step of inputting a real image obtained by shooting a real object corresponding to, the processing unit determines, from the obtained real image, the center or the center of gravity of the image as a reference. A step of creating a real distance profile that represents the distance to the edge for each angle; a processing unit that creates a real distance histogram at a predetermined stage for the real image data based on the created real distance profile; Compares a virtual distance histogram for each viewpoint with an actual distance histogram, extracts a predetermined number of highly matched virtual distance histogram candidates, and stores the data including the camera position related to the candidates in a distance histogram. The matching step and the processing unit read a virtual distance profile corresponding to the extracted virtual distance histogram from the memory, compare the virtual distance profile with the actual distance profile, and extract a matching virtual distance profile candidate. Data including the camera rotation angle for the candidate is stored in memory The distance profile matching step of remembering, the processing unit reads the extracted candidates from the memory, and selects a matching candidate based on the virtual silhouette image and the actual silhouette image that represent the shapes of the virtual image and the actual image. A silhouette image matching step for determining a position and a rotation angle of a camera. 2. The distance profile matching step, wherein the processing unit creates a plurality of virtual distance profiles obtained by shifting the virtual distance profile for each viewpoint by a predetermined angle, and the processing unit includes a plurality of virtual distance profiles. The texture mapping method according to claim 1, further comprising the step of comparing elements of the distance profile and the actual distance profile to perform matching to extract a predetermined number of candidates. 3. The silhouette image matching step, the processing unit creates a virtual silhouette image and a real silhouette image representing the shapes of the virtual image and the real image from the virtual distance profile and the real distance profile; and the processing unit, The step of normalizing by multiplying the virtual silhouette image or the real silhouette image by a ratio according to the ratio of the average value of the virtual distance profile and the average value of the real distance profile; The step of performing rotation processing with a shift amount corresponding to the camera rotation angle obtained in the distance profile matching step, and the processing unit selects a candidate having a high degree of matching between the virtual silhouette image and the real silhouette image, and the camera at that time is selected. Determining the position and the angle of rotation.
The texture mapping method described in. 4. The silhouette image matching step, the processing section moving the viewpoint corresponding to the camera position in the corresponding polyhedron according to the moving displacement amount and the rotational displacement amount of the camera, and the processing unit is moved. 4. The method according to claim 1, further comprising a step of determining a camera parameter by comparing a virtual image and a real silhouette image at a position to obtain a moving amount and a displacement amount having a high degree of coincidence and storing the camera parameter in a memory. The described texture mapping method. 5. The processing unit further comprises a mapping step of reading the camera position and the rotation angle from the memory and mapping the photographed real image as texture on the three-dimensional shape model data. The texture mapping method described in. 6. The texture mapping method according to claim 1, further comprising a step of normalizing the created virtual distance profile so that an average value of the virtual distance profile becomes a predetermined value. . 7. The processing unit, when creating the distance histogram, includes the distance data of a predetermined level or more in the element at the highest stage or discards the distance data. The texture mapping method described in. 8. A processing unit, based on three-dimensional shape model data representing a three-dimensional object, two-dimensional for each viewpoint when the three-dimensional object is photographed from the central viewpoint of each surface of a polyhedron enclosing the three-dimensional object. The step of acquiring virtual image data, and the processing unit creates a virtual distance profile representing the distance from the acquired virtual image data for each viewpoint to the edge for each predetermined angle with the center or center of gravity of the image as a reference. Step, the processing unit creates a virtual distance histogram at a predetermined stage for the virtual image data based on the created virtual distance profile for each viewpoint, and the processing unit uses the real camera to generate the three-dimensional shape model data. The step of inputting a real image obtained by shooting a real object corresponding to, the processing unit determines, from the obtained real image, the center or the center of gravity of the image as a reference. A step of creating a real distance profile that represents the distance to the edge for each angle; a processing unit that creates a real distance histogram at a predetermined stage for the real image data based on the created real distance profile; Compares a virtual distance histogram for each viewpoint with an actual distance histogram, extracts a predetermined number of highly matched virtual distance histogram candidates, and stores the data including the camera position related to the candidates in a distance histogram. The matching step and the processing unit read a virtual distance profile corresponding to the extracted virtual distance histogram from the memory, compare the virtual distance profile with the actual distance profile, and extract a matching virtual distance profile candidate. Data including the camera rotation angle for the candidate is stored in memory The distance profile matching step of remembering, the processing unit reads the extracted candidates from the memory, and selects a matching candidate based on the virtual silhouette image and the actual silhouette image that represent the shapes of the virtual image and the actual image. A texture mapping processing program for causing a computer to execute a silhouette image matching step of determining a position and a rotation angle of a camera. 9. A processing unit, based on three-dimensional shape model data representing a three-dimensional object, two-dimensional for each viewpoint when the three-dimensional object is photographed from the central viewpoint of each surface of a polyhedron enclosing the three-dimensional object. The step of acquiring virtual image data, and the processing unit creates a virtual distance profile representing the distance from the acquired virtual image data for each viewpoint to the edge for each predetermined angle with the center or center of gravity of the image as a reference. Step, the processing unit creates a virtual distance histogram at a predetermined stage for the virtual image data based on the created virtual distance profile for each viewpoint, and the processing unit uses the real camera to generate the three-dimensional shape model data. The step of inputting a real image obtained by shooting a real object corresponding to, the processing unit determines, from the obtained real image, the center or the center of gravity of the image as a reference. A step of creating a real distance profile that represents the distance to the edge for each angle; a processing unit that creates a real distance histogram at a predetermined stage for the real image data based on the created real distance profile; Compares a virtual distance histogram for each viewpoint with an actual distance histogram, extracts a predetermined number of highly matched virtual distance histogram candidates, and stores the data including the camera position related to the candidates in a distance histogram. The matching step and the processing unit read a virtual distance profile corresponding to the extracted virtual distance histogram from the memory, compare the virtual distance profile with the actual distance profile, and extract a matching virtual distance profile candidate. Data including the camera rotation angle for the candidate is stored in memory The distance profile matching step of remembering, the processing unit reads the extracted candidates from the memory, and selects a matching candidate based on the virtual silhouette image and the actual silhouette image that represent the shapes of the virtual image and the actual image. A computer-readable recording medium recording a texture mapping processing program for causing a computer to execute a silhouette image matching step of determining a position and a rotation angle of a camera.