JP3672739B2 - Texture image generator - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a texture image generation apparatus. More particularly, the present invention relates to a texture image generation apparatus capable of automatically generating a texture image with high quality and a small storage capacity in an arbitrary viewpoint image display method using a three-dimensional shape restoration process based on feature point correspondence.
[0002]
[Prior art]
Due to the recent high performance of computer systems and the development of multimedia processing technology, an environment in which advanced three-dimensional computer graphics (hereinafter abbreviated as CG) can be processed even in personal computers and the like is being prepared. In constructing systems that realistically reproduce objects on computer screens, such as electronic shopping and electronic museums that use 3D CG technology, it is important how efficiently the existing objects are captured as 3D image data on the computer. It is a difficult issue.
[0003]
The simplest method for generating data for capturing a three-dimensional object into a computer is to generate and prepare two-dimensional image data obtained by photographing the three-dimensional object from several viewpoints. The two-dimensional image taken from the closest viewpoint is displayed on the display. However, according to this method, there is a problem that an image of a three-dimensional object cannot be displayed from a viewpoint other than the photographed viewpoint. Moreover, since it is a two-dimensional image to the last, there also existed a problem that image synthesis with the other three-dimensional object produced | generated as three-dimensional CG data was difficult to perform. Therefore, in order to display the captured 3D object from any viewpoint, the 3D shape of the object is restored as 3D image data based on the 2D image data obtained by photographing the object from a plurality of viewpoints. It is necessary to generate and display a three-dimensional projection image for the viewpoint.
[0004]
Here, some methods have been proposed as a method of restoring the three-dimensional shape of an object as three-dimensional image data from the photographed two-dimensional image data. As a typical method, there is a method based on feature points. The three-dimensional image data generation process by this method includes the following processing steps as shown in the flowchart of FIG. FIG. 20 shows a concept of generating three-dimensional image data based on conventional feature points.
[0005]
First, a captured image is read and feature points are extracted (step S1901). In extracting the feature points, it is preferable to select a location where the brightness changes in the photographed two-dimensional image as a feature point. This extraction processing may be executed by automatically extracting feature points by a computer by examining changes in brightness, or may be executed by manually specifying the feature points. This is shown in FIG. In this example, the four feature points shown in FIG. 20B are extracted from the captured image of FIG.
[0006]
Next, the extracted feature points are associated with each other between the captured images (step S1902). The association between the feature points can be automatically processed by a computer by using the photographing positional relationship of the photographed images, the similarity of the images, and the positional relationship between the extracted feature points. Moreover, you may perform by specifying manually. In the example of FIG. 20, each feature point 1 to feature point 4 is associated as shown in FIG.
[0007]
Next, the three-dimensional coordinates of each feature point are obtained from the correspondence between the feature points by the principle of stereo measurement (step S1903). By this processing, each feature point is expressed as one point in the three-dimensional space.
[0008]
Next, a face is assigned to each feature point expressed in the three-dimensional space to form a polyhedron (step S1904). Here, the combination of the feature points forming the surface is not an arbitrary combination, but must be selected so as to form the outer surface of the reproduced object. In the example of FIG. 20, a face is assigned as shown in FIG. 20D, and a tetrahedron is formed.
[0009]
Next, a texture for each surface is generated (step S1905). When generating the texture, the texture to be applied to each surface can be calculated by referring to the pattern of the photographed image.
[0010]
Through the processing shown in steps S1901 to S1905, three-dimensional image data can be generated from the captured image. In the example of FIG. 20, the three-dimensional image data of the original three-dimensional object can be generated as shown in FIG.
[0011]
On the other hand, the reproduction process on the computer screen of the three-dimensional object using the generated three-dimensional image data includes the following processing steps as shown in the flowchart of FIG.
[0012]
First, the position of the viewpoint with respect to the user's object is designated. That is, the direction and distance of the viewpoint with respect to the object are designated (step S2101).
Next, the position on the projection plane of each feature point with respect to the projection plane according to the direction and distance of the viewpoint with respect to the designated object is calculated to obtain a projection image (step S2102). At this stage, the texture mapping process has not yet been performed.
[0013]
Next, a texture is pasted on each surface of the projected image (step S2103). Here, when pasting the texture, the size, shape, and direction of each projected surface are calculated and the texture is adjusted and pasted.
[0014]
Finally, after performing necessary special effect processing such as lighting, tracing, and shading, display on the computer screen is performed (step S2104).
[0015]
Through the processing shown in steps S2101 to S2104, an image at an arbitrary viewpoint of the three-dimensional object can be displayed on the computer screen using the generated three-dimensional image data.
[0016]
[Problems to be solved by the invention]
An object to be captured as a three-dimensional image has many complicated shapes and patterns, and high-definition data is required to realize a better virtual reality. In this case, since the required data capacity becomes enormous, reduction of the storage capacity of the three-dimensional image data becomes an important issue. In addition, 3D image playback must be performed at a location that is separated in time and space via a recording medium or a network. To reduce line costs and transfer time, further increase the data capacity. Reduction is required.
[0017]
Here, since the feature points and their connection relations are expressed by a data set of XYZ values and corresponding numbers of feature points, the data amount related to the feature points is usually sufficiently smaller than the data amount related to the texture. Therefore, the reduction of the data amount of the three-dimensional image data is an important issue in how to reduce the data amount related to the texture.
[0018]
The two conventional texture image generation processing methods listed below have problems from the viewpoint of reducing the data amount.
As the first texture image generation processing method, when the 3D image data is generated, the captured image data is stored as it is without being restored as a 3D object in which the texture data is pasted into the 3D shape. There is a method of generating a texture to be attached to each surface in accordance with the viewpoint designated at the time of display processing. According to this method, texture data may be generated dynamically using image data taken from a viewpoint close to the designated viewpoint, but image data taken from any viewpoint is required, and is necessary for texture processing. There arises a problem that the image data becomes a huge capacity.
[0019]
As a second texture image generation processing method, there is a method of storing a texture to be attached to each surface as an image (texture image) as it is. As shown in FIG. 22, each surface forming a polyhedron is separated and placed on a plane, and a pattern of a captured image is drawn and stored thereon. However, according to this texture processing method, there is a problem that a lot of useless portions are included in the data held as texture-related data, and an extra storage capacity is required. That is, generally, when a polyhedron is developed on a plane, a lot of useless gaps are generated between the surfaces as shown in FIG. It is not easy to arrange the faces so as to fill in the gaps, and the gaps that are generated increase as the total number of faces is increased in order to improve the image quality, resulting in a problem that the useless storage capacity further increases.
[0020]
Here, in order to fill the gap, it is possible to appropriately deform the shape of the surface, but since the image is expressed as two-dimensional array data in units of pixels, the shape of the surface is distorted from the original shape. As a result, the influence of the quantization error in units of pixels increases, resulting in a problem that the quality of the reproduced image is degraded.
[0021]
In view of the problems of the conventional texture image generation processing method, the present invention provides texture image data used for image display processing at an arbitrary viewpoint using a three-dimensional object restoration process based on feature point correspondence, and has high quality and storage. An object of the present invention is to provide a texture image generation apparatus that automatically generates texture image data with a small capacity.
[0022]
[Means for Solving the Problems]
In order to solve the above problems, a texture image generation apparatus of the present invention includes an input unit, a projection unit, a division unit, a development unit, and an image writing unit, and the input unit is a feature point of a three-dimensional object. The projection unit receives the feature point of the inputted three-dimensional object as a known shape surface that is a developable surface and includes the feature point. Projecting onto a projection surface to obtain a projection point, the dividing unit divides the projection surface into small surfaces based on the projection points of the feature points, and the developing unit expands the projection surface into a plane to form the small surface The developed image plane is generated, and the image writing unit writes a captured image of the outer surface of the three-dimensional object in which the corresponding feature point is captured for each small surface on the expanded plane. To produce And butterflies.
[0023]
With this configuration, the feature points of the three-dimensional object can be projected onto a projection surface of a known shape surface, which is a developable surface, and developed on a plane, and a texture image that is efficient by writing captured image data on the development plane Can be generated.
[0024]
Next, the projection unit includes a projection plane adjustment unit, and the projection plane adjustment unit adjusts the type and size of the projection plane generated by the projection unit to the shape of the three-dimensional object formed by the feature points. It is preferable to minimize distortion of the shape of the projection plane and the shape of the three-dimensional object formed by the feature points.
[0025]
With this configuration, since the type and size of the projection plane can be matched to the shape of the three-dimensional object formed by the feature points, the projection plane formed on the projection plane generated by the projection and the original three-dimensional object are excluded. The distortion with the surface can be minimized, and the quality of the texture image can be improved.
[0026]
Next, the projection unit includes a projection center point appropriateness confirmation unit, and by the projection center point appropriateness confirmation unit, a captured image of the outer surface of the three-dimensional object on each facet on the development plane by the image writing unit It is preferable to be able to check whether or not the correspondence between each facet on the development plane and the photographed image obtained by photographing the corresponding feature point is correctly obtained.
[0027]
With this configuration, when projecting feature points of a three-dimensional object onto a known shape surface, it can be ensured that each surface of the three-dimensional object does not overlap on the formed projection surface. Can be deployed.
[0028]
Note that the center of gravity of the three-dimensional object, a predetermined point such as the origin, etc. can be adopted as the projection center point, and if the appropriateness of the projection center point cannot be confirmed by the projection center point appropriateness confirmation unit, the projection center point Redo the selection.
[0029]
Next, it is preferable that the projection unit includes a projection center point designating unit for inputting designation of a projection center point, and the projection center point is determined based on designation from the projection center point designating unit.
[0030]
With this configuration, the apparatus user can designate the projection center point, and the processing of the texture image generation apparatus can be tuned according to the situation.
Next, feature point group information that divides the feature points of the three-dimensional object into groups is attached, and the projection unit performs a projection process on each group of feature points according to the feature point group information. preferable.
[0031]
With this configuration, when the shape of the three-dimensional object to be restored is complex, it is possible to perform projection processing separately on object portions that can be easily processed, and to improve the quality of the texture image. .
[0032]
Next, when there is a transboundary surface projected across the developable surface and the plane on the projection surface among the projection forming surfaces formed by the projection processing of the characteristic points, the transboundary surface A projecting point of the feature point on the transboundary surface, comprising a developable surface projection adjustment unit that adjusts the developable surface so that all of the feature points are projected onto the developable surface when projecting the feature points forming Among these, the projection point on the developable surface is defined as a developable boundary projection point, the projection point on the plane is defined as a plane boundary projection point, and the developable surface projection adjustment unit is configured to A projection line that passes through a plane boundary projection point is extended to obtain a point that intersects with an extension surface of the developable surface as a developable surface adjustment projection point, and the dividing unit calculates a boundary edge line of the developable surface as the developable surface boundary. A line connecting the projection point and the developable surface adjustment projection point in order, and the boundary ridge line of the plane is the plane boundary It is preferable to divide the friendly exhibition surface and a plane as a line connecting the projected points in order.
[0033]
With this configuration, each surface of the three-dimensional object projected onto the known shape surface is not formed across the developable surface and the plane, and the boundary ridge line between the developable surface and the plane is the boundary ridge line between the projection forming surfaces. And a texture image with less distortion can be obtained.
[0034]
Next, when there is a transboundary surface projected across the developable surface and the plane on the projection surface among the projection forming surfaces formed by the projection processing of the characteristic points, the transboundary surface A plane projection adjustment unit that adjusts the plane so that all of the feature points are projected onto the plane when projecting the feature points forming the feature points. A projection point on the surface is defined as a developable boundary projection point, a projection point on the plane is defined as a plane boundary projection point, and the plane projection adjustment unit passes through the developable boundary projection point from the projection center point. A point that extends the projection line and intersects the extended surface of the plane is obtained as a plane adjustment projection point, and the dividing unit sets a boundary ridge line of the plane as a line that sequentially connects the plane boundary projection point and the plane adjustment projection point. , The boundary ridge line of the developable surface in order of the developable surface boundary projection point It is preferable to divide the friendly exhibition surface and a plane as a line connecting.
[0035]
With this configuration, each surface of the three-dimensional object projected onto the known shape surface is not formed across the developable surface and the plane, and the boundary ridge line between the developable surface and the plane is the boundary ridge line between the projection forming surfaces. And a texture image with less distortion can be obtained.
[0036]
Next, the projection unit projects the boundary between the first and second developable surfaces on the projection surface among the projection forming surfaces formed by the projection processing of the feature points. When the feature points that form the transboundary surface are projected, the developable surface projection adjustment unit that adjusts the developable surface so that all of the feature points are projected onto the first developable surface. Among the projection points of the feature points of the transboundary surface, a projection point on the first developable surface is defined as a first developable surface boundary projection point, and a projection point on the second developable surface The second developable surface boundary projection point, and the developable surface projection adjustment unit extends the projection line passing through the second developable surface boundary projection point from the projection center point to the first developable surface boundary projection point. A point that intersects the extended surface of the surface is obtained as a developable surface adjustment projection point, and the dividing unit determines a boundary ridge line of the first developable surface as the first developable surface boundary projection point. The first developable surface and the second as the line connecting the developable surface adjustment projection points in order, and the boundary ridge line of the second developable surface as the line connecting the second developable surface boundary projection points in order. It is preferable to divide the developable surface.
[0037]
With this configuration, each surface of the three-dimensional object projected on the known shape surface is not formed between the two developable surfaces, and the boundary between the first and second developable surfaces. A ridge line can be used as a boundary ridge line between projection forming surfaces, and a texture image with less distortion can be obtained.
[0038]
Next, it is preferable that the developing unit cuts and develops along a ridge line connecting the projection points of the feature points in the process of developing the developable surface into a plane.
With this configuration, in the process of cutting and developing the developable surface, the surface formed by the projected feature points is not cut from the middle, and it is possible to cut with the ridgeline between the projection forming surfaces, and distortion A few texture images can be obtained.
[0039]
Next, in the dividing process of the projection plane, the dividing unit generates a convex hull having the projection point of the feature point as a vertex, and a small surface obtained by dividing the projection surface with the small surface forming the convex hull. Approximate handling is preferred.
[0040]
With this configuration, a convex hull having the projection point of the feature point as an apex can be approximated to the projection plane and developed on a plane, and a texture image with low waste of data capacity can be generated efficiently.
[0041]
Next, in the process of writing the photographed image of the three-dimensional object on the development plane, the image writing unit captures the largest facet area to be written as the photographed image data of each facet to be written. It is preferable to use image data.
[0042]
With this configuration, it is possible to generate a texture image closest to the texture of the actual outer surface of the three-dimensional object.
Next, in the process of writing the photographed image of the three-dimensional object on the development plane, the image writing unit captures the facet to be written from the closest viewpoint as data of the photographed image of each facet to be written. Is preferably used.
[0043]
With this configuration, it is possible to generate a texture image closest to the texture of the outer surface of the three-dimensional object when viewed from a specified viewpoint.
Next, in the process of writing the captured image of the three-dimensional object to the development plane, the image writing unit sets the shape of the captured image data of each facet to be written to the shape of the corresponding facet on the development plane. It is preferable to write by affine transformation so as to match.
[0044]
With this configuration, it is possible to adjust the distortion of the shape of the actual three-dimensional object surface and its projection forming surface in the projection processing to a known shape consisting of a developable surface and a flat surface, and the distortion of the generated texture image It can be corrected.
[0045]
Next, a computer-readable recording medium in which a processing program for realizing the texture image generating apparatus of the present invention is recorded is an input processing step for receiving input of feature point data of the three-dimensional object and photographed image data of the outer surface, Projecting the input feature point of the three-dimensional object onto a projection plane that is a known shape surface that is a developable surface and includes the feature point, and projecting by the projection processing step A dividing process step of dividing the projected plane into small planes according to the projection points of the feature points, a developing process step of developing the projected plane into a plane to generate a developed plane composed of the small planes, An image writing processing step for writing a photographed image of the outer surface of the three-dimensional object on which the corresponding feature point is photographed for each facet; For example, it characterized by recording a processing program and generates a texture image.
[0046]
By causing the computer to read this processing program, the feature point of the three-dimensional object can be projected onto a projection surface of a known shape surface, which is a developable surface, and developed on a plane using the computer. A texture image generation apparatus that generates an efficient texture image by writing photographed image data to can be configured.
[0047]
DETAILED DESCRIPTION OF THE INVENTION
The texture image generation apparatus of the present invention will be described with reference to the drawings.
First, the basic concept of the processing method of the texture image generation apparatus of the present invention will be described below.
[0048]
As described in the description of the prior art, the texture image is obtained by drawing texture data obtained by photographing the outer surface of the three-dimensional object on the development surface of the outer surface of the three-dimensional object. The smaller the gap between the planes, the smaller the data capacity. If the shape of the 3D object is already known, it is possible to optimize the image data generation process in advance so that the space between the expanded planes can be reduced by examining the method of expanding the 3D object onto the plane in advance. It becomes.
[0049]
For example, assuming that the shape of the three-dimensional object is a cylindrical body, the development on the plane need only be separated into three parts: an upper surface, a side surface, and a lower surface, as shown in FIG. If a texture image is written on this development plane, it is possible to obtain a texture image with high quality and a small storage capacity. This property is attributed to the fact that the shape of the three-dimensional object, which is a cylindrical body to be developed, was a developable body. As described above, if the shape of the three-dimensional object is a known developable surface, the gap between the development planes developed by a predetermined procedure is reduced, so that a texture image with high quality and a small storage capacity can be obtained.
[0050]
Here, the developable surface generally refers to a curved surface that can be developed into a plane in a narrow sense. For example, the outer surface of the cylindrical body is a combination of a side surface that is a developable surface and an upper surface and a lower surface that are flat surfaces. The term “developable body” as used in the present invention refers to a material whose outer surface is a combination of developable surfaces or a combination of a developable surface and a flat surface and can be appropriately developed on a flat surface. That is, the boundary line between the surfaces by the development process is generated between the developable surfaces or between the developable surface and the plane, and does not occur between the planes. The “developable body surface” refers to a surface that forms a developable body, and refers to the outer surface of the developable body that becomes a projection surface in the projection processing described later.
[0051]
As an example of the developable body of the present invention, there are a columnar body, a cone body, and a trapezoidal body in addition to the above-described cylindrical body as shown in FIG. Here, the cylinder includes a cylinder, an elliptic cylinder, the cone includes a cone, an ellipse, and the like, and the frustum includes a cone and the like. In addition, examples of the developable body composed of a combination of developable surfaces include a combination of two cones and a sphere. Strictly speaking, the spherical surface is not a developable surface. However, if attention is paid to the one-to-one correspondence between the development plane and the spherical surface, the spherical surface can be developed into a rectangular plane by focusing on the latitude and longitude. However, since the shape on the projection surface is expected to be greatly distorted when expanded into a square, it is not always preferable to use a spherical surface as the projection surface.
[0052]
By the way, a three-dimensional object handled in image processing generally has a complicated shape and is not a developable body as described above. Therefore, the three-dimensional object is put in the known shape made of the above-described developable surface, and the known shape is used as the projection surface, and the three-dimensional object is projected onto the projection surface from the internal projection center point as shown in FIG. Consider handling the projection plane.
[0053]
In FIG. 3, reference numeral 301 denotes a projection surface, which is a cylindrical surface that is a combination of a developable surface and a flat surface. Reference numeral 302 denotes a projection center point, 303 denotes a three-dimensional object to be projected, and 304 denotes a focused surface of the three-dimensional object outer surface. In the projection processing, first, a half line is drawn from the projection center point 302 toward each feature point of the three-dimensional object 303. These half lines are extended as they are to the projection plane 301, and a projection point on the projection plane 301 is obtained. Each projection point for each feature point is obtained, and a surface 304 ′ projected onto the projection surface 301 for the surface 304 of the three-dimensional object is obtained. In this way, when all the feature points of the three-dimensional object 303 are projected onto the projection plane 301, projection points corresponding to the feature points of the original three-dimensional object to be developed can be obtained. Here, when the projection plane is considered as a set of planes (projection formation planes) having the projection points as vertices, the three-dimensional object is expressed as an object having the shape of the projection plane 301 that is a developable plane as a result of the projection process it can. The projection plane 301 can be developed on a plane as shown in FIG. 1, and the development plane does not have an extra space. If the texture image for the three-dimensional object is used, the texture image of the three-dimensional object can be of high quality and have a small storage capacity.
[0054]
The shape of a three-dimensional object is generally unknown, and each outer surface thereof needs to be estimated (or calculated) from a three-dimensional feature point together with a texture image. As described above, the three-dimensional object is expressed as an object having a projection point as a vertex as a result of the projection process. By regarding the set of feature points corresponding to the projection points as the outer surface of the three-dimensional object, the outer surface can be estimated. Thereby, the outer surface corresponding to the texture image is obtained.
[0055]
The basic flow of texture image generation in the texture image generation apparatus according to the present invention is summarized as follows. As shown in FIG. 4, three feature points of the three-dimensional object shape are obtained from the image data of the three-dimensional object (step S401). A representation with a three-dimensional point sequence (step S402), a representation with a projection point sequence on the projection surface by projection processing (step S403), a surface is formed based on the projection point sequence, and a projection formation surface on the projection surface (Step S404), and development processing is used to express on the projection plane on the development plane (step S405). Finally, the captured image is applied to the projection plane to generate a texture image (step S406).
[0056]
As described above, if a texture image generation device capable of generating a texture image using projection onto a projection surface having a known shape composed of a developable surface is realized, a texture image with high quality and low storage capacity can be generated. .
[0057]
A texture image generation apparatus according to the first embodiment that realizes the basic processing method of the present invention described above will be described below.
An outline of the overall configuration of the texture image generation apparatus according to the first embodiment and an overall image of the processing flow of the apparatus will be described with reference to the drawings. FIG. 5 shows a schematic configuration diagram of the present apparatus. FIG. 6 shows an example of the data structure handled by this apparatus.
[0058]
As shown in FIG. 5, the texture image generation apparatus according to the first embodiment is roughly divided into an input unit 10, a feature point extraction unit 20, a projection unit 30, a division unit 40, a development unit 50, an image writing unit 60, and an output unit 70. It has. Although not shown, it is assumed that a control unit, a memory, a monitor, and devices necessary for control processing of the entire system are provided.
[0059]
Hereinafter, the operation of each unit will be described in order.
Data input processing by the input unit 10 will be described. FIG. 6a shows an example of the data structure handled by the texture image generation apparatus of the present invention. The data 100 includes an image information area 101, a feature point information area 102, a projection information area 103, a projection plane information area 104, another information area 105, and a header 106.
[0060]
Each region in the structure of the data 100 is given data as shown in FIG. As shown in FIG. 6b, the input data includes captured image information as essential information, and includes feature point three-dimensional coordinate information, feature point group information, projection center information, and projection plane information as arbitrary information. Here, the feature point group information is information that divides a set of feature points into a plurality of small groups, and an appropriate projection plane and projection center point are individually selected for each small group of feature points to perform projection processing. This information is used when A method for performing projection processing by individually selecting an appropriate projection plane and projection center point for each small feature point group using this feature point group information will be described later as a second embodiment.
[0061]
Next, feature point extraction processing by the feature point extraction unit 20 will be described.
When the feature point three-dimensional coordinate information is given as the feature point information of the input data, the feature point extraction unit 20 takes the feature point three-dimensional coordinate information as a feature point and passes it to the projection unit 30. If the feature point 3D coordinate information is not given, a point where the brightness is changed is detected from the captured image information, and the point is automatically extracted as a feature point. The feature points of the object are calculated as a three-dimensional point sequence on the coordinate space.
[0062]
The feature point extraction unit 20 preferably includes a feature point designation unit 21 in view of the fact that it may be difficult to extract feature points due to changes in image brightness depending on the state of the captured image. It is preferable that the device user can specify the feature points as needed through the feature point specifying unit 21 and tune the feature point extraction process.
[0063]
Next, projection processing by the projection unit 30 will be described. As shown in FIG. 5, the projection unit 30 includes a feature point arrangement unit 31, a projection plane generation unit 32, a projection center point designation unit 33, and a projection processing unit 34.
[0064]
The feature point placement unit 31 places the feature point in the three-dimensional coordinate space based on the feature point data obtained by the feature point extraction process of the feature point extraction unit 20.
When projection plane information is given in the input data 100, the projection plane generation unit 32 selects a type of projection plane according to the designation, and when there is no projection plane information, the projection plane generation unit 32 uses a specific projection plane as a default. Here, a cylindrical surface is selected. A projection plane is generated so as to include all the feature points arranged by the feature point arrangement unit 31 with the selected type of projection plane.
[0065]
The projection center point designating unit 33 determines the projection center point in the projection process according to the projection center information when the projection center information is given in the input data 100, and defaults when the projection center information is not given. As a calculation of a specific projection center point, here, the center of gravity of the feature point sequence is calculated and designated.
[0066]
The projection processing unit 34 performs a feature point projection process. A projection line is drawn from the projection center point toward each feature point, the projection line is extended to the projection plane, and an intersection with the projection plane is set as a projection point of the feature point.
[0067]
Next, the dividing process by the dividing unit 40 will be described. The dividing unit 40 divides the projection surface projected by the projecting unit 30 based on the projection points of the feature points formed on the projection surface. In other words, a small surface having projection points at the vertices is formed on the projection surface, and the entire projection surface is filled with the formed small surfaces for division.
[0068]
The above dividing process of the projection plane by the dividing unit 40 can be approximately realized by obtaining a convex hull of a three-dimensional point sequence of projection points formed on the projection plane. Here, as shown in FIG. 7, the convex hull of a three-dimensional point sequence is a shape obtained by arranging a three-dimensional point sequence in a space and reducing a huge rubber balloon including all of them inside as much as possible. That is. For example, the convex hull of a sequence of points distributed on a convex surface such as a cylindrical surface can be approximated to a cylindrical surface with a certain number of points, and it can be said that the cylindrical surface is divided by polygonal surfaces. is there.
[0069]
Next, the expansion process by the expansion unit 50 will be described. The developing unit 50 develops the projection surface into a plane that has been optimized in advance. Here, since the projection surface is a developable surface, each can be developed without a gap. When the projection surface is a cylindrical surface, the projection surface is composed of a side surface that is a developable surface, an upper surface that is a flat surface, and a lower surface, and thus can be developed as shown in FIG. On the developed plane, the position of the projection point corresponding to each feature point of the three-dimensional object is determined.
[0070]
Next, the texture of the captured image is applied by the image writing unit 60 to generate a texture image. The surface fitting can be automatically performed by assigning the feature points on the input captured image and the corresponding projection points on the development plane so as to coincide with each other.
[0071]
The output unit 70 is a part that outputs the generated texture image. An example of output data is shown in FIG. Each area of the output data includes texture image information and feature point connection relation information as essential information. Arbitrary information includes projection center information, projection plane information, and feature point group information. As described in the input information, feature point group information is information that divides a set of feature points into a plurality of small groups, and selects an appropriate projection plane and projection center point for each small group of feature points. Information used when projection processing is performed. What uses this feature point group information will be described later as a second embodiment.
[0072]
As described above, a texture image generation apparatus characterized by being able to generate a texture image with high quality and a small storage capacity by the above system configuration and processing flow is provided.
[0073]
In the above example, the cylindrical surface is used as the projection surface. However, it goes without saying that the surface may be a developable surface such as another conical surface.
(Embodiment 2)
Similar to the first embodiment, the texture image generation apparatus according to the second embodiment projects and develops a known shape surface made of a developable surface as a projection surface, and automatically generates a texture image by pasting the texture of the photographed image on the development plane. However, the projection unit of the texture image generation apparatus according to the first embodiment includes a projection unit with higher performance.
[0074]
An outline of the overall configuration of the texture image generation apparatus according to the second embodiment and an outline of a processing flow by the apparatus will be described with reference to the drawings.
FIG. 8 is a schematic configuration diagram of the texture image generation apparatus according to the second embodiment. As shown in FIG. 8, the apparatus according to the second embodiment includes a projection unit 30a instead of the projection unit 30 in the configuration of the first embodiment. In addition to the configuration of the projection unit 30 of the first embodiment, the projection unit 30a further includes a projection plane adjustment unit 35 and a projection center point appropriateness confirmation unit 36. The components other than the projection plane adjustment unit 35 and the projection center point appropriateness confirmation unit 36 of the projection unit 30a are the same as the components described with the same numbers in the first embodiment, so the description here will be given. The description is omitted with the projection unit 30a as the center. As in the first embodiment, although not shown, the memory and devices necessary for the control processing of the entire system are provided.
[0075]
First, the operation of the projection plane adjustment unit 35 will be described below.
The projection plane adjustment unit 35 selects the type of projection plane used for projection processing and adjusts the projection plane shape and size. In general, a wide variety of shapes of three-dimensional objects handled by the texture image generation apparatus are expected. In the projection processing, due to the difference between the shape of the three-dimensional object and the shape of the projection surface, the shape of the projection formation surface formed as a result of the projection processing is slightly distorted from the shape of the actual surface of the three-dimensional object. In order to suppress such distortion, it is preferable to select a developable surface as close as possible to the three-dimensional object as the projection surface. The projection surface adjustment unit 35 selects an appropriate one close to the three-dimensional object shape as the type of projection surface used for the projection process, and further adjusts the size of the developable surface so as to be close to the three-dimensional object shape. As a selection criterion for the type of projection plane, for example, there is a method of calculating the sum of distances from feature points to the projection plane and selecting the type of projection plane with the smallest value.
[0076]
In addition, when the shape of the three-dimensional object is complicated, the projection plane adjustment unit 35 divides the three-dimensional object into several object parts, and selects an appropriate projection plane for the shape of each object part. Can be adjusted.
[0077]
The process of assigning a projection plane to each of the object parts will be described in detail below.
First, input data is input from the input unit 10 (step S901 in FIG. 9). Here, it is assumed that feature point group information is given as feature point information in the input data shown in FIG.
[0078]
The projection plane adjustment unit 35 groups the feature points according to the feature point group information and grasps them as each object part (step S902).
The projection plane adjustment unit 35 selects the type of projection plane for each object part grasped in step S902 (step S903). In selecting the type of projection plane, here, the sum of the distances from the feature points of the object portion to the projection plane is calculated, and the type of projection plane with the smallest value is selected.
[0079]
The projection plane adjustment unit 35 notifies the projection plane generation unit 32 of the type of projection plane, causes the projection plane generation process to be performed, and also controls the adjustment of the size of the projection plane (step S904).
[0080]
Note that a series of projection processing by the feature point arrangement unit 31, the projection plane generation unit 32, the projection center point designation unit 33, and the projection processing unit 34 for each object part is performed in the same manner as in the first embodiment (step S905). .
[0081]
As described above, the texture image generation apparatus according to the second embodiment can perform optimum projection processing for each object portion even when the shape of the target three-dimensional object is complicated, and has high quality and low storage capacity. A texture image can be generated.
[0082]
Next, the operation of the projection center point appropriateness confirmation unit 36 will be described.
The projection center point appropriateness confirmation unit 36 confirms whether or not the selection of the projection center point in the projection process has been appropriately performed.
[0083]
The appropriateness of the projection center point here will be described below. FIG. 10 is an example of an inappropriate projection center point. As shown in FIG. 10, since a part of the projection formation surfaces overlap on the projection surface such that the projection formation surfaces 1004 ′ and 1005 ′ of the surfaces 1004 and 1005 overlap, a normal development plane cannot be obtained. It has become. In this case, it is not possible to generate a texture image by the projection processing and development processing of the present invention. The fact that there is no overlapping of such projection forming surfaces is said to be appropriate for the projection center point.
[0084]
However, since the surfaces 1004 and 1005 in FIG. 10 are true outer surfaces of the three-dimensional object and this information is not given to the apparatus, it is not possible to directly determine the overlap of the projection forming surfaces. The projection center point appropriateness confirmation unit 36 indirectly uses the feature point arrangement (clockwise or counterclockwise) for the contradiction between the estimation result of the outer surface obtained by dividing the projection plane and the captured image of the three-dimensional object. To determine whether the projection center point is appropriate. Specifically, for all the outer surfaces obtained by the division, determine whether the feature points constituting the surface are aligned clockwise or counterclockwise when the surface is viewed from the outside, If the result and the arrangement of the feature points when each surface is viewed from the outside of the three-dimensional object on the photographed image match, it is determined that the projection center point is appropriate. The outer direction of the projection plane is used as the outer direction of the outer surface obtained by the division. As the outer direction of each surface on the photographed image, the photographing direction of the photographed image from which all feature points constituting the surface are extracted is used.
[0085]
An example in which a cylindrical surface is employed as the projection surface will be described. As described in the first embodiment, the projection plane is generated by the projection plane generation unit 32 so as to include the three-dimensional feature point sequence arranged by the feature point arrangement unit 31.
[0086]
First, the projection center point designating unit 33 designates a projection center point according to the reference shown in the first embodiment (step S1101). That is, it is specified in accordance with projection center data in the input data, calculation of the center of gravity of feature points, or specification input by the user.
[0087]
Next, as in the first embodiment, the projection processing unit 34 obtains a projection point on the projection plane by drawing a projection line for each feature point from the designated projection center point (step S1102), and sets the projection point as a vertex. The cylindrical surface that is the projection surface is divided without gaps and is filled with polygonal surfaces (step S1103). Next, the developing unit 50 develops the cylindrical surface that is the projection surface into a flat surface (step S1104). Here, as shown in FIG. 1, the cylindrical surface that is the projection surface is developed into a side surface that is a developable surface and an upper surface and a lower surface that are planes.
[0088]
Here, the projection center point appropriateness confirmation unit 36 applies the captured image of each surface of the three-dimensional object based on the relationship between the feature points by the image writing unit 60 and monitors the writing process (step S1105). Here, if the arrangement of the feature points obtained in step S1104 matches the arrangement of the feature points on the captured image of each surface of the three-dimensional object, the projection center point has been appropriately selected. Can be confirmed (step S1106). Thereafter, as long as the same three-dimensional object and projection center point are used, normal projection processing is guaranteed.
[0089]
If the appropriateness of the projection center point cannot be confirmed, the projection center point may be selected again, or the feature points may be appropriately grouped and divided into object portions to execute the projection process.
[0090]
As described above, the texture image generation apparatus according to the second embodiment can confirm that the relationship between the shape of the target three-dimensional object and the projection center point is appropriate in the projection processing, and can perform appropriate projection processing. Therefore, it is possible to generate a texture image with high quality and low storage capacity.
[0091]
(Embodiment 3)
Similar to the first embodiment, the texture image generating apparatus according to the third embodiment projects and develops a known shape surface composed of a developable surface as a projection surface, and automatically generates a texture image by pasting the texture of the photographed image on the development plane. However, the projection unit and the division unit of the texture image generation apparatus according to the first embodiment are provided with a projection unit and a division unit that further improve the performance.
[0092]
The outline of the overall configuration of the texture image generation apparatus according to the third embodiment and the outline of the processing flow of this apparatus will be described with reference to the drawings.
FIG. 12 is a schematic configuration diagram of the texture image generation apparatus according to the third embodiment. As shown in FIG. 12, the apparatus of the third embodiment includes a projection unit 30 b instead of the projection unit 30 in the configuration of the first embodiment, and includes a division unit 40 a instead of the division unit 40. In addition to the configuration of the projection unit 30 of the first embodiment, the projection unit 30b further includes a projection adjustment unit 37. The components other than the projection adjustment unit 37 of the projection unit 30b are the same as the components described with the same numbers in the first embodiment, and thus description thereof will be omitted, and the projection unit 30b will be mainly described. explain. Further, the dividing unit 40 a includes a development boundary line adjustment unit 41 and a developable surface development boundary line adjustment unit 42. As in the first embodiment, although not shown, the memory and devices necessary for the control processing of the entire system are provided.
[0093]
The projection adjustment unit 37 of the projection unit 30b and the development boundary line adjustment unit 41 of the division unit 40a are arranged so that the boundary between the developable surface and the flat surface is generated in the projection process on the projection surface. The expansion / reduction is executed to adjust the development boundary between the two, and the boundary ridge line of the projection forming surface is used as the development boundary line.
[0094]
Here, the cross-border plane will be described. In the projection processing, the projection point sequence is arranged on the projection surface, but the projection points are formed on the developable surface and the flat surface, so that a part of the projection points that are the vertices forming the projection formation surface are developable. In some cases, a surface (called a transboundary surface) that is formed on the surface and in which other portions are formed on the plane is generated. For example, in a case where a cylindrical surface is employed as the projection surface as shown in FIG. 13a, the projection forming surface 1303 is a transboundary surface formed across the upper surface 1301 and the developable surface 1302. In such a case, if the developable surface and the flat surface are adjusted, and the development boundary line is adjusted so that the boundary edge of the projection forming surface can be used as the development boundary line, the surface on which the texture image is pasted is not divided. The distortion can be reduced, and the performance of the texture image generation apparatus can be further improved.
[0095]
Adjust the expandable surface, extension / retraction of the plane, and adjustment of the development boundary as follows.
The first adjustment method is a method including a developable surface projection adjustment unit 37a and a development boundary line adjustment unit 41 that adjust projections of feature points that form a transboundary surface and project all of them onto the developable surface. As shown in FIG. 13a, among the projection points forming the transboundary surface 1303, the projection points 1305 and 1306 on the developable surface are set as the developable surface boundary projection points, and the projection point 1307 on the plane is the plane boundary projection point. And The developable surface projection adjustment unit 37a first extends the developable surface as indicated by 1302 ′ in FIG. 13b. Further, a projection line passing through the plane boundary projection point 1307 is extended from the projection center point 1304, and a point intersecting the extension surface of the developable surface is obtained as a developable surface adjustment projection point 1307 ′. This process is executed along the boundary surface between all the developable surfaces and the flat surface. Next, in the dividing process, the development boundary line adjustment unit 41 sets the developable surface boundary ridge line as a line connecting the generated developable surface adjustment projection points in order, and the plane boundary ridge line as a line connecting the plane boundary projection points in order. Then, as shown in FIG. 13c, the development boundary line between the developable surface and the plane becomes a boundary ridge line of the projection forming surface, and one projection forming surface is not divided into the developable surface and the flat surface. As described above, the texture image generation apparatus can further improve the performance of texture image generation by including the developable surface projection adjustment unit 37a and the development boundary adjustment unit 41.
[0096]
The second adjustment method includes a plane projection adjustment unit 37b that adjusts projections of feature points that form a transboundary surface and projects all of them onto a plane, and a development boundary line adjustment unit 41. As shown in FIG. 14b, among the projection points forming the transboundary surface 1303, the projection points 1305 and 1306 on the developable surface are set as the developable surface boundary projection points, and the projection point 1307 on the plane is the plane boundary projection point. And The plane projection adjustment unit 37b obtains, as plane adjustment projection points 1305 ′ and 1306 ′, points that extend from the projection center point 1304 through the developable surface boundary projection points 1305 and 1306 and intersect the plane extension plane. This process is executed along the boundary surface between all the developable surfaces and the flat surface. If the development boundary adjustment unit 41 sets the plane boundary ridge line as a line connecting the generated plane adjustment projection points in order, and the development plane boundary ridge line as a line connecting the development plane boundary projection points in order, FIG. As shown, the boundary line between the developable surface and the plane becomes the boundary ridge line of the projection forming surface, and one projection forming surface is not divided into the developable surface and the flat surface. As described above, the texture image generation apparatus can further improve the performance of texture image generation by including the plane projection adjustment unit 37b and the development boundary adjustment unit 41.
[0097]
The third adjustment method is a method in which the projection of the feature points forming the transboundary surface when the projection surface has a boundary between the expandable surfaces is adjusted and projected onto one expandable surface. In this method, the developable surface projection adjustment unit 37c and the development boundary adjustment unit 41 are provided. Since the basic idea is the same as the first adjustment method and the second adjustment method described above, description thereof is omitted here.
[0098]
Next, the developable surface development boundary line adjustment unit 42 cuts along a boundary ridge line of the projection forming surface on the developable surface when cutting a part of the developable surface in developing the developable surface. It is. As shown in FIG. 15a, the projection forming surfaces 1501 and 1502 are formed across the development boundary 1500 of the developable surface given as a default. As shown in FIG. 15B, the developable surface development boundary line adjustment unit 42 adjusts the developable surface development boundary line 1500 to the left and right to obtain a new developable surface boundary ridge line 1500 ′ with the boundary ridge line of the projection formation surface. By performing this developable surface development boundary adjustment along all the developable surface development boundaries, one projection forming surface is not divided into left and right in the developable surface development process.
[0099]
As described above, the texture image generation apparatus according to the third embodiment can further improve the performance of texture image generation by including the developable surface development boundary line adjustment unit 42.
[0100]
(Embodiment 4)
Similar to the first embodiment, the texture image generation apparatus according to the fourth embodiment projects and develops a known shape surface including a developable surface and a plane as a projection surface, and automatically pastes the texture of the photographed image on the development plane to automatically generate the texture image. What is generated is an image writing unit 60a in which the image writing unit 60 of the texture image generating apparatus of the first embodiment is further improved in performance.
[0101]
FIG. 16 is a schematic configuration diagram of a texture image generation apparatus according to the fourth embodiment. As shown in FIG. 16, the apparatus according to the fourth embodiment includes an image writing unit 60a instead of the image writing unit 60 in the configuration of the first embodiment. In addition to the configuration of the image writing unit 60 of the first embodiment, the image writing unit 60a further includes a captured image selection unit 61 and an affine transformation unit 62. The components other than the photographed image selection unit 61 and the affine transformation unit 62 of the image writing unit 60a are the same as the components described with the same numbers in the first embodiment, and thus description thereof is omitted here. The photographic image selection unit 61 and the affine transformation unit 62 will be mainly described. As in the first embodiment, although not shown, the memory and devices necessary for the control processing of the entire system are provided.
[0102]
The photographed image selection unit 61 and the affine transformation unit 62 adjust both distortions in fitting and writing a texture of a photographed image on a development plane in the image writing process.
[0103]
First, the first adjustment method provided in the photographed image selection unit 61 is that each surface on the photographed image is photographed from multiple directions, so the texture of the photographed image photographed in the shape closest to the shape of the surface to be applied is obtained. It is to select as a captured image to be applied. As texture data of a photographed image to be applied by selecting the photographed image, data with the least distortion can be selected, and the performance of texture image generation can be further improved.
[0104]
The second adjustment method provided in the photographed image selection unit 61 selects each face on the photographed image as a photographed image to which the texture of the photographed image photographed from the closest viewpoint is applied because each face is photographed from multiple directions. That is. As texture data of a photographed image to be applied by selecting the photographed image, data with the least distortion can be selected, and the performance of texture image generation can be further improved.
[0105]
Next, the affine transformation unit 62 adjusts the distortion by transforming the texture data of the captured image into the shape of the writing surface by performing affine transformation. Since the projection forming surface formed on the projection surface is formed by projecting the original three-dimensional object onto the projection surface, it has a slightly distorted shape from the outer surface of the original three-dimensional object. Therefore, it is preferable to adjust the shape of the texture data of the photographed image by the affine transformation unit 62 prior to surface fitting / texture writing. Here, affine transformation is transformation represented by a linear expression of coordinates, and refers to transformation that changes the shape of an image by enlargement, reduction, translation, rotation, or the like of the image.
[0106]
The processing flow of the affine transformation unit 62 will be described below.
First, an affine transformation coefficient is determined (step S1701 in FIG. 17). That is, the coefficient of affine transformation that matches the coordinates of the feature points on the photographed image and the coordinates of the projection points of the corresponding feature points projected on the development plane is obtained.
[0107]
In order to generate an image in which the distortion of the shape generated by the projection is adjusted, the corresponding pixel value on the photographed image may be affine transformed and written in the corresponding coordinates as each pixel value of the surface generated as the texture image. First, the target pixel of the target surface of the texture image, for example, the lower left pixel having the smallest XY coordinate value in the XY coordinates is designated as the target pixel (step S1702).
[0108]
The position of the pixel on the captured image with respect to the position of the target pixel is determined by affine transformation (step S1703).
The pixel value of the captured image at the position obtained in step S1703 is read and written as the pixel value of the pixel of interest on the texture image (step S1704).
[0109]
Steps S1702 to S1704 are executed for all the pixels. In other words, the pixel of interest noted in step S1702 is sequentially moved so as to scan in the X direction and the Y direction, and the pixel of interest is designated one after another, and the pixel corresponding to that pixel of interest in steps S1703 to S1704 is designated. Write the value.
[0110]
As described above, the texture image generation apparatus according to the fourth embodiment includes the image selection unit 61, so that the image with the least distortion can be selected as the image of the writing surface among the input images, and further includes the affine transformation unit 62. Thus, it is possible to adjust the distortion of the shape generated by the projection and execute the surface fitting / writing correctly, and the texture image generation performance can be further improved.
[0111]
(Embodiment 5)
The texture image generation device of the present invention can be constructed using various computers by recording and providing a program describing processing steps for realizing the configuration described above on a computer-readable recording medium. As shown in the example of the recording medium shown in FIG. 18, the recording medium on which the program having the processing steps for realizing the texture image generating apparatus of the present invention is recorded is a portable recording medium such as a CD-ROM 202 or a flexible disk 203. 201 may be any of the recording medium 200 in the recording device on the network and the recording medium 205 such as a computer hard disk or RAM. When the program is executed, the program is loaded onto the computer 204 and Runs on memory.
[0112]
【The invention's effect】
According to the texture image generation device of the present invention, a known shape surface composed of a developable surface and a plane can be projected and developed as a projection surface, and a texture image can be automatically generated by pasting the texture of the photographed image on the development plane. Therefore, it is possible to generate a texture image with high quality and low storage capacity.
[0113]
Further, according to the texture image generation apparatus of the present invention, it is possible to select an appropriate projection plane according to the shape of the three-dimensional object, and even if the three-dimensional object is complicated, the projection plane is optimized for each object part. Can be selected.
[0114]
Moreover, according to the texture image generation apparatus of this invention, it can be confirmed whether the projection center point in a projection process is appropriate, and an appropriate texture image can be produced | generated. Further, according to the texture image generation device of the present invention, when the projection surface is a combination of a developable surface and a flat surface, the development boundary line in the development process can be used as the boundary ridge line of the projection formation surface. Thus, a texture image with less distortion can be obtained.
[0115]
Further, according to the texture image generation apparatus of the present invention, it is possible to select an image with the least distortion when writing an image on a development plane, and to write an image by adjusting shape distortion by affine transformation. The texture image quality can be improved.
[Brief description of the drawings]
FIG. 1 is a diagram showing an example of development of a cylindrical surface
FIG. 2 is a diagram showing examples of combinations of developable surfaces
FIG. 3 is a diagram showing a concept of projection processing using a projection surface.
FIG. 4 is a diagram showing a basic flow of texture image generation in the texture image generation apparatus of the present invention.
FIG. 5 is a schematic configuration diagram of a texture image generation apparatus according to the first embodiment of the present invention.
FIG. 6 is a data structure example handled by the texture image generation apparatus according to the first embodiment of the present invention.
FIG. 7 is a diagram for explaining the formation of a convex hull.
FIG. 8 is a schematic configuration diagram of a texture image generation device according to a second embodiment of the present invention.
FIG. 9 is a flowchart showing processing for assigning a projection plane to each object part;
FIG. 10 is a diagram showing an example of an inappropriate projection center point
FIG. 11 is a flowchart showing processing steps for confirming whether or not the projection center point is proper by the projection center point appropriateness confirmation unit 36 according to the second embodiment of the present invention.
FIG. 12 is a schematic configuration diagram of a texture image generation apparatus according to a third embodiment of the present invention.
FIG. 13 is a diagram for explaining projection adjustment by the developable surface projection adjustment unit 37a and the plane projection adjustment unit 37b according to the third embodiment of the present invention.
FIG. 14 is a diagram for explaining projection adjustment by the developable surface projection adjustment unit 37c according to the third embodiment of the present invention.
FIG. 15 is a diagram illustrating adjustment of the development boundary line by the developable surface development boundary line adjustment unit according to the third embodiment of the present invention.
FIG. 16 is a schematic configuration diagram of a texture image generation device according to a fourth embodiment of the present invention.
FIG. 17 is a flowchart showing processing steps of the affine transformation unit 62 according to the fourth embodiment of the present invention.
FIG. 18 shows an example of a recording medium.
FIG. 19 is a flowchart showing conventional texture image generation processing steps;
FIG. 20 is a diagram showing the concept of conventional 3D image data generation based on feature points
FIG. 21 is a flowchart showing a conventional three-dimensional image reproduction processing step on a computer screen.
FIG. 22 is a diagram for explaining a conventional method of storing texture images to be pasted on each surface by arranging them on a plane.
[Explanation of symbols]
10 Input section
20 Feature point extraction unit
21 Feature point specification part
30, 30a, 30b Projection unit
31 Feature point placement section
32 Projection plane generator
33 Projection center point designation part
34 Projection processing unit
35 Projection plane adjustment unit
36 Projection center point appropriateness confirmation section
37, 37a Projection adjustment unit
40, 40a Dividing part
41 Development boundary adjustment part
42 Expandable surface boundary adjustment part
50 Development department
60, 60a Image writing unit
61 Image selection section
62 Affine transformation unit
70 Output section
100 data
101 Image information area
102 Feature point information area
103 Projection information area
104 Projection plane information area
105 Other information area
106 header
200 Recording medium such as hard disk at line destination
201 Portable recording media such as CD-ROM and flexible disk
202 CD-ROM
203 Flexible disk
204 Computer
205 Recording medium such as RAM / hard disk on computer

Claims (14)

A texture image generation device that generates a texture image of 3D image data based on feature point data extracted from a 3D object and 2D image data of an outer surface of a captured 3D object,
An input unit, a projecting unit, a dividing unit, a developing unit, and an image writing unit;
The input unit accepts input of feature point data of a three-dimensional object and two-dimensional photographed image data of an outer surface,
The projection unit projects a feature point of the input three-dimensional object onto a projection plane that is a known shape surface that is a developable surface and includes the feature point, and obtains a projection point.
The dividing unit divides the projection surface into small surfaces by projection points of the feature points,
The development unit develops the projection plane into a plane to generate a development plane composed of the facets,
The image writing unit generates a texture image by writing a photographed image of the outer surface of the three-dimensional object in which a corresponding feature point is photographed for each facet on the development plane ,
The projection unit monitors writing of a photographed image of the outer surface of the three-dimensional object to each facet on the development plane by the image writing unit, and corresponding feature points on each facet on the development plane A texture image generation apparatus, comprising: a projection center point appropriateness confirmation unit that confirms whether or not the correspondence with the captured image obtained is correctly obtained .   The projection unit includes a projection plane adjustment unit, and the projection plane adjustment unit adjusts the type and size of the projection plane generated by the projection unit to match the shape of the three-dimensional object formed by the feature points. The texture image generating apparatus according to claim 1, wherein the distortion of the shape of the projection plane and the shape of the three-dimensional object formed by the feature points is minimized. Projection in which the projecting unit adjusts the developable surface or the plane when there is a transboundary surface that straddles different developable surfaces and planes of the projection surface among the projection forming surfaces formed by the projection processing of the feature points It has an adjustment part,
  The texture image generation device according to claim 1, wherein the division unit includes a development boundary adjustment unit that uses a boundary ridge line of the projection forming surface as a development boundary. The projection portion is provided with a projection center point designation unit for inputting information of the projection center point, in any one of claims 1 to 3 for determining the projection center point based on the designation from said projection center point designation unit The texture image generation device described. The feature point group information for adding the feature points of the three-dimensional object into groups is attached, and the projection unit performs a projection process on each group of feature points according to the feature point group information . The texture image generation device according to any one of the above. The projection unit forms the transboundary surface when there is a transboundary surface projected across a developable surface and a plane on the projection surface among the projection formation surfaces formed by the projection processing of the feature points. When projecting the feature points, it comprises a developable surface projection adjustment unit that adjusts the developable surface so that all of the feature points are projected onto the developable surface, and among the projected points of the feature points of the transboundary surface, The projection point on the developable surface is set as a developable surface boundary projection point, the projection point on the plane is set as a plane boundary projection point, and the developable surface projection adjustment unit is configured to project the plane boundary projection from the projection center point. Obtain a point that intersects the extended surface of the developable surface by extending the projection line passing through the point as a developable surface adjustment projection point,
The dividing unit sets a boundary ridge line of the developable surface as a line connecting the developable surface boundary projection point and the developable surface adjustment projection point in order, and a line connecting the plane boundary ridge line in order with the plane boundary projection point The texture image generation apparatus according to claim 3 , wherein the developable surface and the plane are divided as follows. The projection unit forms the transboundary surface when there is a transboundary surface projected across a developable surface and a plane on the projection surface among the projection formation surfaces formed by the projection processing of the feature points. A plane projection adjustment unit that adjusts the plane so that all of the feature points are projected onto the plane when the feature points are projected, and among the projected points of the feature points on the cross-border plane, A projection point is defined as a developable boundary projection point, a projection point on the plane is defined as a plane boundary projection point, and the plane projection adjustment unit calculates a projection line passing through the developable boundary projection point from the projection center point. Find a point that extends and intersects the extended surface of the plane as a plane adjustment projection point,
The dividing unit uses the boundary ridge line of the plane as a line connecting the plane boundary projection point and the plane adjustment projection point in order, and the boundary ridge line of the developable surface as a line connecting the developable surface boundary projection point in order. The texture image generation device according to claim 3 , wherein the developable surface and the plane are divided. The projection unit has a transboundary surface projected across the first developable surface and the second developable surface on the projection surface among the projection formation surfaces formed by the projection processing of the feature points. In addition, when projecting the feature points that form the transboundary surface, a developable surface projection adjustment unit that adjusts the developable surface so that all of the feature points are projected onto the first developable surface, Among the projection points of the feature points on the transboundary surface, the projection point on the one developable surface is set as the first developable surface boundary projection point, and the projection point on the second developable surface is the second The developable surface boundary projection point, and the developable surface projection adjustment unit extends a projection line passing through the second developable surface boundary projection point from the projection center point to extend the first developable surface. Find the point that intersects the surface as the developable surface adjustment projection point,
The dividing unit is configured such that a boundary ridge line of the first developable surface is a line that sequentially connects the first developable surface boundary projection point and the developable surface adjustment projection point, and a boundary of the second developable surface The texture image generation device according to claim 3 , wherein the first developable surface and the second developable surface are divided by using a ridge line as a line connecting the second developable surface boundary projection points in order. The texture image generation device according to any one of claims 1 to 3, wherein the developing unit cuts and develops along a ridge line connecting the projection points of the feature points in the development processing of the developable surface into a plane. In the dividing process of the projection surface, the dividing unit generates a convex hull having the projection point of the feature point as an apex, and approximates the small surface obtained by dividing the projection surface with the small surface constituting the convex hull. The texture image generation device according to any one of claims 1 to 3 to be handled. In the process of writing the photographed image of the three-dimensional object on the development plane, the image writing unit uses image data captured with the largest facet area to be written as data of each facet to be written. The texture image generation device according to any one of claims 1 to 3 to be used. The image writing unit uses image data obtained by photographing a facet to be written from the closest viewpoint as data of a photographed image of each facet to be written in the process of writing the photographed image of the three-dimensional object on the development plane. Item 4. The texture image generation device according to any one of Items 1 to 3 . In the process of writing the captured image of the three-dimensional object to the development plane, the image writing unit matches the shape of the captured image data of each facet to be written with the shape of the corresponding facet on the development plane. The texture image generation device according to any one of claims 1 to 3, wherein the texture image is written by affine transformation. A computer-readable recording of a processing program for realizing a texture image generation device that generates a texture image of 3D image data based on feature point data extracted from a 3D object and image data of the outer surface of the 3D object A recording medium,
The processing program includes an input processing step for receiving input of feature point data of a three-dimensional object and photographed image data of an outer surface, and inputting the input feature point of the three-dimensional object on a known shape surface that is a developable surface. A projection processing step for obtaining a projection point by projecting on a projection plane including the feature point, and a division processing step for dividing the projection plane projected by the projection processing step into a small surface by the projection point of the feature point; A development processing step of developing the projection plane into a plane to generate a development plane composed of the small faces; A texture image is generated by causing a computer to execute an image writing processing step of writing a photographed image of the surface ,
In the projection processing step, the writing of the photographed image of the outer surface of the three-dimensional object to each small surface on the development plane by the image writing processing step is monitored and corresponds to each small surface on the development plane. A recording medium having recorded therein a processing program for causing the computer to execute a projection center point appropriateness confirmation processing step for confirming whether correspondence with a photographed image obtained by photographing a feature point is correctly obtained .

Images