JP2002251627A - Method and device for generating image data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that a method for generating image data which processes whole image data by each frame restricts image quality of the image data to be generated by performance of a hardware and image quality is far lower than that of image data generated by spending sufficient time, and a problem that a method for generating image data which switches a plurality of scenarios by a user command can present high-quality image data to the user but restricts the number of scenarios depending on capacity of a storage device, and a degree of freedom the user is allowed is low. SOLUTION: Virtual world data is classified into two kinds as a background part which does not interact with a user and an object part which interacts with the user. As for the background part, high-quality image data is previously generated by spending sufficient time. As for the object part, image data for only a part which does not hide behind the background part is generated in a phase that a way the image data is seen on each frame is determined by the user command.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for generating image data from virtual world data, and more particularly to an image data generation including a process for synthesizing a plurality of image data.

[0002]

2. Description of the Related Art Conventional techniques for generating image data from virtual world data include the following.

As a technique in the case where there is an interaction with a user, there is a technique in which the shape of an object in a virtual world is approximated and represented by a polyhedron, and image data is generated in real time. The contours of the object and the effects of light rays
It is calculated by calculation from dimensional data (hereinafter referred to as “polygon data”). In order to express the texture, data called texture data is used separately from the polygon data. The texture data is pasted on the surface of each polygon after the shape and color tone are appropriately modulated according to the position of the viewpoint and the inclination of the polygon. These techniques are used in games that generate image data in consideration of three-dimensional data.

[0004] In order to improve the image quality of the finally obtained image data, it is desirable to represent an object by a set of polygons as much as possible. There is a hardware-dependent upper limit on the number of polygons. Therefore, a method of reducing the number of polygons representing each object so as to minimize the deterioration of the image quality of the finally obtained image data has been studied. For example, "D. Luebke, C.E.
rickson: View-DependentSimp
lectionation of Arbitrary P
polygonal Environments, Com
puter Graphics (Proceeding
s of SIGGRAPH '97), pp. 199-
208, 1997 ".

[0005] When there is no interaction with the user, it is possible to rigorously perform ray tracing or the like from the precisely modeled virtual world data and generate high-quality image data in a sufficient time in advance. By preparing a mechanism that can restore the three-dimensional structure in advance at the time of capturing the actual image data, the operator makes fine adjustments, but restores the three-dimensional structure of the scene from the actual image data, and It is also possible to embed an object generated by computer graphics (hereinafter, referred to as “CG”) in the real image data. These technologies are used in fields such as movies. As the data obtained by restoring the three-dimensional structure of the scene, one of two types, that is, a "depth map" in which depth data is recorded for each pixel of image data and a polygon representation is used. ing.

[0006]

SUMMARY OF THE INVENTION It is an object of the present invention to realize a content that interacts with a user while presenting high-quality image data to the user and contradicting the three-dimensional virtual world data as little as possible. Is the purpose.

When there is interaction with the user, it is necessary to represent the object with a large number of polygons in order to present high-quality image data to the user, but the object is represented with a large number of polygons. In such a case, there is a problem that very high hardware performance is required to process them in real time. As a countermeasure, if the method of reducing the number of polygons representing each object is used so that the image quality degradation of the finally obtained image data is minimized, the object is approximated by a rougher model than the original model. Due to the error caused by being performed, the contours of the object in the image data presented to the user are not smooth, and the edges of the approximated polyhedron are easily perceived in the image data presented to the user. Occurs.
The latter is due to the fact that the function for modulating the texture data differs greatly between adjacent polygons.

In the technique in the case where there is no interaction with the user, the amount of calculation for generating the image data is enormous, and therefore the following problem occurs. Processing real-time interactions with users using precisely modeled virtual world data requires very high hardware performance. When the CG is synthesized by restoring the three-dimensional structure from the actual image data, there are the following problems depending on what is used as the data obtained by restoring the three-dimensional structure of the scene. When a depth map is used as data obtained by restoring the three-dimensional structure of a scene, the data becomes resolution-dependent, so the depth map itself must be recreated every time the resolution of the image data to be generated changes. There is a problem that the amount of data is larger than that of. Also, when using polygon representation, it is still a research topic and it is generally difficult to strictly restore the scene using a large number of polygons. The number must be small,
Therefore, there is a problem that the obtained three-dimensional structure includes many errors.

[0009]

According to the present invention, in order to solve the above-mentioned problems, it is necessary to take sufficient time beforehand for a portion having no interaction with a user (hereinafter referred to as a "background portion"). In addition, for a part having an interaction with the user (hereinafter referred to as an “object part”), image data is generated in real time, and the image data is combined and output. By taking such means, it is possible to provide high-quality image data and also to interact with the user. This is because the portion in which an image is to be generated in real time is limited to the object portion, and the amount of data to be processed in real time is reduced.

According to the present invention, a means is provided for generating data as an image data of the object portion by deleting a portion of the object portion hidden behind the background portion. By taking such a measure, it is possible to generate image data in consideration of the visibility of the entire virtual world simply by overlaying the image data of the object portion on the image data of the background portion, thereby deteriorating the image quality. And the amount of data to be processed in real time can be reduced.

Further, according to the present invention, when the image data of the object part is generated from the virtual world data of the background part and the virtual world data of the object part by selection processing and reconstruction processing of the polygon, the background part is hidden. When the second virtual world data which is equivalent to the virtual world data of the above and has a smaller data amount than the virtual world data of the background portion is generated in advance, and the image data of the object portion is generated, the virtual world of the background portion is generated. Instead of the data
Using virtual world data. By taking such means, it is possible to reduce the amount of data to be processed in real time without deteriorating the image quality (or suppressing the deterioration).

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to FIGS. FIG. 1 is a data flow diagram in the present embodiment. Virtual world data 100
Includes background data 101, common data 102, and object data 103. The virtual world data 100 is data given in advance. Further, the interaction with the user is performed by the user input 104.

The background data 101 is the polygon data 10
10 and texture data 1011. The background data 101 is data of a portion of the polygon data and the texture data included in the virtual world data 100 that is not related to the interaction with the user.

The common data 102 includes viewpoint data 1020
Is provided. The common data 102 is the virtual world data 100
This is data obtained by removing polygon data and texture data from the data provided in. Therefore, the common data 102
May include ray data in addition to the viewpoint data 1020.

The object data 103 includes the polygon data 10
30 and texture data 1031. The object data 103 is data of a part of the polygon data and the texture data included in the virtual world data 100 that is related to the interaction with the user.

The background image data generation processing 108 receives the polygon data 1010, the texture data 1011 and the common data 102 as input and outputs the background image data 105
Is output. Background image data generation processing 108
For example, a known technique used in commercially available image rendering software or the like can be used. It is preferable that the background image data generation processing 108 be performed in advance by preparing sufficient resources in terms of processing time, storage capacity, and the like so that the background image data 105 satisfies the required image quality.

The object image data generation processing 109 outputs the object image data 106 using the polygon data 1010, the common data 102, the polygon data 1030, the texture data 1031 and the user input 104 as inputs. Details of the object image data generation processing 109 will be described with reference to FIG.

The image data synthesizing process 110 receives the background image data 105 and the object image data 106 and outputs the output image data 107. The image data synthesizing process 110 is a process of overwriting the object image data 106 on the background image data 105.

In the present embodiment, according to the flow described above, the output image data 10
Get 7.

In the above description, it is assumed that the shapes of all objects existing in the virtual world are represented by a set of polygons. The polygon data may be any data that can specify the shape, position, and orientation of an object existing in the virtual world, and is data that enumerates the three-dimensional positions of vertices of all polygons existing in the virtual world. No need to be. For example, when there are a plurality of objects having the same shape, only one reference object has data listing the three-dimensional positions of the vertices of the polygon. The relative position and the relative posture may be described.

Next, the description will be continued with reference to FIG. FIG.
It is a conceptual diagram of the image data generation processing of a present Example. The objects 2011 and 2012 in the virtual space 201 do not interact with the user, and their shapes are described by polygon data 1010. An object 2021 in the virtual space 202 is an object that interacts with the user, and its shape is described by polygon data 1030. Note that the virtual space 201 and the virtual space 202 are actually the same virtual space.

Common data 102 and polygon data 101
The object image data 106 is generated from 0 and the polygon data 1030 in consideration of the relationship between visibility and occlusion. For example, when the virtual space 201 and the virtual space 202 are actually considered in the same virtual space 200, each object is
It is assumed that the objects are arranged in the order of 011 → object 2021 → object 2012. At this time, the object image data 106 is generated using the texture data 1031 in addition to the data described above with respect to the remaining area 2022 excluding the part hidden from the object 2011 from the object 2021. Here, it is assumed that an area other than the area 2022 of the object image data 106 is transparent.

By generating the object image data 106 as described above, the output image data 107 obtained by overdrawing the object image data 106 on the background image data 105 which has been generated in advance can be used as the output image data 107 of each object. , And has no contradiction.

In the above example, the object image data 106 is generated on the assumption that "the region other than the region 2022 is transparent", and the background image data and the object image data are synthesized by the image data synthesis processing. Data 106
, The area 2022 may be drawn directly on the background image 105. That is, it is not necessary to determine the values for all the pixels constituting the image data, and it is also possible to perform a partial update process for only necessary portions.

Next, a procedure for generating image data will be described with reference to FIG. FIG. 3 is a flowchart of the object image data generation processing of the present embodiment. When the object image data generation processing is started in step 400, first, in step 4
At 01, the pixel values of all the pixels of the object image data are initialized to “transparent”.

Next, in step 402, object image data is generated on the assumption that there is no background data and only object data exists.

Finally, in step 403, the pixel value of the portion hidden by the background data is set to "transparent", and in step 404, the object image data generation processing ends.

In order to perform a process equivalent to the process described with reference to FIG. 3, instead of using the "transparent" value of the pixel, Z
A buffer algorithm can also be used.

Next, the configuration of the apparatus of this embodiment will be described with reference to FIG. FIG. 4 is an apparatus configuration diagram in the present embodiment. The same symbols as those in FIG. 1 represent the same as those in FIG.

A background image data reproducing program 305;
An object image data generation program 306, an image data synthesis program 307, an image display program 308,
May be transferred from the auxiliary storage device 303 to the main storage device 30 as necessary.
2 is executed by the central processing unit 301. The data used by each program is also read from the auxiliary storage device 303 into the main storage device 302 as needed, and processed.

The background image data reproduction program 305 is
A process of reading the background image data 105 generated in advance from the auxiliary storage device 303 is performed. When the background image data is stored in a compressed state, a decompression process is also performed after the background image data is read.

The object image data generation program 306 includes:
A process of generating object image data 106 using the polygon data 1010, the common data 102, the object data 103, and the user input 104 input from the user command input unit 304 is performed.

The image data synthesizing program 307 performs a process of overwriting the generated object image data 106 on the background image data 105.

The image display program 308 performs a process of outputting the generated output image data 107 to the image display means 309.

As described above, according to the first embodiment, the object image data reflecting the interaction with the user is overwritten on the background image data generated in advance by providing sufficient resources. In addition, it is possible to interact with a user while providing high-quality image data. Since the object image data is generated by the overdrawing processing so as not to cause inconsistency in the three-dimensional structure with the background data, the output image data can be generated by a simple synthesis processing without deteriorating the image quality. it can.

The scope of application of this embodiment is not limited to still image data. It is easy to apply this embodiment to the case where moving image data is generated. This is the same for the following second embodiment.

The second embodiment of the present invention will be described below with reference to FIGS. The second embodiment is an example in which the amount of data necessary for the object image data generation processing is reduced in addition to the first embodiment.

FIG. 5 is a data flow diagram in this embodiment. The same symbols as those in FIG. 1 represent the same as those in FIG. Hereinafter, portions different from FIG. 1 will be described.

The shielding object data generation processing 501 receives the polygon data 1010, the polygon data 1030, and the common data 102, and outputs the shielding object data 502.

The shielding object data generation processing 501 includes an effective polygon selection processing described with reference to FIGS. 6 and 7, and FIGS.
Either or both of the polygon reconstruction processing described in 4 can be used. If you do both,
From the viewpoint of the load at the time of calculation, the effective polygon selection processing is preferably performed first.

Here, the shielding object data 502 is polygon data. Therefore, as the processing for generating the object image data 106, the object image data generation processing 109 can be used as it is. That is, as input to the object image data generation processing 109, the polygon data 101
The shielding object data 502 may be used instead of 0. As an alternative to polygon data, shield data 5
02 alone.

Although the interaction with the user is performed by the user input 104 as in the first embodiment, the interaction with the user that can be realized in this embodiment is one in the virtual world according to the user's instruction. Or multiple objects move,
One or more objects appear or disappear according to the user's instruction, or one or more objects deform according to the user's instruction.

More strictly speaking, all possible positions of the object, possible postures of the object, and possible shapes of the object are all given in advance, and for each of them, the user directly determines whether to display or not to display. Or the interaction of indirect selection.

Hereinafter, the effective polygon selection processing will be described with reference to FIGS. The valid polygon selection process is
This is a process for selecting polygon data from the polygon data 1010 that is meaningful to be left as the shield data 502. The polygon data that is meaningful to keep may be data specified by the user. Also, instead of polygon data that is meaningful to keep, polygon data specified by the user may be used.

FIG. 6 is a flowchart of the effective polygon selection process in this embodiment. First, step 601
, An effective polygon selection process is started.

Next, in step 602, data necessary for calculation is read. The data read here includes polygon data 1010, common data 102,
This is polygon data 1030.

Subsequently, for the polygon data 1010, the back surface polygon removal processing is performed in step 603, the out-of-view volume polygon removal processing is performed in step 604, and
In step 5, the shielded polygon removal processing is performed. These processes include, respectively, backface culli
ng, view-frustum culling, o
Known techniques, known under the name cculation culling, can be used. Polygon data 1
The polygon data 608 is obtained as a result of performing the above-described three types of polygon removal processing on 010.

Next, at step 606, an unshielded polygon removal process is performed. The unobstructed polygon removal process is a process of removing polygons that do not hide any one of the polygons represented by the polygon data 1030 from among the polygons represented by the polygon data 608 as unnecessary polygons. is there. This can be performed by determining the relationship between each polygon represented by the polygon data 608 and each polygon represented by the polygon data 1030. The “relationship” is the common data 1
02, it may be set to one of three types of “hide”, “hidden”, and “neither”.

In step 607, the polygon data which is not removed by the polygon removal processing is stored.
In step 608, the valid polygon selection processing ends.

FIG. 7 is a conceptual diagram of the effective polygon selection processing in this embodiment. In FIG. 7, the viewpoint position 700 and the viewing volume 701 are determined from the common data 102.

Object 710 and object 711 and object 7
12 and object 713 and object 714 and object 71
Reference numeral 5 denotes an object whose display / non-display is determined by interaction with the user.

At this time, the polygon 720, the polygon 721, the polygon 722, the polygon 723, and the polygon 724 are polygons to be removed by the back surface polygon removal processing. The polygon 730 is a polygon to be removed by the extra-viewing volume polygon removal processing, and the polygon 740 and the polygon 741 are polygons to be removed by the occluded polygon removal processing.

Further, the polygon 750 is removed by the unobstructed polygon removal processing. Ultimately, only the polygon 760 remains as valid polygon data.

Hereinafter, the polygon reconstruction processing will be described with reference to FIGS. The polygon reconstruction process generates polygon data having a smaller data amount from the polygon data 1010 under the condition that the same output image data 107 as that obtained by calculation using the polygon data 1010 is obtained. This is the processing to be performed.

FIG. 8 is a flowchart of the polygon reconstruction processing in this embodiment. When the polygon reconstruction processing is started in step 800, first, in step 8
01 performs polygon grouping processing. The polygon grouping process will be described in detail with reference to FIGS.

Next, in step 802, reconstruction processing in the image plane is performed for each group created in step 801. The reconstruction processing in the image plane will be described in detail with reference to FIGS.

Finally, in step 803, each polygon reconstructed in step 802 is embedded in the virtual space, and in step 804, the polygon reconstruction processing ends. The embedding process in the virtual space will be described in detail with reference to FIGS.

FIG. 9 is a flowchart of the polygon grouping process in this embodiment. When the polygon grouping process is started in step 900, first, in step 901, a characteristic vector is generated. The characteristic vector is defined for each polygon belonging to the polygon data 1010. The i-th element of the characteristic vector of the polygon 910 is determined by the context of the i-th polygon 920 belonging to the polygon data 1030 based on the viewpoint data 1020. That is, the polygon 910 becomes the polygon 92
"1" to hide 0, polygon 920 is polygon 91
"0" is set to hide 0, and "*" is set to neither of them.

Next, in step 902, the characteristic vectors of each polygon obtained in step 901 are sorted. Under the magnitude relation of “0” <“*” <“1”, the characteristic vectors may be arranged in lexicographic order.

Finally, in step 903, the characteristic vectors sorted in step 902 are merged by a greedy algorithm to form a group of polygons having similar features in context. Then, in step 904, the polygon grouping process ends.

The method of forming groups is such that the number of groups is as small as possible under the condition that "characteristic vectors of arbitrary polygon pairs belonging to the same group do not have both 0 and 1 at the same element position". It is good to make it. In the present embodiment, a method is adopted in which sorting is performed and merging is performed by a greedy algorithm in consideration of processing time.

Note that a characteristic vector can be defined for each group. As for the k-th element, the k-th element of the characteristic vector of a polygon belonging to the group is “0”.
Is “0”, if the k-th element of the characteristic vector of a certain polygon belonging to the group is “1”, “1”; otherwise, “*”.

By using the “group characteristic vector”, it is possible to efficiently perform the characteristic vector merge processing and the reconstruction processing in the image plane described below.

Steps 902 and 90
3 is that if the group can be finally formed so as to satisfy the condition that "the characteristic vector of any polygon pair belonging to the same group does not have both 0 and 1 at the same element position" Well, it is not necessary to limit to the method shown here.

Further, in the actual processing, based on the relationship of the data processing amount, instead of generating a characteristic vector from the context of each polygon belonging to the object data, the object data is expressed in advance as a plurality of convex polyhedrons. In this case, it is preferable to generate a characteristic vector according to the context of each convex polyhedron. FIG. 10 also illustrates this case.

As a method of expressing object data as a plurality of convex polyhedrons in advance, there is a method of first forming a convex hull from vertices constituting each polyhedron in the object data. This means that the outline of each polyhedron in the object data is approximated by a convex hull. Note that even when this method is adopted, the condition that the same output image data 107 as that obtained by calculation using the polygon data 1010 is obtained is satisfied.

Another method for expressing the object data as a plurality of convex polyhedrons in advance is as follows: "The protruding portions of the non-convex polyhedron are sequentially cut out, and the joints are covered with polygons." There is also a method of performing an operation while adding a surface to the object data.

FIG. 10 is a conceptual diagram (plan view) of the polygon grouping process in this embodiment. Viewpoint position 140
The context is determined based on 0. The first element is the object 14
When the characteristic vectors of the polygon 1420, the polygon 1421, and the polygon 1422 are obtained assuming the front-rear relationship with respect to 10, the second element with respect to the object 1411, and the third element with respect to the object 1412, the following is obtained.

First, since the polygon 1420 hides the object 1410 and has nothing to do with the appearance of the object 1411 and the object 1412, the characteristic vector of the polygon 1420 is (1, *, *). Similarly, polygon 14
The characteristic vector of 21 is (*, 1, *) and the polygon 142
The characteristic vector of No. 2 is (*, 0, 1). Under the condition that “0 and 1 do not exist at the same element position”, polygon 1421 and polygon 1422 are set to 1
Cannot be in two groups. This is because the second element of the characteristic vector of the polygon 1421 is 1 and the second element of the characteristic vector of the polygon 1422 is 0.

Then, the polygon 1420 and the polygon 14
22 as one group is group 1430
It is. The characteristic vector of the group 1430 is (1, 0,
1). A group 1431 includes only the polygon 1421 as one group. The characteristic vector of the group 1431 is (*, 1, *).

The characteristic vector of each group can be regarded as representing the context of the group and each object.

FIG. 11 is a flowchart of the reconstruction processing in the image plane in this embodiment. Here "image side"
Is a perspective plane when the output image data 107 is generated by the perspective transformation.

When the reconstruction processing in the image plane is started in step 1100, first, in step 1101,
Based on the characteristic vector of the group currently being processed, the image projection plane is set to “essential covering area” or “coverable area”
It is classified into three types of “forbidden area”. Here, the “essential covering area” is an area that needs to be covered with a triangle in order to obtain correct output image data 107.
The “coverable area” is an area in which correct output image data 107 can be obtained even when covered with a triangle, and includes an essential covered area. The “forbidden area” is an area that must not be covered with a triangle in order to obtain correct output image data 107.

Next, in step 1102, a triangulation of the coverable area is performed. Since each area is a polygon, a method generally well known in the field of computational geometry can be used for this processing.

Finally, in step 1103, triangles not including the essential covering area are removed, and in step 1104, the reconstruction processing in the image plane is completed.

A triangle that does not include an essential covering area can obtain correct output image data 107 even if it is removed, and can be removed to reduce the data amount.

The purpose of this processing is to set the
And a set of triangles that does not include the “forbidden area” is not required. The specific processing need not be limited to the processing illustrated in steps 1102 and 1103. For example, a combination algorithm method at the pixel level can be used.

FIG. 12 is a conceptual diagram of the reconstruction processing in the image plane in the present embodiment. 12, the same symbols as those in FIG. 10 represent the same ones as those in FIG. Image surface 14
An area 1460 in 61 is an area output as the output image data 107. In addition, the black area is "essential covering area"
, A white area indicates a “forbidden area”, and a hatched area indicates a “coverable area” other than the essential area.

The triangles 1423 and 1424 are
It is a triangle newly generated by the reconstruction processing in the image plane. Since FIG. 12 is a conceptual diagram, it is a diagram for explaining “what kind of triangle should be made in principle”. Therefore, it should be noted that the triangle 1423 and the triangle 1424 are different from the triangle configured by the flowchart described in FIG.

FIG. 13 is a flowchart of the embedding process into the virtual space in this embodiment. An alignment problem, which is a basic technique in the field of geometry, may be applied as long as the constraints of the context are satisfied. Here, the outline will be described.

When the embedding process into the virtual space is started in step 1300, first in step 1301,
A group of linear inequalities is generated according to the characteristic vector of the polygon 1310 currently being processed. At this time, a dual relationship between a point and a plane in a three-dimensional space is used.

Here, the characteristic vector of the polygon currently being processed can be created by modifying the characteristic vector 1320 of the group from which the polygon was generated with the following changes.

The polygon 1310 is used as the bottom surface and the viewpoint position 1
A polygonal pyramid 1330 having 400 as a vertex is determined.
When the p-th element of the characteristic vector 1320 is “0” and the p-th polygon belonging to the polygon data 1030 has no common part with the polygon 1330, the p-th element of the characteristic vector 1320 is represented by “*”. I do. The p-th element of the characteristic vector 1320 is “1”, and the p-th polygon belonging to the polygon data 1030 is a polygon pyramid 133.
If it has no common part with 0, the p-th element of the characteristic vector 1320 is set to “*”.

The inequality group is represented by “0” in the characteristic vector.
Alternatively, it is necessary and sufficient to generate only a portion related to the element of “1”.

Next, in step 1302, an objective function is set to reduce the problem to linear programming. However, it is only necessary to calculate a feasible solution, and it is not necessary to find an optimal solution. Therefore, an appropriate objective function may be set. For example, a condition such as “minimize the size of the inner product between the normal of the plane and the line-of-sight vector” may be set as the objective function.

Next, in step 1303, a feasible solution is calculated when the system composed of the objective function and the group of linear inequalities is viewed as a linear programming problem. Since the feasible solution specifies a plane on which the polygon currently being processed is to be laid, the vertex position of the polygon can be determined by the principle of perspective transformation. This is step 1304. In step 1305, the embedding process into the virtual space is completed.

FIG. 14 shows a processing result (plan view) of the polygon reconstruction processing in this embodiment. FIG. 10 or FIG.
The same symbols as in FIG. 2 indicate the same as in FIG. 10 or FIG.

As shown in this figure, the polygon data required for three polygons is replaced by two without changing the output image data 107 by the polygon reconstruction processing in this embodiment. You can see that.

FIG. 15 is a block diagram of the apparatus according to this embodiment. The same symbols as in FIG. 4 represent the same as in FIG. Hereinafter, portions different from FIG. 4 will be described.

The object image data generation program 306 is
The object image data 10 is obtained by using the shield data 502, the common data 102, the object data 103, and the user input 104 input from the user command input unit 304.
6 is performed. Therefore, in the case of the present embodiment, it is not necessary to store the polygon data 1010 in the auxiliary storage device 303.
Is stored in the auxiliary storage device 303.

As described above, according to the second embodiment, the image quality is degraded in order to reduce the amount of polygon data used for the object image data generation processing within the range where the same output image data can be obtained. The frame rate can be improved without using the same. Alternatively, when maintaining the same frame rate, the data amount of the polygon data belonging to the object data can be increased by the decrease in the data amount of the polygon data belonging to the background data, so that the image quality can be improved. it can.

In the above embodiment, the background data 101
Is provided with polygon data 1010, but the present invention can also be applied to a case where background data 101 has depth maps instead of polygon data as data representing a three-dimensional structure. That is, the present invention can be applied to a case where a three-dimensional structure is restored from a real image by a technique such as stereoscopic vision.

In this case, each pixel of the depth map is considered to be a quadrilateral polygon existing at a corresponding position in the virtual world, and if necessary, the shielding object data 502 may be generated after being divided into triangular polygons.

In this case, the background image data 105
If it is necessary to generate the data, a known technology may be used. Further, in the above-described embodiment, the object data 103 is read from the auxiliary storage device 303. However, in the present invention, part or all of the object data 103 may be read via the communication unit.
That is, the present invention can be applied to a game via a communication unit, a remote conference system, and the like.

When the present embodiment is applied to a remote conference system, for example, an approximate position for synthesizing a user in a remote place is determined, and a range in which the user can move is defined as a polyhedron including the range. Then, the shielding object data can be generated by the effective polygon selection processing.

[0096]

According to the present invention, when the viewpoint data is known, the appearance changes according to a command from the user.
The G object can be synthesized with high-quality image data generated in advance without causing inconsistency in the three-dimensional structure, and new image data can be generated. That is, it is possible to interact with the user while presenting high-quality image data to the user.

[Brief description of the drawings]

FIG. 1 is a data flow diagram in a first embodiment of an image data generation method of the present invention.

FIG. 2 is a conceptual diagram of a first embodiment of the image data generating method of the present invention.

FIG. 3 is a flowchart of an object image data generation process in the first embodiment of the image data generation method of the present invention.

FIG. 4 is an apparatus configuration diagram in a first embodiment of an image data generation method of the present invention.

FIG. 5 is a data flow diagram in a second embodiment of the image data generating method of the present invention.

FIG. 6 is a flowchart of an effective polygon selection process in a second embodiment of the image data generation method of the present invention.

FIG. 7 is a conceptual diagram of an effective polygon selection process in a second embodiment of the image data generation method of the present invention.

FIG. 8 is a flowchart of a polygon reconstruction process in the second embodiment of the image data generation method of the present invention.

FIG. 9 is a flowchart of a polygon grouping process in the second embodiment of the image data generating method of the present invention.

FIG. 10 is a conceptual diagram of a polygon grouping process in a second embodiment of the image data generating method of the present invention.

FIG. 11 is a flowchart of a reconstruction process in an image plane according to a second embodiment of the image data generation method of the present invention.

FIG. 12 is a conceptual diagram of a reconstruction process in an image plane according to a second embodiment of the image data generation method of the present invention.

FIG. 13 is a flowchart of an embedding process into a virtual space in a second embodiment of the image data generating method of the present invention.

FIG. 14 shows a processing result of a polygon reconstruction process in the second embodiment of the image data generating method of the present invention.

FIG. 15 is an apparatus configuration diagram in a second embodiment of the image data generation method of the present invention.

[Explanation of symbols]

 100 virtual world data 101 background data 1010 polygon data 1011 texture data 102 common data 1020 viewpoint data 103 object data 1030 polygon data 1031 texture data 104 user input 105 background image data 106 object image Data 107: Output image data 108: Background image data generation processing 109: Object image data generation processing 110: Image data synthesis processing 501: Shielding object data generation processing 502: Shielding object data

Claims (5)

[Claims] 1. An image generation method for generating image data using a computer, comprising: generating first image data including a content that can be a background of an image; and an object represented by the first image data. Generating first three-dimensional structure data including three-dimensional structure information, generating virtual world data including viewpoint information in image data, the three-dimensional structure information, and texture information; and generating the first three-dimensional structure data and the virtual world data. Can be integrated with the first image data based on an instruction from a user of the computer, and
The second image data is generated by integrating the image with the first image data, the second image data becomes a relationship between the background and the target object, using the virtual world data, the first image data And by integrating the second image data, even when the first image data in the third image data is moved, the correspondence between the first image data and the second image data. Generating a third image data that keeps the same. 2. An image generating apparatus for generating a predetermined image using first image data including a content that may be a background of an image, comprising: Means for generating first three-dimensional structure data comprising: viewpoint information in image data; means for generating virtual world data comprising the three-dimensional structure information and texture information; and the first three-dimensional structure data and the virtual world data Can be integrated with the first image data based on an instruction from a user of the computer, and
Means for generating second image data in which the relationship with the first image data becomes the relationship between the background and the object by being integrated with the image of the first image data, and using the virtual world data, the first image By integrating the data and the second image data, even when the first image data in the third image data is moved, the correspondence between the first image data and the second image data is obtained. Means for generating third image data for maintaining the relationship. 3. The image data generating apparatus according to claim 2, wherein a data amount of said second three-dimensional structure data is smaller than a data amount of said first three-dimensional structure data. . 4. The image data generating apparatus according to claim 2, further comprising: a unit for selecting a part of the first three-dimensional structure data to generate third three-dimensional structure data. An image data generating apparatus, wherein the means for generating the second image data generates the second image data using the third three-dimensional structure data. 5. The image data generating device according to claim 3, wherein the second three-dimensional structure data is polygon data, and at least one polygon included in the polygon data is:
An image data generating device, wherein the polygon is different from a polygon included in the first three-dimensional structure data.