JP3157015B2 - Image processing method and image processing apparatus - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image generating apparatus for generating an image of a three-dimensional shape, and more particularly to a technique of texture mapping for giving a pattern to a three-dimensional shape for generating an image.

[0002]

2. Description of the Related Art When an image of a three-dimensional shape is generated, an image having a pattern on the surface of the three-dimensional shape may be generated in order to realistically represent the three-dimensional shape.

In general, in a process of applying a pattern to a three-dimensional surface, a pattern (hereinafter referred to as "texture") represented as a two-dimensional image is coordinated on the three-dimensional surface. , Texture mapping.

Techniques for such texture mapping include Toshiaki Kato, "Various Expression Methods Used in CG Video Production-Focusing on Mapping", Pixel, Paragraphs 51 to 58, Lance Williams, SIGGRAPH 83 Proceedings, Volume 17, Number 3, July 1983 (Lance William
s, 'Pyramidal Parametrics', SIGGRAPH83 Proceedings,
Vol. 17, No. 3, July 1983) and the like are known.

Further, according to the conventional texture mapping technique, in order to repeatedly use an image generated once, an image created by computer graphic or a photograph is read in advance by an optical device. An image is stored, and this image is used as a pattern. In addition, by using an image created in advance as a pattern as described above, it is possible to quickly generate an image having a three-dimensional shape.

Further, as a technique for giving a pattern to the surface of a three-dimensional shape, Darwin R.P.
RAPH85 Proceedings, Volume 19, Number 3, July 1985 (Darwyn R. Peachey, 'Soli
d Texturing of Complex Surface ', SIGGRAPH85 Procee
dings, Vol.19, No.3, July, 1985) is known.

[0007]

According to the conventional texture mapping technique, an image created in advance is used as a texture. Therefore, if the size of a pattern given to a three-dimensional shape is increased, the image quality of the displayed pattern becomes rough. There was such a problem.

That is, since the resolution of an image created in advance is constant, if the size of the pattern is increased, shaggy or the like occurs, the image quality becomes rough, and the contour lines and the like are not smooth. On the other hand, if a smooth filtering process or the like is performed in order to smooth contour lines or the like, the image quality will be blurred.

This problem can be avoided by increasing the resolution of an image created in advance in accordance with the largest pattern that may be used. However, increasing the resolution causes a problem that the amount of image data increases.

According to the conventional texture mapping technique, image data created in advance is used as a pattern. However, in general, an image has a large amount of data. A problem arises that a huge amount of memory is required for storing the information.

According to the functional mapping technique, it is extremely difficult to represent irregular patterns such as characters and various types of patterns.

Accordingly, the present invention provides an image processing method capable of giving a high quality and various types of patterns of any size to a three-dimensional shape of an image generation target based on a small amount of data. The purpose is to:

[0013]

To achieve the above object,
The present invention relates to an image processing method for generating a photographed image of a subject having a three-dimensional shape which is assumed to be photographed on an assumed camera screen. The resolution of the texture is determined according to the relative positional relationship, a texture having the determined resolution is generated from the texture pattern defined as a vector value function on a two-dimensional plane, and the texture is captured by the camera screen. An image processing method for generating a photographed image of the subject obtained by coordinating the generated texture.

Here, "coordinate" means that a two-dimensional image is attached to a three-dimensional surface.

In a particularly preferred embodiment of the image processing method according to the present invention, in the image processing method according to the present invention, the resolution of the texture is determined based on the relative position between the camera screen and the subject. The projected image of the subject on the assumed camera screen is calculated according to the relative positional relationship, the size of the calculated projected image of the subject is calculated, the resolution of the calculated projected image is calculated, and the calculated projected image is calculated. And an image processing method for determining the resolution equivalent to the resolution of the texture as the resolution of the texture.

[0016]

According to the image processing method of the present invention, a texture is generated from a texture pattern defined as a vector value function on a two-dimensional plane. The vector value function can represent a shape with a smaller data amount than image data represented by pixel data. Therefore, various types of textures can be stored and used. Further, since the shape is defined by the vector, it is possible to cope with an image of an arbitrary resolution without shaggy due to the size of the shape and deteriorating the image quality.

Further, the resolution of the texture is determined according to the assumed relative positional relationship between the camera screen and the subject, and the resolution is determined from the texture pattern defined as a vector value function on a two-dimensional plane. Since a high-resolution texture is generated, a high-quality texture can be coordinated with a subject.

In particular, the resolution of the texture is determined by obtaining a projected image of the subject on the assumed camera screen in accordance with the relative positional relationship between the assumed camera screen and the subject. If the calculated size of the projected image of the subject is calculated, the resolution of the calculated projected image is determined, and the resolution equivalent to the determined resolution of the projected image is determined as the resolution of the texture, the resolution of the texture is insufficient or insufficient. Therefore, it is possible to prevent the occurrence of excessive data and to eliminate the occurrence of unnecessary processing.

[0019]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below.

First, the configuration of the image processing apparatus according to the first embodiment is shown in FIG.

As shown in the figure, the image processing apparatus according to the first embodiment has a three-dimensional scene database 11, a three-dimensional image generator 12, an image database 13, and a display 14. I have.

The three-dimensional image generation device 12 includes a three-dimensional scene input device 121, a three-dimensional scene interpretation device 122, a three-dimensional image generation processing device 123, and a two-dimensional image generation device 12.
It has a four- and three-dimensional image output device 125.

The three-dimensional scene database 11 is a database for storing various parameters relating to the three-dimensional scene generated by the image processing apparatus.

First, the three-dimensional scene database 11 will be described.

Table 1 shows data for generating one three-dimensional scene among the data stored in the three-dimensional scene database 11. FIG. 3 specifically shows a three-dimensional scene represented by the data shown in Table 1.

[0026]

[Table 1]

The data shown in Table 1 includes a command for defining a camera for photographing a three-dimensional scene, a command for defining a shape of a subject, and a command for defining attributes of a subject.
Thus, a three-dimensional scene is specified by specifying a subject, a light source for irradiating the subject, and a camera for photographing the subject.

The subject 111 specified by the data shown in Table 1 is a sphere having a radius of 1 with (0, 1, 0) as the center, as shown in FIG. Has a pattern. The light source 112 is (5,
It is a point light source that exists at coordinates of (5, 5), has a radiation intensity of 1 from the light source, and the color of the light source is (1, 1, 1). The camera 113 exists at the coordinates (0, 5, 0), and faces the negative direction of the y-axis to capture the subject 111. Hereinafter, the space shown in FIG. 3 is called a world space.

The image processing apparatus generates an image of the subject photographed by the camera in the virtual world space as described above. In the present embodiment, the image processing apparatus is independent of the world space for generating the image. , A camera space, a subject space, and a texture space.

FIG. 2 shows the relationship between these spaces.

The space defined by the coordinate system indicated by A in the figure is the camera space, and in the example shown in the figure, the screen of the camera is set on the plane of ZA = 0. Also, it is assumed that the points on the screen where each point in the camera space is projected are known.

The space defined by the coordinate meter indicated by B in the figure is the subject space. In the illustrated example, the center of the sphere as the subject coincides with the origin of the subject space.

A space defined by a coordinate system indicated by C in the figure is a texture space, and in the illustrated example, the texture is set on a plane of ZC = 0. Also, it is assumed that the points on the texture that are mapped to each point in the texture space when they are to be mapped are known.

The space defined by the coordinate system indicated by W in the figure is the world space.

The data shown in Table 1 is described in a language called PostScript. Postscript is written in reverse Polish notation.
That is, a statement including a command is described in the order of a parameter and a command using the parameter. For example, in the statement 1105 shown in Table 1, "001" described first is a parameter indicating blue, and "color" is a command to change the color displayed to blue indicated by the parameter.

The data shown in Table 1 will be described in the order described.

HRBegin of statement 1101
Declares the start of image generation. Statement 110
2 / emboss is the name of the "procedure" for generating the pattern.

"/" Following / emboss
The procedure between "@", that is, between statements 1102 and 1108, is a procedure for generating the pattern of the subject.

The statement 1103 is a parameter characterizing the pattern. Statement 1104-1
Reference numeral 107 denotes a statement for generating a pattern.

Reference numeral 1104 denotes a statement in which the type of character to be displayed is set to gothic and the size of the character is set to 0.02. Statement 1105 is a statement that defines the color of the character to be displayed as blue. Statement 1106 is a statement defining that the position of the character string to be displayed is displayed at the position (0.5, 0.5) in the two-dimensional coordinate system.

Statements 1109 to 1115 are statements defining the camera 113. Statement 1109 is a statement that defines the resolution of the generated image. Statement 1110 is a statement that defines the aspect ratio of the camera frame.
Statement 1111 is a statement that defines the size of the screen that captures the subject. A statement 1113 is a statement for defining a method of projecting a subject onto a screen for capturing a subject, and the like. Statement 1114
And statement 1115 are statements defining the camera arrangement.

The statements 1116 to 1122 are statements for defining a subject. Statement 1116 is a statement declaring the start of the definition of a subject with respect to a three-dimensional scene.
Statement 1117 is a statement that defines a light source that illuminates a three-dimensional scene. Thus, the light source is also defined in the definition section of the subject. Statement 1117 is a statement that defines point light source 112. The parameters of statement 1117 are light source intensity, light source color, and light source position. Statement 1
Reference numeral 118 denotes a statement for defining the color of the subject 111. A statement 1119 is a statement that defines the pattern of the subject 111 to be a pattern / emboss defined in advance. A statement 1120 is a statement for defining the position of the subject 111.
Statement 1121 is a statement that defines the subject 111 as a sphere having a radius of one. Statement 1122 is a statement for declaring the end of the definition of the subject regarding the three-dimensional scene.

A statement 1123 is a statement for declaring the end of all parameter definitions for a three-dimensional scene.

The three-dimensional image generating device 12 reads data on the three-dimensional scene described above from the three-dimensional scene database 11 and generates an image of the three-dimensional scene. Then, the generated image is displayed on the display 14. Alternatively, the generated image is stored in the image database 13.

Hereinafter, an image generating operation of the three-dimensional scene generating apparatus will be described.

The three-dimensional scene input device 121 reads the data of the three-dimensional scene database 11 for each character string, and sends the read character string to the three-dimensional scene interpretation device 122.

Here, the configuration of the three-dimensional scene interpretation device 122 is shown in FIG.

As shown, the three-dimensional scene interpretation device 12
Reference numeral 2 denotes a character string determination device 1220, a camera attribute setting process group 1221, a subject attribute setting process group 1222, a shape setting process group 1223, a coordinate conversion process group 1224, a mode control process group 1225, a camera attribute table 1226, and a subject attribute table 1227. , Result table stack 1228,
It is composed of a coordinate conversion stack 1229.

Here, the camera attribute setting processing group 1221
Is a set of processes corresponding to the statements 1109 to 1115 in the data shown in Table 1, respectively. The process of setting the camera attribute specified by the parameter in the statement in the camera attribute table 1226 is shown in FIG. It is a set.

The subject attribute setting processing group 1222 includes statements 1117 to 1120 in the data shown in Table 1.
Is a set of processes corresponding to the respective processes, and is a set of processes for setting subject attributes in the subject attribute table 1227.

The coordinate conversion processing group 1224 includes statements 1114, 1115, and 112 in the data shown in Table 1.
0 is a set of processes respectively corresponding to 0, and is a set of processes for setting the definition of coordinate conversion in the coordinate conversion stack 1229.

The mode control processing group 1225 includes statements 1101, 1116, and 112 in the data shown in Table 1.
2, a set of processes respectively corresponding to 1123,
This is a process for controlling the process of setting the current camera attribute and the process of setting the subject attribute. The camera attribute table 1226 has a resolution 1226 as shown in FIG.
1, pixel aspect ratio 12262, screen window 12263, lettering area 12264, projection method 12265 on screen, camera space 12268
Items.

As shown in FIG. 6, the subject attribute table 1227 is a table for defining all attribute parameters of the subject. As shown in FIG. 6, the subject attribute table includes a color 12271, a light source 12272, an array 12273 of procedures for defining a pattern, a “procedure” parameter 12274,
Texture mapping space 12275, gloss 12276,
And subject coordinates 12277.

Upon receiving the character string from the three-dimensional scene input device 121, the character string determination device 1220 determines whether the received character string is a parameter to be passed to the command or a command. If the received character string is a parameter, the parameter is stacked on the result table stack 1228.

On the other hand, if the received character string is a command, if the received character string is a command in the camera attribute setting processing group 1221, the subject attribute setting processing group 1222, the shape setting processing group 1223, the coordinate conversion processing group 1224, or the mode control processing group 1225, Execute the processing corresponding to the command.

When "@" is input, the character string judging device 1221 does not execute processing even if the command is a command until the "@" is input. Interpreted result table stack 122
Stack on 5.

Hereinafter, the operation of the three-dimensional scene interpretation device 122 that has received the data shown in Table 1 will be described.

First, when the statement 1101 is received, the character determination device 1220 determines that it is a command, and the mode control processing group 122 corresponding to the statement 1101.
5 is executed. The processing corresponding to the statement 1101 initializes the three-dimensional generation processing device and sets the mode to the camera attribute setting mode. At this time, the coordinate transformation stack 1229 stores a space transformation matrix for transforming the camera space into world space coordinates. In the present embodiment, each coordinate axis ZA, YA, XA in the camera space
Are the corresponding coordinate axes ZW, YW, XW of the world space.
Is stored (see FIG. 2).

Next, the character judgment device 1220 stores the input statements 1102 to 1108 as parameters in the result table stack 1228.

When the statemant 1109 is input, the character determination device 1220 stores the parameters “640, 480, 1.0” described in the first half in the result table stack 1228. Then, the processing in the camera attribute setting processing group 1221 corresponding to the command “HRFormat” described in the latter half is started. HRFormat "
After confirming that the mode is in the camera attribute setting mode, the image resolution 12261 of the camera attribute table 1226 is changed to the statement 1109 stored in the result table stack 1228 by the character string determination device 1220. The three parameters 640, 480, 1.0
Set to.

A subsequently input statement 1110
To 1113, the pixel aspect ratio 12262 of the camera attribute table 1226, the screen window 12263, the lettering area 122
64, each item of the projection method 12265 on the screen is set.

Subsequently, when a statement 1114 is input, the processing in the coordinate conversion processing group 1224 corresponding to the statement 1114 is performed, a conversion matrix as shown in the following equation is created, and stored in the coordinate conversion stack 1228. The product is multiplied by the camera space transformation matrix, and the result is stored in the coordinate transformation stack 1228 as a transformation matrix.

[0063]

(Equation 1)

Here, θ is 90.0 of the first parameter of the statement 1114, and x is 1.0.0 of the second parameter.
0 and y are 0.0 of the third parameter, and z is 0.0 of the fourth parameter.

Subsequently, when the statement 1115 is input, the processing in the coordinate conversion processing group 1224 corresponding to the statement 1115 is performed, and in the equation 1, the first parameter 0.0 of the statement 1115 is x, the second parameter is Create a transformation matrix with 5.0 as y and the third parameter 0.0 as z, take the product with the transformation matrix in the coordinate transformation stack 1228, and use the resulting transformation matrix as a new transformation matrix. Is stored in the coordinate conversion stack 1229.
This conversion matrix is a conversion matrix for converting the camera space into a space at a position indicated by a in FIG. 2 with the camera position in the world space as the origin.

Subsequently, when a statement 1116 is input, the character determination device 1220 executes the processing in the mode control processing group 1225 corresponding to the command 1116,
Camera space 12268 of camera attribute table 1226
Then, the conversion matrix stored in the coordinate conversion stack 1229 is set, and the mode is changed to the subject setting mode. At this time, the coordinate conversion stack 1229 stores an object space conversion matrix for converting the object space into world space coordinates. In the present embodiment, each coordinate axis Z in the subject space
B, YB, and XB are the corresponding coordinate axes Z of the world space.
A linear subject space conversion matrix that is converted into W, YW, and XW is stored (see FIG. 2). The coordinate conversion stack 1229 also stores a texture space conversion matrix for converting the texture space into world space coordinates. In the present embodiment, a linear texture space conversion matrix is stored such that each coordinate axis ZC, YC, XC in the texture space is converted into each corresponding coordinate axis coordinate ZW, YW, XW in the world space (see FIG. 2). .

Next, the character determination device 1220 stores the input parameters in the statement 1117 in the result table stack 1228. After confirming that the mode is in the subject attribute setting mode, the process in the subject attribute setting processing group 1223 corresponding to the command in the statement 1117 is executed, and the subject attribute table 1 is set.
The parameters in the statement 1109 once stored in the result table stack are set to the light source 12272 of 227.

Similarly, when a statement 1118 is input, the character determination device 1220 executes a process corresponding to the statement 1118 in the subject attribute processing group 1222, and
1 in the statement 1118 once stored in the result table stack 1228.
1.0 "is set.

Next, when the statement 1119 is input, the character judging device 1220 firstly sets "emboss" as the first parameter and "emboss" as the second parameter.
Km0.5 ”is stored in the result table stack 1228. Then, the command“ HR Texture ”is stored.
Are executed in the subject attribute processing group 1222 corresponding to. That is, the first parameter stored in the result table stack 1228 is determined, and the pattern definition “procedure” 12274 of the subject attribute table 1227 is first added together with the same character string “/ emboss” as the first parameter. The result is changed to the arrangement of the statements 1102 to 1108 stored in the table stack 1228.

The second parameter of the result table stack 1228 is a procedure parameter 1227 as a parameter.
4 is stored.

Then, the texture mapping space 122
At 75, the texture space conversion matrix stored in the coordinate conversion stack 1229 is set.

Subsequently, when the statement 1120 is input, the processing in the coordinate transformation processing group 1224 corresponding to the statement 1114 is performed, and in the equation 1, the first parameter 0.0 of the statement 11205 is set to x, and the second parameter A conversion matrix in which 1.0 is y and the third parameter 0.0 is z is created, the product of the conversion matrix with the object space conversion matrix in the coordinate conversion stack 1228 is obtained, and the resulting conversion matrix is used as the object conversion matrix. Coordinate transformation stack 1229 as matrix
To be stored. This subject conversion matrix is a conversion matrix for converting the subject space into a space at a position indicated by b in FIG. 2 with the origin of coordinates (0, 1, 0) in the world space.

Next, when the statement 1121 is input, the character determination device 1220 outputs the command “HRSph”.
The processing in the shape setting processing group 1223 corresponding to “ere” is performed, and the subject conversion matrix stored in the coordinate conversion stack 1229 is set in the subject space 12277 of the subject attribute table 1227.

Then, the camera attribute table 1226, the subject attribute table 1227, the result table stack 12
The parameters in the statement 1121 and the shape identifier representing the sphere are sent to the three-dimensional image generation processing device 123.

Next, when the statement 1122 is input, the character string determination device 1220 sets the mode to the camera attribute setting mode by the processing in the corresponding mode control processing group 1225.

When the statement 1123 is input,
The character string determination device 1220 terminates the processing of the three-dimensional scene interpretation device by the processing in the corresponding mode control processing group 1225.

Next, the camera attribute table 1226 and the subject attribute table 122
7 and the three-dimensional image generation processing device 123 that has received the parameters stored in the result table stack 1228 and the shape identifier representing the sphere will be described.

The three-dimensional image generation processing device 123 performs three-dimensional processing based on the shape identifier sent from the three-dimensional scene interpretation device 122, the parameters of the statement 1121, and the contents of the camera attribute table 1226 and the subject attribute table 1227. Create a three-dimensional image on the three-dimensional image memory 1236,
This is a device that outputs the contents of the three-dimensional image memory 1236 to the three-dimensional image output device 125.

FIG. 7 shows the configuration of the three-dimensional image generation processing unit 123.

The three-dimensional image generation processing device 123 includes a projection processing device 1231, a pattern setting processing device 1232, a three-dimensional sampling processing device 1233, a texture mapping processing device 1234, a pattern image memory 1235, and a three-dimensional image memory 1236. .

In the three-dimensional image generation processing device 123,
The projection processing device 1231 performs a projection process for obtaining a projection image of a subject on a camera screen.

The projection process is performed as follows.

First, the projection processing device 1231 interprets the parameters stored in the result table stack 1228 based on the shape identifier sent from the three-dimensional scene interpretation device 122.

In this embodiment, statement 11 in Table 1 is used.
21 in the result table stack 1228 corresponding to 1.
The parameters 0, 1.0, -1.0, 360.0 are stored. When the received shape identifier indicates a sphere, the projection processing device 1231 sets the first parameter stored in the result table stack 1228 to a radius, the second parameter to a lower cut plane value, and the third parameter to an upper cut plane value. , The fourth parameter is interpreted as a sweep angle.
The subject sphere interpreted in this way is a sphere centered on the origin in the subject space (see FIG. 2B).

Therefore, the position of the sphere in the object space on the camera space is obtained, and the projected image of the object on the camera screen is obtained.

That is, the object conversion matrix stored in the object space 12276 of the object attribute table 1227 is applied to the sphere to convert the parameters of the sphere into the parameters in the world space. The parameters of the sphere are obtained by applying the inverse of the camera transformation matrix stored in the camera space 12266 of the camera attribute table 1226 to the sphere parameters. However, the subject conversion matrix stored in the subject space 12276 of the subject attribute table 1227 and the camera space 12
The product of the sphere and the inverse of the camera transformation matrix stored in 266 may be applied to the sphere to directly convert the parameters in the subject space to the parameters in the camera space.

When the parameters of the sphere in the camera space are obtained in this manner, the parameters are projected onto the camera screen in the camera space, thereby obtaining the camera screen of the object.
The parameters of the projected image on the screen can be obtained. The parameters of the projected image of the sphere projected on the camera screen are circular parameters.

On the other hand, the pattern setting device 1232 determines the resolution of the pattern.

The resolution of the pattern is determined as follows.

First, the area of the projected image of the subject projected on the camera screen in the camera space determined by the projection processing device 1231 is determined. Then, the resolution of the pattern is determined based on the size of the obtained area, and the subject attribute table 1 is determined.
The obtained resolution of the pattern is transmitted to the two-dimensional image generation device 124 (see FIG. 1) together with the “procedure” array 12273 for generating the pattern stored in the 227.

A method for obtaining the pattern resolution will be described with reference to FIG.

FIG. 8 illustrates a camera screen.

Resolution 12 of camera attribute table 1226
According to the content of H.261, the size of the camera screen 113 is 640 (horizontal) × 480 (vertical).

As shown in FIG. 8, the coordinate system of the screen is defined by a coordinate system having the upper left as the origin and the lower right as (640, 480).

At this time, the projected image 111B of the subject sphere is
It is obtained from the parameters of the circle previously obtained by the projection processing device 1231.

Next, when a rectangular area including the area where the sphere is projected (the area indicated by the wavy line in the figure) is obtained, the vertices on the diagonal are the areas of (272, 192) and (368, 288).

Therefore, the area occupied by the projected sphere is a 96 × 96 area.
Calculates the resolution of the pattern from the obtained area by Expression 2.

Tres = region × 2−2 Expression 2 where Tres is the resolution of the pattern to be obtained, and 2 of the occupied region region of the rectangular area shape including the projected image.
That is, the resolution is 192 in the vertical direction and 192 in the horizontal direction.

According to Nyquist's theorem, the pattern of the subject is
It is sufficient if the image has a spatial frequency twice as high as the spatial frequency determined by the screen resolution (that is, an image having the resolution). Therefore, if the resolution of the pattern is obtained by Equation 2, the pattern is sharp and excessive. An image having a sufficient resolution can be generated.

The pattern setting processor 1232 compares the resolution thus obtained together with the “procedure” 12273 and “procedure” parameters of the pattern of the subject attribute table 1227 by two.
Hand over to the two-dimensional image generation device.

Even if the resolution of the pattern is simply obtained from the size of the subject and the distance between the subject and the camera screen, a value close to the appropriate resolution can be obtained. Is not required, the size of the subject and the distance between the subject and the camera screen
The resolution of the pattern may be simply obtained.

Here, the three-dimensional image generation processing device 12
3, the pattern generation operation of the two-dimensional image generation device 124 will be described.

First, FIG. 9 shows the configuration of a two-dimensional image generation device.

As shown, the two-dimensional image generation device 12
4 is a data determination device 1240, a two-dimensional attribute setting processing group 1241, a character development processing 1242, and an image development processing 124.
It comprises a three-dimensional sampling processing device 1244, a two-dimensional graphic attribute table 1245, a work memory 1246, a parameter stack memory 1247, a character font database 1248, and an image database 1249.

The pattern “procedure” 12273 of the subject attribute table 1227 received by the two-dimensional image generation device 124 includes the statements 1103 to 1103 in the data shown in Table 1.
1107 is stored. Therefore, when the data determination device 1240 of the two-dimensional image generation device 124 receives the array 12273 of the pattern “procedure” of the subject attribute table 1227, it reads the character strings in the array in order and determines whether the parameter is a parameter to be passed to the command. , Command
If it is a parameter, the parameter stack memory 124
7 is stored. The “procedure” parameter 12274 of the subject attribute table 1227 is stored in the two-dimensional graphic attribute table 1245.

In the case of a command, a process corresponding to the command read in the two-dimensional attribute setting processing group 1241, the character development processing group 1242, or the image development processing group 1243 is executed.

When a statement 1103 is input from the pattern “procedure” 12273 of the subject attribute table 1227, the data judging device 1240 makes the statement 1
After storing the parameter “Km1.0” in the parameter 103 in the parameter stack memory 1247, the command “para”
The processing in the two-dimensional attribute setting processing group 1241 corresponding to “mdef” is executed, and the parameter “Km1.0” 12274
Is stored in the two-dimensional graphic attribute table 1245.

Statements 1104, 1105 and 11
Similarly, for 06, the type of character font, font size, character color, and character arrangement position, which are parameters in each statement, are set in the two-dimensional graphic attribute table 1245.

In this example, Table 1 shows that the font type is Gothic, the font size is 0.02, and the starting position of the character string is 0.5 or 0.5.

Next, when the statement 1207 is input, the parameter “Aio” of the statement 1207 is stored in the parameter stack memory 1247, and then a character expansion processing group corresponding to the command “show” is performed. 1
The processing in 242 is performed.

The processing in the character expansion processing group 1242 is based on the character codes constituting the character strings stacked in the parameter stack memory 1247 and the two-dimensional graphic attribute table 124.
This is a process of extracting character outline data from the character font database 1248 based on the font type 5 and converting the character outline data into character outline data in the pattern definition space.

The character outline data is a vector value function indicating a curve constituting the character outline. A vector-valued function representing a two-dimensional curve is generally expressed as Pu = ｛X (u), Y
(U)}, for example, a vector value function representing a circle having a radius of 1 is Pu = {cos (u), sin (u)}. The shape defined by such a vector-valued function can be expanded without shaggy or the like. For example, a vector value function Pu = {cos (u), sin (u)} that defines a circle having a radius of 1 centered on the origin.
Thus, a vector value function Pu = ＰKco representing a circle having a radius K
s (u), Ksin (u)}, and this vector value function Pu = {Kcos (u), Ksin
(U) If a curve is drawn according to｝, a circle having a radius K can be obtained without occurrence of shaggy.

As a method for defining a shape using such a vector value function, a method based on the Bezier method and a method based on the spline method are known.

The pattern definition space has an origin (0.0, 0.0)
And a coordinate (1.0, 1.0) as a diagonal vertex. .

Conversion to character outline data in the pattern definition space is performed as follows.

That is, the three-dimensional image generation processing unit 123
The character outline data extracted from the character font database 1248 received is converted into a size of 0.02 for each character font in the pattern definition space, and the position of each character is sequentially set from coordinates (0.5, 0.5). Conversion is performed so that the position is shifted horizontally by the size of the font. The converted character outline data in the converted pattern definition space is also a vector value function. The converted character outline data in the pattern definition space is stored in the work memory 1246.

The character outline data in the pattern definition space is
After being stored in the memory 1246, the two-dimensional sampling processor 1244 expands the character outline data in the pattern definition space stored in the work memory 1246 into the texture space, and determines the pixels surrounded by the character outline first. Then, a process of coloring with the color attribute of the character stored in the two-dimensional graphic attribute table 1245 is performed.

The expansion of the character outline data in the pattern definition space into the texture space is performed as follows.

First, based on the resolution, vertical 192, and horizontal 182 received from the three-dimensional image generation processing device 123, the vertices of the diagonal line are set to (0.0, 0.0) on the XC-YC plane of the texture space (see FIG. 2). 0.0) and (192.0, 19
2.0) is assumed.

Then, the character outline data in the pattern definition space is converted into character outline data in the texture space. The conversion is performed by a linear conversion that converts the pattern definition space so that the pattern definition space and the rectangular image area assumed on the texture space overlap as much as possible. In the case of the present embodiment, this conversion is a simple enlargement conversion.

The XC-YC plane of the texture space is set in the work memory based on the character outline data in the texture space thus obtained, and the XC-Y
The value of the pixel on the work memory at the point in the area defined by the character outline on the C plane is stored in the two-dimensional graphic attribute table 12 first.
The setting is made according to the color attribute of the character stored in 45.

FIG. 11 is an image in which the “procedure” of the pattern defined by Table 1 is generated by the two-dimensional image generating apparatus. The generated image has a resolution of 192 (vertical) × 192 (horizontal). As shown in the figure, a character string “Aioe” defined horizontally is displayed from 96 positions to 96 positions.

The description of the two-dimensional image generation device 124 has been completed, and the description returns to the three-dimensional image generation processing device 123.

The three-dimensional image generation processing device 123
The dimensional sampling processing device 1233 generates an image of a projected image of the patterned subject on the camera screen on the three-dimensional image memory.

FIG. 12 shows the processing procedure of the three-dimensional sampling processor 1233.

First, the three-dimensional sampling processor 123
3 sets a camera screen plane in the camera space on the three-dimensional image memory 1236.

Next, the points inside the projected image of the subject sphere on the camera screen are determined from the parameters of the previously determined circle on the camera screen.

Then, the following processing is performed for each point (hatched portion in FIG. 6) inside the obtained projected image on the camera screen 12331. A point inside the projection image on the camera screen which is to be processed now is called a processing point.

First, the point on the subject projected on the processing point is determined 12332. In this process, the position of the subject in the camera space may be obtained in the same manner as when the projection image is obtained, and the point on the subject projected to the processing point may be obtained.

Next, the position in the texture space of the obtained point on the object in the camera space is calculated 12333. This processing is performed in the camera space 1 of the camera attribute setting table 1226.
The inverse matrix of the camera conversion matrix set in 2266 and the texture mapping space 12 in the subject attribute table 1227
The product is obtained by calculating the product of the matrix and the texture space conversion matrix set in 274, and converting the point on the subject in the camera space by the obtained matrix. The determined position exists in the texture space.

Processing 12334 is processing for activating the texture mapping processing device 1234. When the texture mapping processing device 1234 is started, the process 12334 is executed.
Gives the position in the texture space of the point on the subject determined in step 12333.

Texture mapping processor 1234
Calculates the color of the pattern assigned to the point on the subject from the position in the texture space of the point on the subject passed from step 12334. The obtained color is the color of the point on the camera screen where the point on the subject is projected.

The processing for obtaining the color of the pattern to be added to the point on the subject from the position in the texture space is performed as follows.

First, the XC coordinate and the YC coordinate of the position of the obtained point on the subject in the texture space are extracted. The extracted XC coordinates are Q, and the YC coordinates are P. The XC of the texture space set on the work memory 1246
-Coordinates on the YC plane (XC = Q, YC = P, ZC = 0)
The value of the pixel on the work memory corresponding to is obtained. Then, the obtained pixel value is set as the color of the pattern added to the point on the subject.

In the above processing, if the pattern area on the XC-YC plane, the range of the XC coordinates where the subject exists, and the range of the YC coordinates are displaced in the texture space, a point on the subject is determined. No pattern will be added. Therefore, in such a case, the texture mapping processing device 1234 moves the position of the point on the subject in the texture space, and the XC coordinate range and the YC coordinate range of the pattern area on the XC-YC plane are: In each case, the color of the pattern attached to a point on the subject is determined so as to fall within the range of the XC coordinates and the range of the YC coordinates where the subject exists.

Alternatively, in the data shown in Table 1, a stage for designating a position in the world space where the texture space is transformed is specified.
The three-dimensional scene interpretation device 122 may set a transformation matrix created in accordance with the statement in the texture mapping space of the subject attribute table 1227.

In the above processing, the position of the point on the subject in the ZC direction in the texture space is not taken into consideration, so that the pattern is slightly distorted and is coordinated with the subject. This distortion is corrected by post-processing.

Now, the texture mapping processing device
The color of the point on the subject thus obtained is passed to the processing 12335.

The processing 12335 uses the color returned from the texture mapping processing unit 1234 as the color of the processing point,
The value of the corresponding pixel on the camera screen plane on the three-dimensional image memory 1236 is set to the determined color.

When the above processing has been completed for each point in the projection image on the camera screen, the three-dimensional output device 125 is activated and the processing is terminated.

The three-dimensional output unit 125 outputs the image of the camera screen plane created in the three-dimensional image memory 1236 in the three-dimensional image generation processing unit 123 to the external image database 13 and the display device 14.

By executing the above-described processing, it is possible to display the pattern of the subject regarding the three-dimensional scene.

In this embodiment, a sphere is used as a subject in a three-dimensional scene for the sake of simplicity. However, other shapes can be handled in the same manner.

In the present embodiment, a subject whose shape is defined by parameters has been described. However, the same processing can be performed for a subject whose shape is directly defined by three-dimensional dot data.

In this embodiment, the processing speed is prioritized, and the texture is coordinated on the object without considering the position of the point on the object in the ZC direction in the texture space. When the processing speed is not so important, it is necessary to perform the coordination in consideration of the position of the point on the subject in the ZC direction in the texture space and the curved shape of the subject to reduce the distortion of the pattern. desirable.

A video production system using the image processing apparatus described in the first embodiment will be described below as a second embodiment.

The video production system is a system for producing a video such as an animation in response to a user's instruction.

First, FIG. 13 shows the configuration of the video production system according to the second embodiment.

As shown in the figure, the image production system according to the present embodiment includes a three-dimensional scene production system 21 for creating and editing a three-dimensional scene, a three-dimensional image generation device 22, and an image for recording and editing the created image. An editing system 23, a mouse 25, a keyboard 26, an image scanner 27 for capturing images such as photographs, and a three-dimensional scene database 28.
And character font database 29 and image database 3
0 and a display device 31.

The three-dimensional scene production system 21 includes a shape editing system 211 for creating and editing a shape, an attribute definition system 212 for defining attributes such as an optical attribute and a pattern in the created shape, and creating a pattern of a subject. The system includes a paint system 213 for editing and an operation definition system 214 for defining an operation in the created shape.

The three-dimensional image generating device 22, the three-dimensional scene database 28, the character font database 29, the image database 30, and the display device 31 constitute the image processing device according to the first embodiment.

An outline of the operation of the video production system will be described.

The three-dimensional scene production system 21 displays an input screen on the display 31, and accepts input of the subject, the motion of the subject, the pattern of the subject, and the like.

The user has a keyboard 26, a mouse 25,
From the image scanner 27, the subject, the motion of the subject, the pattern of the subject, and the like are input.

The three-dimensional scene production system 21 creates the data shown in Table 1 according to the first embodiment in accordance with the input contents, and writes the data in the three-dimensional scene database 28.

The three-dimensional image generation device 22 reads the three-dimensional scene database 28 and interprets the contents of the three-dimensional scene database to create an image. Then, the three-dimensional image generation device 22 stores the created image in the image database 30. Alternatively, the created image is transferred to the close video editing system 23.
Send to

[0157] The video editing system 23 edits the image database 30 or the image sent directly to generate an animation or a final image.

Next, the operation of the three-dimensional scene generation system will be described in detail.

When activated, the three-dimensional scene production system 21 displays an input screen shown in FIG.

In FIG. 14, an area 2111 is an area for inputting an arrangement of a subject.

The area 2112 is used for the shape editing system 211.
This is an area for receiving a start instruction. An area 2113 is an area for receiving a start instruction of the attribute defining system 212 for a subject. An area 2114 is an area for receiving a start instruction of the operation definition system 213. An area 2115 is an area for receiving an instruction to start processing for canceling the input processing. An area 2116 is an area for receiving the start of the input work end processing. When these areas are picked by the mouse 25, the system corresponding to the specified area is activated.

In FIG. 14, the shapes 2117 and 2118
Is the input three-dimensional shape.

The shape editing system 211 is a system for defining and editing a three-dimensional shape. Shape editing system 21
1 accepts a secondary curved surface, a free-form surface defined by a control point, or the like as the shape of the subject. When the processing of the shape editing system 211 is completed, the screen returns to the input screen of the three-dimensional scene generation system 21 shown in FIG.

The attribute definition system 212 is a system that displays the input screen shown in FIG. 15 on the display 31 and accepts input of color, gloss, and pattern parameters as optical attributes of the subject. In FIG. 15, a region 2121, a region 2122, and a region 2123 are sliders for receiving color parameter settings. An area 2124 is a slider for receiving setting of a gloss parameter. The user moves the position of the slider by moving up and down while picking the area of the slider with the mouse 25. The attribute definition system 212 receives, as a parameter, a value determined according to the position of the moved slider.

An area 2125 is an area for receiving activation of the paint system 214 for setting a pattern. Region 212
When 5 is picked by the mouse 25, the attribute definition system 21 passes the processing to the paint system. The paint system 214 performs the processing described below, and returns the pattern data to the attribute definition system 212. The attribute definition system 212 receives the returned pattern data as a pattern attribute. An area 2126 is an area for receiving activation of processing for canceling the set attribute. An area 2127 is an area for accepting activation of processing for terminating the attribute definition system. . When the process of the attribute definition system 212 is completed, the screen returns to the input screen of the three-dimensional scene generation system 21 shown in FIG.

The paint system 214 is a stem that displays the input screen shown in FIG. 16 on the display 31 and receives input of a two-dimensional figure, image, or character as a pattern of a subject.

In FIG. 16, an area 2141 is an area where the user performs an input operation. Processing such as movement, editing, and deformation of characters, figures, and images input on this area is performed. An area 2142 is an area for accepting activation of a process of taking in an image such as a photograph by the image scanner 27 and a process of taking out an image from the image database 30. An area 2143 is an area for receiving activation of a process for receiving input of a character string or a character. An area 2144 is an area for receiving activation of processing for defining a two-dimensional graphic. Region 214
Reference numeral 5 denotes an area for accepting activation of processing for canceling the contents of the input in the paint system 214 and returning to the attribute definition system. This is an area for accepting activation of a process for terminating the paint system 214 and returning to the attribute definition system. An area 2146 is an area for starting a process of returning to the attribute definition system when all input operations have been completed.

Now, in the paint system 214,
The input figures, characters, and images are registered in the order in which they were generated. At this time, for the graphic, the vector data and the attribute of the graphic are directly registered. For a character, the attribute of the character and the code of the character are registered. For the image, the pixel information is stored in the image database 27, and the name and attribute of the stored image are registered. Paint system 21
No. 4 returns the registered character, graphic, and image data to the attribute definition system 212 when the area 2146 is picked, and ends the processing. In addition, the character 2147 in FIG.
148 is a figure. It is. Character string is keyboard 26
To enter. The figure is input by the mouse 25. The figure is a rectangle, a circle, a free curve or the like.

The operation definition system 213 defines
Accepts dimensional shape actions. The operation is input by operation definition data that defines the operation.

[0170] Such a three-dimensional scene production system 21
Is completed, the three-dimensional scene production system 21 analyzes the shape and attributes of the received registered subject while explaining the first embodiment with reference to Table 1. The data in the data format of the three-dimensional scene database is created and stored in the three-dimensional scene database 22, and the process is terminated. When storing the data, the data relating to the pattern is also stored in the database as the “procedure” as described above.

The three-dimensional image generation device 22 reads data stored in the three-dimensional scene database 28 and generates an image. At this time, if the pattern is an image, the data read from the three-dimensional scene database 28 contains only the information on the name of the definition of the image. Therefore, the image database 30 is searched based on this name, and the image is fetched. I do. Characters and graphics are processed in the same manner as the image spine device of the first embodiment. Then, the finally generated image is stored in the image database 30 or the image editing system 23
To end the process.

The video editing system 23 is a system for editing an image created by the three-dimensional image generating device 22 and creating an animation.

FIG. 17 shows the configuration of the video editing system 23. As shown, the video editing system 23 includes a VTR
It comprises an editing system 231, an audio editing system 232, a frame buffer 233, and a VTR 234.

The frame buffer 233 is a memory for temporarily storing images. The VTR 234 is a device that records an image. The VTR editing system 231 captures the images from the image database 30 into the frame buffer 233 and sequentially stores the captured images in the
Record on TR234.

The audio editing system 232 is a system for adding sound to the video recorded on the VTR 234. The audio generated by the audio editing system 232 is recorded on the VTR 234 under the control of the VTR recording system 231.

[0176]

As described above, according to the present invention, it is possible to provide a high-quality and various types of patterns in an arbitrary size to a three-dimensional shape of an image generation target based on a small amount of data. It is possible to provide a possible image processing method.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a space used for image generation processing in one embodiment of the present invention.

FIG. 3 is an explanatory diagram showing a method of defining a generated image according to an embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of a three-dimensional scene interpretation apparatus according to an embodiment of the present invention.

FIG. 5 is an explanatory diagram showing a configuration of a camera attribute table used in one embodiment of the present invention.

FIG. 6 is an explanatory diagram showing a configuration of a subject attribute table used in an embodiment of the present invention.

FIG. 7 is a block diagram illustrating a configuration of a three-dimensional image generation processing device according to an embodiment of the present invention.

FIG. 8 is an explanatory diagram showing a projected image of a subject on a camera screen in one embodiment of the present invention.

FIG. 9 is a block diagram illustrating a configuration of a two-dimensional image generation device according to an embodiment of the present invention.

FIG. 10 is an explanatory diagram showing a pattern definition space and a pattern generation space in one embodiment of the present invention.

FIG. 11 is an explanatory diagram showing a pattern generated in one embodiment of the present invention.

FIG. 12 is a flowchart showing processing of a three-dimensional sampling processing apparatus according to one embodiment of the present invention.

FIG. 13 is a block diagram illustrating a configuration of a video production system according to an embodiment of the present invention.

FIG. 14 is an explanatory diagram showing an input screen generated by the video production system according to one embodiment of the present invention.

FIG. 15 is an explanatory diagram showing an input screen generated by the attribute definition system according to one embodiment of the present invention. Of the present invention

FIG. 16 is an explanatory diagram showing an input screen generated by the paint system according to one embodiment of the present invention.

FIG. 17 is an explanatory diagram showing a configuration of a video editing system according to one embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 11 3D scene database 12 3D image generator 13 Image database 14 Display 121 3D scene input device 122 3D scene interpreter 123 3D image generation processor 124 2D image generator 125 3D image output Apparatus 21 Three-dimensional scene production system 22 Three-dimensional image generation device 23 Video editing system 25 Mouse 26 Keyboard 27 Image scanner 28 Three-dimensional scene database 29 Character font database 30 Image database 31 Display device 211 Shape editing system 212 Attribute definition system 213 Paint system 214 Behavior Definition System

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Tsugio Tomita 4026 Kuji-cho, Hitachi City, Ibaraki Prefecture Within Hitachi Research Laboratory, Hitachi, Ltd. (72) Inventor Akihiro Sakamoto 4026 Kuji-machi, Hitachi City, Ibaraki Prefecture Hitachi Research, Ltd. (56) References JP-A-58-219664 (JP, A) JP-A-3-107897 (JP, A) Williams L, "Pyramidal Parametrics", Computer Graphics, Vol. 17, No. 3 (Jully 1983), p1-11 (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 15/00 G06T 11/20

Claims (8)

(57) [Claims] An image processing method for generating a photographed image of a subject having a three-dimensional shape, which is assumed to be photographed on an assumed camera screen, comprising: It is assumed that images are taken on the camera screen according to the relative positional relationship.
The size of the subject to be
Determine the image size of the texture to be coordinated with the object
When a photographed image of a subject is generated, a texture having a determined image size is generated from a texture pattern defined as a vector value function on a two-dimensional plane, and the generated texture is coordinated. An image processing method for generating a photographed image of the subject, which is assumed to be photographed on the camera screen. 2. An image processing apparatus for generating a photographed image of a subject having a three-dimensional shape which is assumed to be photographed on an assumed camera screen, wherein a two-dimensional texture pattern is defined. Texture data storage means for storing a vector value function, subject data storage means for storing subject data for defining a three-dimensional shape of a subject, and designation of a relative positional relationship between the camera screen and the subject. input means, the object data storage means and the object data read from the response to said input means and said relative positional relationship is accepted, the
The size of the subject assumed to be photographed on the camera screen
Size, and coordinate the subject with the size determined.
Determine the image size of the texture to be
When generating the pickup image, the texture data - than the vector-valued function which is stored in the data storage means, the determined image
A texture supply unit for generating an image-size texture;
An image processing apparatus comprising: a texture mapping unit configured to generate a captured image of the subject obtained by coordinating the generated texture, which is assumed to be captured by the camera screen. 3. An image processing apparatus according to claim 2 , wherein said texture data storage means stores a plurality of vector value functions respectively defining a plurality of texture patterns, and said input means includes a plurality of texture functions. A texture generation procedure including a specification of a texture pattern to be used as a parameter and a specification of an attribute of a texture to be generated, wherein the texture supply unit defines the texture pattern specified by the texture generation procedure received by the input means. An image processing apparatus for reading a vector value function to be performed from the texture data storage means and generating a texture with an attribute specified by a texture generation procedure. 4. The image processing apparatus according to claim 3 , wherein the plurality of texture patterns stored in the texture data storage means are character fonts of a plurality of characters,
An image processing apparatus according to claim 1, wherein the designation of the texture pattern to be used as a texture in said texture generation procedure is a designation by a character code of a character of a character font to be used as a texture. 5. An image processing apparatus according to claim 3 , comprising an image editing apparatus, and an image output apparatus, wherein said image editing apparatus specifies a texture pattern to be used as a texture and an attribute of a texture to be generated. A user interface unit that interactively accepts the specification of a texture, and a procedure creation unit that creates a texture generation procedure including the specification of the texture pattern and the attribute of the texture received by the user interface unit. The input means of the image processing apparatus receives the texture generation procedure created by the procedure creation unit of the image editing apparatus, and converts the vector value function defining the texture pattern specified by the texture generation procedure into the texture data. Read out from the data storage means, and generate a texture with the attribute specified by the texture generation procedure; 3D scene generation system and outputs a pickup image of the subject texture mapping of the image processing apparatus is generated. 6. The three-dimensional scene generating system according to claim 5 , wherein the user interface of the image editing device specifies a texture pattern to be used as a texture and a attribute of a texture to be generated. In addition to the above, the procedure creation unit interactively receives designation of a relative positional relationship between the camera screen and the subject and designation of a three-dimensional shape of the subject. Creating a three-dimensional scene generation procedure including designation of a relative positional relationship between the camera screen and the subject accepted by the interface unit and designation of a three-dimensional shape of the subject; The means accepts a three-dimensional scene generation procedure created by a procedure creation unit in the image editing apparatus, and the texture supply unit receives the three-dimensional scene creation procedure specified by the input means. As specified in the three-dimensional shape read the object data from the object data storage means,
In accordance with the read subject data and the designation of the relative positional relationship between the camera screen and the subject specified in the three-dimensional scene generation procedure, an image is taken on the camera screen.
The size of the subject assumed to be
Texture image that coordinates with the subject according to
When determining the size and generating a photographed image of the subject, a vector value function that defines the texture pattern specified by the texture generation procedure is read from the texture data storage means and specified by the texture procedure command. A three-dimensional scene generation system, wherein a texture having a determined image size is generated by an attribute. 7. The three-dimensional scene generation system according to claim 5 , wherein the plurality of texture patterns stored in the texture data storage means are character fonts of a plurality of characters,
An image processing apparatus, wherein the user interface unit of the image editing apparatus accepts designation of a character font to be used as a texture by designating a character. 8. The three-dimensional scene generation system according to claim 5 , wherein the generation of the image and the output of the image based on the specification received by the user interface unit of the image editing apparatus are executed in real time. A three-dimensional scene generation system.