JP3382536B2 - Image generation method, image generation device, and recording medium recording image generation program - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image generating apparatus and method, and more particularly to an image generating apparatus and method capable of generating an image having a natural depth of field at high speed or controlling the depth of field. ..

[0002]

2. Description of the Related Art Conventionally, in order to obtain a depth of field similar to that in actual shooting when an image is generated by computer graphics, it is roughly divided into (1) a picture generated by a conventional method How to add blur with defocus filter after generating (focus in area) (Potmesil, Michael et al. "AL
ens and Aperture Camera Model for Synthetic Image
Genaration ", Computer Graphics, vol.17, no.3, pp13
7-145, 1981.) (2) Distributed ray tracing method (Cook, RL et al. "Distributed
Ray Tracing ", Computer Graphics, vol18, no.3, pp1
37-145, 1984.).

[0003]

By the way, one of the most important characteristics of the natural depth of field found in live-action images is that the image of a large blurred object in front of the subject is blurred due to the distance at which the subject is focused. At the same time, it can be seen through the edge. In the method of (1) above, since the picture generated by completely removing the hidden surface is blurred, it is basically impossible that the object beyond it can be seen through, no matter how much the object in front is blurred. There was a problem that it made an unnatural picture. Although the method of (2) above basically does not cause such a problem, it is necessary to trace at least tens of dispersed rays based on the ray tracing method which originally takes time. Therefore, there is a problem that it takes a very long time to generate a desired image. Further, in general, in order to combine a plurality of images after generation regardless of the generation means, it is necessary to clarify the boundary to be combined according to the conventional technique, and the object to be combined is focused in front. It had to be generated in the state and the edge of the image of the object had to be made clear. When it is desired to put the object synthesized in the foreground in the defocused state, it has to be synthesized by a skilled manual work, and therefore the amount of work in the moving image is very large. In order to avoid this, in general, when shooting a live-action image, it is necessary to shoot in advance so that everything is in focus from the front to the back (hereinafter referred to as "pan focus"), which greatly limits the shooting conditions.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image generating apparatus and method capable of rapidly generating an image having a natural depth of field. Further, the present invention provides an image generating apparatus and method capable of controlling the depth of field in the same specified manner (focal length of lens, F value, distance for focusing) of shooting with an actual camera. To aim.

[0005]

In order to achieve the above object, the invention according to claim 1 of the present invention is an object field which is a range in which the focus is practically in front of and behind the focus. Defocus data that uses at least coordinate conversion for each pixel of one object model data, hidden surface processing of the object itself, defocus processing as object model data Is an image generation apparatus for generating a plurality of images by inputting a plurality of planes, and is capable of storing R, G, B, A, and Z values as pixel values, and a multi-plane buffer including a plurality of planes, and R, G, and B for each pixel. ，
A plurality of defocus data having the values of A and Z are set as pixels.
For each defocus data, the multi-plane buffer processing unit that rearranges the data so that the distance from the viewpoint is closer to the multi-plane buffer and stores the data in the multi-plane buffer. An output image creation processing unit for synthesizing the contents of the planes of the plane buffer to generate a target image and outputting the image is provided. Further, the invention according to claim 2 is an object model for each of a plurality of objects for which a front-back relationship with respect to a viewpoint is expressed by using a depth of field which is a range in which the focus is practically in focus before and after the focus. An image generation device for generating an image by inputting a plurality of object model data as data, and R, G, B, A, Z as pixel values.
Defocus buffer that can store the values of R, G, B, A, and Z as pixel values, and a multiple buffer that consists of multiple planes, coordinate conversion for each pixel of one object model data, and the object itself. Hidden surface processing and shading processing are performed, and R,
From the shading data having the values of G, B, and Z, for each pixel for expressing a defocused image that is out of focus according to a function that is the same as the focal length of the actual camera, the aperture value, and the distance to the subject. R, G, B,
A defocus processing unit that generates a plurality of defocus data having values of A and Z and stores the defocus data in the defocus buffer, and reads the defocus data from the defocus buffer to determine the distance from the viewpoint for each pixel . The contents of each plane of the multi-plane buffer after the processing by the multi-plane buffer processing unit that stores in the multi-plane buffer while rearranging so as to be closer to each other and the processing by the multi-plane buffer processing unit for all defocus data And an output image creation processing unit that generates a target image by combining the two and outputs the combined image. Further, in the invention according to claim 3, the multi-plane buffer processing unit, when the number of planes of the multi-plane buffer is n, is a Z value Z at a pixel position (X, Y) on defocus data.
Z value Z _m (X, Y) _{#n of} the pixel position corresponding to _d (X, Y) and the n-th plane multi-plane buffer
If the former is smaller, the Z value Z _m (X, X, X) of the corresponding image of each plane of the multi-plane buffer is compared.
Y) _{# 1} to Z _m (X, Y) _{#n in} order of Z _d (X,
Y) as compared _{to, Z d (X, Y)} <Z m (X, Y) #N
When the Z value at the corresponding image position of the N-th plane that satisfies the above condition is found, the pixel values of the corresponding pixels on the N-plane and subsequent planes on the multi-plane buffer are shifted one by one to the corresponding pixels on the subsequent plane, By making the pixel value of the focus data the pixel value of the N-th plane of the multi-plane buffer, a process of storing the plurality of defocus data in the multi-plane buffer in ascending order of distance from the viewpoint is performed. And In addition, claim 4
In the invention described in paragraph 1, the multi-plane buffer is provided with a minimum number of planes in an initial stage, and the multi-plane buffer processing unit includes the plurality of defocus data in the order of decreasing distance from the viewpoint. When performing the processing of storing in the above, a required plane is newly secured every time the plane of the multi-plane buffer becomes insufficient.

Further, the invention according to claim 5 is characterized in that the function includes A as a parameter. Further, in the invention according to claim 6, the defocus processing unit scans the shading data, calculates a diameter Φ of the point light source image when the point light source is blurred, and calculates a point within a circle of the diameter. The intensity ratio of the light source image is obtained, the intensity ratio coefficient M is obtained so that the sum of the intensity ratios in the circle of the diameter becomes 1.0, and R in the circle of the diameter Φ is utilized by using the intensity ratio coefficient M. , G, B values and alpha value A values are added to the values stored in the defocus buffer, and Z values at pixel positions (a, b) within the circle of the diameter Φ on the defocus buffer are calculated. And the Z value at the same pixel position (X, Y) of the shading data are compared, and if the Z value of the shading data is smaller, the Z value at the pixel position (X, Y) of the shading data is set to the defocus buffer. Circle pixel position of said diameter Φ of (a, b) defocus by transferring the Z value in
The feature is that data is generated. In addition, claim 7
In the invention described in (3), the defocus processing unit obtains the intensity ratio of the point light source image within the circle of the diameter by approximate calculation using cosine. Further, in the invention according to claim 8, the image generating apparatus is configured to perform a Z buffer capable of storing R, G, B, and Z values as pixel values, and coordinate conversion, hidden surface processing, and shading processing in the object model data. The defocus processing unit further includes a coordinate conversion and hidden surface removal processing unit that stores the result in the Z buffer, and the defocus processing unit processes the data stored in the Z buffer as the shading data.

According to a ninth aspect of the present invention, the image
The generator records the values of R, G, B, A, and Z as pixel values.
Z buffer that can be stored and the object model data
Coordinate conversion, hidden surface processing, shading processing to
Transformation and hidden surface removal processing for storing
And a defocus processing unit,
The data stored in the Z buffer is converted into the shading data.
It is characterized by processing as a data. In addition, claim 1
In the invention described in 0, the object is a pan focus
The background generated by S and given infinity as Z value
It is characterized by including an image. Also, in claim 11.
The invention uses a natural image for the object,
Chroma key area that is not used at the time of image composition,
Extracted with a threshold value given by the user, and extracted
Z value at infinity in the chroma key area and R as color information,
0 is given to each of G and B, and R and
The G and B values remain the same, and the Z value given by the user is given.
It is characterized by including a foreground image. Also, the contract
In the invention according to claim 12, the image generating device is configured so that the pixel value is
The background image bag that can store the values of R, G, B, and Z as
To give infinity as the Z value of the background image
More RGBZ conversion processing, the background image is an object
If it is generated with the same shooting parameters as
Write the result of BZ conversion processing to the background image buffer
Only the background image is generated by pan focus.
If the result of the RGBZ conversion process is
After that, the defocus processing unit
Write the processed data to the background image buffer.
And an output image creation process.
The processing unit includes the background image buffer and the multi-plane.
Read buffer values, generate image, output
It is characterized by

According to a thirteenth aspect of the present invention, the image generating apparatus extracts a chroma key area which is an area which is not used at the time of composition using a natural image with a threshold value given by a user. And Z value at infinity and R as color information in the extracted chromakey area.
0 is given to each of G and B, and R and
It is characterized by further comprising an RGBZ conversion processing unit for giving the Z value given by the user while writing the G and B values as they are and writing them to the Z buffer. According to a fourteenth aspect of the present invention, the image generation apparatus extracts a chroma key area, which is an area that is not used at the time of composition, by using a natural image, with a threshold value given by the user, and contacts the extracted area. A that changes gently in the boundary area with other areas
And a chroma key region processing section for calculating the Z value at infinity and R, G, B, A as color information in the extracted chroma key region.
Give 0 to each, and R, G, B for other areas
Value as it gives the A value the calculated, the Z value designated by the user is written into the Z buffer RGBAZ
A conversion processing unit is further provided.

Further, in the invention described in claim 15, a plurality of objects for which an anteroposterior relationship with respect to a viewpoint is expressed by utilizing the depth of field which is a range in which the focus is practically in focus before and after the focus is achieved. An image generation method for generating an image by inputting at least coordinate conversion for each pixel of one object model data, a hidden surface processing of the object itself, and a plurality of defocus data subjected to defocus processing as object model data, A plurality of defocus data having R, G, B, A, and Z values for each pixel are rearranged in order of decreasing distance from the viewpoint for each pixel, and R, G, B, A, and The multi-plane buffer processing step of storing the value of Z in the multi-plane buffer consisting of multiple planes and all defocus data After the processing by the multi-plane buffer processing step is completed, the contents of each plane of the multi-plane buffer are combined to generate a target image, and an output image creation processing step is output. . The invention according to claim 16 is an object model for each of a plurality of objects for which a front-back relation with respect to a viewpoint is expressed by utilizing a depth of field which is a range in which the focus is practically in focus before and after the focus. This is an image generation method for generating an image by inputting a plurality of object model data as data, and coordinate conversion, hidden surface processing of the object itself, shading processing are performed for each pixel of one object model data, and pixel value For expressing defocused image defocused according to a function consisting of the same focal length of the camera, aperture value and distance to the subject from the shading data having R, G, B and Z values as A plurality of defocus data having R, G, B, A, and Z values for each pixel is generated and used as a pixel value. ,
A defocus processing step of storing the values of G, B, A, and Z in a defocus buffer, and reading the defocus data from the defocus buffer so that the distance from the viewpoint is closer to each pixel. A multi-plane buffer processing step in which the values of R, G, B, A, and Z can be stored as pixel values while rearranging and stored in a multi-plane buffer composed of a plurality of planes; and the multi-plane buffer processing step for all defocus data. After completion of the processing according to (1), the contents of the planes of the multi-plane buffer are combined to generate a target image, and an output image generation processing step of outputting the target image is output. Further, in the invention according to claim 17, in the multi-plane buffer processing step, when the number of planes in the multi-plane buffer is n, a Z value Z _d (at a pixel position (X, Y) on defocus data is set. X, Y) and the Z value Z _m (X,
Y) _#n is compared, and if the former is smaller, Z value Z _m of the corresponding image of each plane of the multi-plane buffer
Z from (X, Y) _{# 1} to Z _m (X, Y) _{#n in} order
compared to _{d (X, Y), Z} d (X, Y) <Z m (X,
Y) Z of the corresponding image position of the N-th plane that satisfies _#N
When the value is found, the pixel values of the corresponding pixels on the N plane and subsequent planes on the multi-plane buffer are shifted one by one to the corresponding pixels on the subsequent plane, and the pixel value of the defocus data is changed to N on the multi-plane buffer. By using the pixel value of the plane eye, a process of storing the plurality of defocus data in the multi-plane buffer in ascending order of distance from the viewpoint is performed. Also,
In the invention according to claim 18, the multi-plane buffer is provided with a minimum number of planes in an initial stage, and in the multi-plane buffer processing step, the plurality of defocus data are arranged in the order of decreasing distance from a viewpoint. When performing the process of storing in the multi-plane buffer, a required plane is newly secured every time the plane of the multi-plane buffer becomes insufficient.

The invention according to claim 19 is characterized in that the function includes A as a parameter. In the invention according to claim 20, the defocus processing step scans the shading data,
Calculate the diameter Φ of the point light source image when blurring the point light source,
The intensity ratio of the point light source image in the circle of the diameter is calculated, the intensity ratio coefficient M is calculated so that the sum of the intensity ratios in the circle of the diameter becomes 1.0, and the intensity ratio coefficient M is used to The values of R, G, B and the alpha value A in the circle of the diameter Φ are calculated and added to the values stored in the defocus buffer, and the pixel position within the circle of the diameter Φ on the defocus buffer ( comparing the Z value at a, b) with the Z value at the same pixel position (X, Y) of the shading data,
If the Z value of the shading data is smaller, the Z value at the pixel position (X, Y) of the shading data is moved to the Z value at the pixel position (a, b) within the circle of the diameter Φ on the defocus buffer. Thus, the defocus data is generated. Further, the invention according to claim 21 is characterized in that, in the defocus processing step, an intensity ratio of a point light source image within a circle of the diameter is obtained by an approximate calculation using a cosine.
Further, in the invention according to claim 22, in the image generation method, coordinate conversion, hidden surface processing, and shading processing are performed on the object model data, and the result is set to R,
Further comprising a coordinate conversion and hidden surface removal processing step of storing the G, B, and Z values in a Z buffer, wherein the defocus processing step processes the data stored in the Z buffer as the shading data. Is characterized by.

According to a twenty-third aspect of the present invention, the image
The image generation method is based on the coordinate transformation of the object model data.
Conversion, hidden surface processing, shading processing, and the result is the pixel value
Z buffer that can store R, G, B, A and Z values as
The coordinate transformation and hidden surface removal processing steps to be stored in
In addition, the defocus processing step includes
Data stored in the buffer
It is characterized by processing as. In addition, in claim 24
According to the invention described above, the object is pan-focused.
A background image generated by Z and given infinity as the Z value
It is characterized by including. In addition, according to claim 25,
Ming said the object is a natural image,
Use the chroma key area that is not used during composition
Extracted with a threshold value given by
Z value at infinity in the Romakey area and R, G, B as color information
Give 0 to each, and R, G, B for other areas
The value is left as it is, and the foreground with the Z value given by the user is given.
It is characterized by including a round image. In addition, claim 2
In the invention described in 6, the background image is generated by the image generation method.
RGBZ conversion processing by giving infinity as Z value
And the background image is the same as the object.
If it is generated with shadow parameters, the RGBZ conversion
The values of R, G, B, and Z can be stored with the result of processing as a pixel value.
The background image buffer and the spine
If the scene image was generated by pan focus,
Write the result of RGBZ conversion processing to the Z buffer
After that, the defocus process is performed by the defocus processing step.
Write the processed data to the background image buffer
And an output image creation process.
The processing step includes the background image buffer and the multi-image buffer.
Read the values in the plane buffer, generate an image,
It is characterized by outputting.

According to the 27th aspect of the present invention, in the image generation method, a chromakey region processing step of extracting a chromakey region, which is a region that is not used at the time of synthesis, using a natural image, with a threshold value given by a user. And Z value at infinity in the extracted chroma key area and 0 for each of R, G, B as color information, and R, G, B values as they are for other areas, Z value given by the user. And an RGBZ conversion processing step of writing the data to the Z buffer and writing the data to the Z buffer. In addition, claim 28
The invention according to, in the image generation method, a chroma key area that is an area that is not used during composition using a natural image,
A chroma key region processing step of extracting with a threshold value given by the user, and calculating A that changes gently in a boundary region between the extracted region and another region in contact with the region;
The Z value at infinity is given to the extracted chroma key area and 0 is given to each of R, G, B, and A as color information, and the R, G, and B values remain unchanged for the other areas, and the calculated A value is used. And an RGBAZ conversion processing step of giving a Z value specified by the user and writing the Z value in the Z buffer.

Further, the invention according to claim 29 is characterized in that a plurality of objects for which an anteroposterior relationship with respect to a viewpoint is expressed by utilizing a depth of field which is a range in which the focus is practically in focus before and after the focus. A computer in which an image generation program for generating an image by inputting a plurality of defocus data that has been subjected to at least coordinate conversion, hidden surface processing of the object itself, and defocus processing for each pixel of object model data This is a readable recording medium, and a plurality of defocus data having R, G, B, A, and Z values for each pixel are rearranged in order of decreasing distance from the viewpoint for each pixel, and pixel values are arranged. R, G, B, A, and Z values can be stored as a multiplex buffer that is stored in a multiple plane buffer composed of multiple planes. Buffer processing step and after the processing by the multi-plane buffer processing step for all defocus data is completed, the contents of each plane of the multi-plane buffer are combined to generate a target image and output And processing steps are realized by a computer.
Further, the invention according to claim 30 is an object model for each of a plurality of objects for which a front-back relationship with respect to a viewpoint is expressed by utilizing a depth of field which is a range in which the focus is practically in focus before and after the focus. A computer-readable recording medium in which an image generation program for inputting a plurality of object model data and generating an image is recorded as data, coordinate conversion for each pixel of one object model data, and hidden surface processing of the object itself. , Shading processing is performed, and out of focus from shading data having R, G, B, and Z values as pixel values according to a function consisting of the same focal length and aperture value of the camera and the distance to the object. R, G, for each pixel for expressing the defocused image
A defocus processing step of generating a plurality of defocus data having values of B, A and Z and storing the defocus data in a defocus buffer capable of storing the values of R, G, B, A and Z as pixel values; The defocus data is read from the buffer and rearranged so that the distance from the viewpoint is closer to each pixel, and the pixel values R,
The multi-plane buffer processing step of storing the values of G, B, A, and Z in a multi-plane buffer composed of a plurality of planes, and the multi-plane buffer processing step for all defocus data, It is characterized in that the contents of each plane of the plane buffer are combined to generate a target image, and an output image generation processing step of outputting the image is realized by a computer. Further, in the invention according to claim 31, in the multi-plane buffer processing step, when the number of planes in the multi-plane buffer is n,
Z at pixel position (X, Y) on defocus data
The value Z _d (X, Y) and the Z value Z _m (X, Y) _#n of the corresponding pixel position on the n-th multi-plane buffer
If the former is smaller, the Z value Z _m (X, X, X) of the corresponding image of each plane of the multi-plane buffer is compared.
Y) _{# 1} to Z _m (X, Y) _{#n in} order of Z _d (X,
Y), and at the time when the Z value of the corresponding image position of the N-th plane that satisfies Z _d (X, Y) <Z _m (X, Y) _#N is found, the N-plane and subsequent N-planes on the multi-plane buffer are detected. By shifting the pixel value of the corresponding pixel one by one to the corresponding pixel of the subsequent plane and setting the pixel value of the defocus data as the pixel value of the Nth plane of the multiplane buffer, It is characterized in that the data is stored in the multi-plane buffer in the ascending order of the distance from the viewpoint. In addition, claim 32
In the invention described in the above, the multi-plane buffer is provided with a minimum number of planes in an initial stage, and the multi-plane buffer processing step includes the plurality of defocus data in the order of decreasing distance from a viewpoint. When performing the processing of storing in the above, a required plane is newly secured every time the plane of the multi-plane buffer becomes insufficient.

The invention described in Item 33 is characterized in that the function includes A as a parameter. In the invention according to claim 34, the defocus processing step scans the shading data,
Calculate the diameter Φ of the point light source image when blurring the point light source,
The intensity ratio of the point light source image within the circle of the diameter is determined, the intensity ratio coefficient M is determined so that the sum of the intensity ratios within the circle of the diameter becomes 1.0, and the intensity ratio coefficient M is used to Diameter Φ
The values of R, G, B and the alpha value A in the circle are calculated and added to the values stored in the defocus buffer, and pixel positions (a, b) comparing the Z value at the same pixel position (X, Y) of the shading data, and if the Z value of the shading data is smaller, the Z value at the pixel position (X, Y) of the shading data is calculated. The defocus data is generated by shifting to the Z value at the pixel position (a, b) within the circle of the diameter Φ on the defocus buffer. In the thirty-fifth aspect of the invention, the defocus processing step is characterized in that the intensity ratio of the point light source image within the circle of the diameter is obtained by approximate calculation using a cosine.
According to a thirty-sixth aspect of the present invention, the image generation program can perform coordinate conversion, hidden surface processing, and shading processing on the object model data, and can store the values of R, G, B, and Z as a pixel value. Further comprising coordinate transformation and hidden surface removal processing steps for storing in a Z buffer,
In the defocus processing step, the data stored in the Z buffer is processed as the shading data.

According to a thirty-seventh aspect of the present invention, the image generation program performs coordinate conversion, hidden surface processing, and shading processing on the object model data, and R, G, B, A, and Z are used as the pixel values. Further comprising a coordinate transformation and hidden surface removal processing step of storing the value in the Z buffer, and the defocus processing step processes the data stored in the Z buffer as the shading data. Further, the invention according to claim 38, wherein the object is generated by the pan-focus, characterized in that it comprises a background image infinity is given as the Z value. According to a thirty-ninth aspect of the present invention, the object uses a natural image to extract a chroma key area, which is an area that is not used during image composition, with a threshold value given by a user, and the extracted chroma key area. To the Z value at infinity and 0 as R, G, and B as color information, and for the other areas, the R, G, and B values remain unchanged, and the foreground image including the Z value given by the user is included. It is characterized by According to a forty-fourth aspect of the present invention, the image generation program performs a step of performing RGBZ conversion processing on the background image by giving infinity as a Z value, and the background image is generated with the same shooting parameter as the object. If the result of the RGBZ conversion processing is R,
Writing to a background image buffer capable of storing G, B, and Z values; and, if the background image is generated by pan focus, writing the result of the RGBZ conversion processing to the Z buffer, and then performing the defocus processing step. Further including a processing step of writing the defocused data to the background image buffer, the output image creation processing step, read the values of the background image buffer and the multi-plane buffer, to generate an image, It is characterized by outputting.

The image forming program according to the present invention is a chroma key area processing step for extracting a chroma key area which is an area which is not used at the time of composition using a natural image with a threshold value given by a user. And Z value at infinity in the extracted chroma key area and 0 for each of R, G, B as color information, and R, G, B values as they are for other areas, Z value given by the user. And an RGBZ conversion processing step of writing the data to the Z buffer and writing the data to the Z buffer. Further, in the invention according to claim 42, the image generation program extracts a chroma key area, which is an area that is not used at the time of composition, by using a natural image, with a threshold value given by a user, and contacts the extracted area. A chroma key region processing step of calculating A that changes gently in a boundary region with another region, Z value of infinity to the extracted chroma key region, and 0 is given to each of R, G, B, A as color information, For the other areas, the R, G, and B values are left unchanged, and the computer further implements the calculated A value and the Z value specified by the user, and the RGBAZ conversion processing step of writing in the Z buffer. It is characterized by

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described with reference to the drawings.

(First Embodiment) FIG. 1 shows the first embodiment of the present invention.
It is a block diagram of the image generation apparatus of the embodiment. The image generation apparatus according to the present embodiment includes an object model data read processing unit 1, a coordinate conversion / hidden surface removal processing unit 2, a Z buffer 3, a shading / mapping processing unit 4, a defocus processing unit 5, a defocus buffer 6, and a multiplane. It comprises a buffer processing unit 7, a multi-plane buffer 8, an output image creation processing unit 9, a disk writing processing unit 10 and a storage device 11.

The object model data read processing unit 1 reads the object model data from the storage device 11 for each object (object) whose front-to-back relationship is desired to be expressed by using the depth of field, and the next coordinate conversion and hidden surface removal processing unit. Pass to 2. The concept of an object is shown in FIG. An object is usually represented by a combination of a plurality of polygons and a plurality of basic shapes (hereinafter referred to as primitives) (object model data). As shown in FIG. 2, 3 of A, B, C
When one object exists and it is desired to express the context of A, B, and C by the depth of field, the object model data is read out for each of A, B, and C and passed to the coordinate conversion and hidden surface removal processing unit 2. The process described below proceeds for each read object, and the process of reading the next object when the multi-plane buffer process is completed is continued until there are no more objects. In addition,
The term "depth of field" as used herein refers to a range in which there is practical focus before and after focusing.

The coordinate conversion / hidden surface removal processing section 2 performs coordinate conversion, hidden surface removal, shading, etc. for each object passed from the object model data read processing section 1. As the coordinate conversion and hidden surface removal processing unit 2, the same one as a general computer graphics image generation device is used.

The coordinate conversion is a process of converting the object model data to be processed into a coordinate system suitable for each process. As an example, it is a process of converting an object represented by a body coordinate system into coordinates such as a world coordinate system, a viewpoint coordinate system, and a screen coordinate system in accordance with the process. Here, the "body coordinate system" means that the coordinate system is used as a coordinate origin and a coordinate axis where data regarding each object can be easily input. The body coordinate system may be called an object coordinate system or a model coordinate system. The "world coordinate system" refers to a coordinate system for arranging an object represented by a body coordinate system according to a motion model. The “viewpoint coordinate system” refers to a coordinate system in which the position of the viewpoint is the origin with respect to the world coordinate system and the direction of the line of sight is the positive direction of the Z axis. The "screen coordinate system" means a coordinate system for projecting the data placed in the viewpoint coordinate system on the screen. These coordinate transformations are mainly performed by affine transformation.

The hidden surface erasing is a process of erasing a portion that cannot be seen from the viewpoint due to overlapping of objects or the like. As a hidden surface erasing method, a Z buffer method,
There are scan line method, ray tracing method and the like. In the embodiment, hidden surface removal by the Z buffer method is taken as an example. Here, the "Z-buffer method" uses a Z-buffer capable of storing Z-values related to the distance from the viewpoint, and calculates the distance from the viewpoint to each point of each object and the Z-value of the corresponding pixel in the Z-buffer. Compared, the distance to the object point is smaller than the Z value in the Z buffer (closer to the viewpoint)
, The color data of the corresponding pixel in the Z buffer and Z
This is a method to erase the hidden surface by rewriting the value.

The shading / mapping processing unit 4 performs shading, mapping and the like on the object whose coordinates have been converted and hidden. shading·
The same processing unit 4 as a general computer graphics image generation device is used as the mapping processing unit 4.

Here, "shading" means various elements that make up the color of the visible object, that is, the original color of the object, the state of the surface, the direction in which the points on the surface are facing, the position and color of the light source. , A process of determining a color to be applied to a pixel in consideration of the position of the viewpoint, the direction of the line of sight, the reflection of surrounding objects, and the like. This shading algorithm is generally called a shading model, and as this shading model, there are a Lambert model, a von shading model, a Bryn shading model, and the like. Also,"
The term "mapping" refers to a process of expressing a pattern on the surface of an object by mapping planar mapping data onto the surface of an object having a three-dimensional shape. The mapping method includes surface mapping and solid mapping. The shading / mapping etc. processing unit 4 performs a shading process on the object whose coordinate conversion / hidden surface removal has been performed.
Also handles them.

The image generated for each object is stored in the Z buffer 3 of the general format shown in FIG. 3 and passed to the next defocus processing section 5. In the Z buffer 3, all (x, y) are initialized to the initial values in FIG. 3 at the start of processing. In FIG. 3, the size of the image to be obtained as a result is horizontal x _max pixels and vertical y _max pixels.

The format of the defocus buffer 6 is shown in FIG. 4, and the defocus processing is shown in FIG. x _max , y
_max represents the size of an image to be obtained as a result in the number of pixels in the horizontal and vertical directions. At the start of the process, R = 0.0, G = 0.0, B = 0.0 as initial values, the distance Z = ∞ from the viewpoint to the corresponding point, and the alpha value A = 0.0.
Is given to all (x, y). The alpha value A
Takes a value between 0 and 1, and the value is obtained by a calculation method described in detail later.

The defocus processing section 5 coordinates (X, X) containing the corresponding object on the Z buffer 3.
The pixels of Y) are spread in the defocused image, and the spread image is assumed to be circular, and the diameter Φ is obtained by the geometrical optical approach in units of pixels (step 21).
~ 24).

The method will be described with reference to FIGS. Although FIGS. 5A to 5C are shown two-dimensionally,
As explained below, there is no problem in thinking in two dimensions. In this embodiment, the lens surface and the diaphragm ring surface are the same as shown in FIGS. Figure 5 as a general theory of geometrical optics
As shown in (a), the coordinates (α, β) on which the point light source located at (z, y) = (a, b) forms an image are expressed as follows (always a> f).

[Equation 1]

Next, FIGS. 5 (b) and 5 (c) separately show whether the image forming point (α, β) is in front of or behind the film surface. s is a distance adjusted so that the image forming point of the point light source at a certain distance a ₀ from the camera is on the film surface. That is, it is the distance designated by the user so as to focus on the point light source located at the distance a ₀ from the camera. Since a ₀ is always larger than f,

[Equation 2] Is.

Using the triangle similarity from FIGS. 5 (b) and 5 (c),

[Equation 3] Is obtained. Here, F is the aperture value, f is the focal length of the lens, φ _iris is the diameter of the aperture ring, and φ _image is the diameter of the image of the point light source that appears on the film.

Since the diameter φ _image obtained here is the actual size, the unit is converted into pixels, and the diameter Φ of the image actually used is obtained (step 23).

[Equation 4]

Here, a ₀ is the distance of the point light source for focusing from the camera, a is the distance of the target pixel from the camera (Z value on the Z buffer 3), f is the focal length of the lens, and F is Aperture value, φ _film is the diameter of a circle circumscribing an effective rectangle of a film, and x _max and y _max are the number of pixels in the horizontal direction and the number of pixels in the vertical direction which form the image to be generated.

Next, the intensity ratio distribution in the circular image having the diameter Φ obtained by the equation (4) is obtained. Generally, in wave optics, the intensity ratio distribution is

[Equation 5] (Potmesil, Michael et al. "A Lens and Apertur
e Camera Model for Synthetic Image Genaration ", Co
mputer Graphics, vol.17, no.3, pp137-145, 198
1.). Here, the coordinates of (l, m) are set so that the origin (0,0) is the center of the circular image. Also, J
_l (a) is the first series of the Bessel function. Although the intensity ratio distribution may be obtained by this equation, it is not necessary to consider interference in the present invention, and the intensity ratio distribution is obtained by using the following equation as an approximate equation in order to reduce the processing amount.

[Equation 6] The function forms of the equations (5) and (6) are shown in FIG. 6 as (l ² + m ² ) ^1/2 = r.

In the equation (6), since the circular image has a diameter of 2π around the origin, the shape actually used on the defocus buffer 6, that is, the diameter is Φ and the center is (X, Y).
It will be converted into and used. That is,

[Equation 7] To use as.

Next, according to equation (7), the center is (X, Y)
Then, the sum of the intensity ratios in all the pixels in the circular image with the diameter Φ is obtained, and the intensity ratio coefficient M is set so that the sum becomes 1.0.
Is determined (step 25).

[Equation 8]

Using the values thus obtained, the Z buffer 3
The center obtained by defocusing the pixel at a certain coordinate (X, Y) in the corresponding object above is (X, Y).
In Y), a circular image (R _d , G _d , B _d ) having a diameter Φ is obtained by the following equation. After that, R, G, B, A,
Suffixes z, d, and m attached to Z, R (x, y) indicate values on the Z buffer 3, the defocus buffer 6, and the multi-plane buffer 8, respectively.

[Equation 9]

At the same time, the alpha value A used in the output image creation processing section 9 is obtained.

In the equations (9), (10) and (11), initial values of R _d (x, y), G _d (x, y) and B _d (x, y) are given, It indicates that substitution of the calculated value on the right side to the left side is repeated.

[Equation 10]

As described above, the values of R, G, B and A are obtained at all the coordinates in the corresponding object on the Z buffer 3, and the result is added to the corresponding value of the corresponding coordinates in the defocus buffer 6 ( Step 26).

Regarding the Z value in (x, y) on the defocus buffer 6, if the Z value in (X, Y) on the Z buffer 3 is smaller than the Z value in (x, y) on the defocus buffer 6. The Z value at (X, Y) on the Z buffer 3 is set to the Z value at (x, y) on the defocus buffer 6.
Value (step 27).

The format of the multi-plane buffer 8 is shown in FIG. 8, and the multi-plane buffer processing is shown in FIG.
x _max and y _max represent the size of the image to be obtained as a result by the number of pixels in each of the horizontal and vertical directions. n is the number of multi-plane buffers 8 and is a value that is determined depending on the number of defocused objects that appear to overlap on the output image. The multi-plane buffer 8 has arbitrary coordinates (x, y), ((0 ≦ x <x _max ), (0 ≦ y
<Y _max )) for each n R _m (x, y), G _m
(X, y), B _m (x, y), Z _m (x, y), A
_m (x, y) is retained. Multiplane buffer 8
As an initial value of R _m = 0.0, G _m = 0.0, B _m = 0.0, for all n of all (x, y) at the start of processing.
Z _m = ∞ and _Am = 0.0 are given.

All information R (X, Y), G (from the coordinate (X, Y) included in the object being processed on the defocus buffer 6 to the corresponding coordinate (X, Y) in the multi-plane buffer 8). X (Y,), B (X, Y), Z (X,
Y) and A (X, Y) are transferred. The outline of the processing in that case is shown in FIG.

FIG. 9 explains the processing for a certain coordinate (X, Y) included in the object being processed. As described above, n R _m (X, Y), G _m (X, Y) coordinates (X, Y) of the multi-plane buffer 8 are present.
Y), B _m (X, Y), Z _m (X, Y), A _m (X, Y)
Are held, and R _m (X, Y) _#n and G _m (X,
_{Y) # n, B m (} X, Y) #n, Z m (X, Y) #n, A
It is expressed as _m (X, Y) _#n . The multi-plane buffer processing unit 7 causes the multi-plane buffer 8 to always store Z _m (X, Y) _{# 1} ≤ Z _m (X, Y) _{# 2} ≤ Z for (X, Y).
_{m (X, Y) # 3} ··· ≦ Z m (X, Y) is controlled so as to _#n. Regarding (X, Y) of the defocus buffer 6, the multi-plane buffer processing unit 7 first sets Z _d (X, Y) in (X, Y) on the defocus buffer 6 and Z _m (X in the multi-plane buffer 8). , Y) _#n are compared (step 33). As a result Z _d (X, Y)> Z
_{If m} (X, Y) _#n , the multi-plane buffer processing unit 7 does not transfer the information regarding (X, Y) from the defocus buffer 6 to the multi-plane buffer 8.
If Z _d (X, Y) ≤ Z _m (X, Y) _#n , the multi-plane buffer processing unit 7 outputs Z _m (X, Y) _{# 1} to Z _m (X, Y).
Y) Up to _{#n are} sequentially compared with Z _d (X, Y) (steps 34 and 35). Z _d (X, Y) <Z _m (X, Y) _#N,
When N that is (1 ≦ N ≦ n) is found, the multi-plane buffer processing unit 7 performs (R, G, B, Z,
Shift A) backwards one by one (step 36),
Finally, R _d (X, Y) and G _{d of the} defocus buffer 6
(X, Y), B _d (X, Y), Z _d (X, Y), A
_d (X, Y) is R _m of the Nth multi-plane buffer 8
(X, Y) _#N , G _m (X, Y) _#N , B _m (X, Y) _#N , Z
_{m (X, Y) #N,} A m (X, Y) is transferred to _#N (Step 3
7). This work is performed for all (X, Y) including the corresponding object (steps 31, 32).

When the processing of the corresponding object is completed (step 38), if the object is the last object, the process proceeds to the next output image creation processing section 9,
If not, the process returns to the object model data read processing unit 1 to read the next object model data, and the processes described so far are performed.

Here, the number n of the multi-plane buffers 8 is a value determined depending on the number of defocused objects that appear to overlap on the output image, and n multi-plane buffers 8 are secured in advance. I explained that it is. However, if the multi-plane buffer 8 can be dynamically secured, at the initial stage, only the minimum number of multi-plane buffers 8 (for example, one) is secured, and the N-th plane and subsequent planes in step S36 of FIG. When the number of the multi-play buffers 8 becomes insufficient in the process of lowering the information of each plane one by one backward, a new plane buffer for the multi-plane buffer 8 may be secured. That is, when the number of planes in the multi-plane buffer 8 is currently α and the number of planes is required to be (α + 1) for the processing in step S36, a buffer for one plane is newly initialized and secured, and the multi-plane buffer is After setting the number of 8 planes to (α + 1), it is preferable to perform a process of lowering the information of the Nth and subsequent planes one by one. The required number of multi-plane buffers 8 is the maximum number of objects to be processed.

The output image creation processor 9 has 0 ≦ x <x.
_For all (x, y) of _max , 0 ≦ y <y _max , the following formula is calculated to create an output image.

[Equation 11]

In the above first embodiment, the Z buffer 3
The values of R, G, B, and Z are read from, and a defocused image that is out of focus is generated according to the function (equation (12)) that is composed of the same focal length and aperture value of the camera and the distance to the subject. However, from the Z buffer 3, R, G,
A function that reads B, A, and Z values, consists of the same lens as the actual camera, the same focal length, aperture value, and distance to the subject, and includes A as a parameter

[Equation 12] A defocused image that is out of focus may be generated according to. In this case, the format and initial values of the Z buffer 3 are as shown in FIG. Here, the expression (16) means substituting the value on the right side into the left side, and first, A _d (x,
An initial value is given as the value of y).

(Second Embodiment) FIG. 12 is a block diagram of an image generating apparatus according to a second embodiment of the present invention.

In this embodiment, a background image read processing unit 12, an RGBZ conversion processing unit 13, a switch 14 and a background image buffer 15 are added to the first embodiment (FIG. 1). Here, the RGBZ conversion processing unit 13 performs processing for adding the Z value to the R, G, B information.

The background image read processing unit 12 displays the background image.
Read from the storage device 11 and stored in the storage device 11
The image file format must be RGB format
First, convert the image file format to RGB format
To do. The conversion method is the same as the general one.
Therefore, the explanation is omitted. Image size to be generated x
_max, Y_maxIf the size is the same as
It is passed to the conversion processing unit 13, otherwise x_max, Y
_maxThe process of adjusting to is performed here. The processing method at that time is
The description is omitted because it is the same as a general one.

The background image is logically placed at infinity in the generated image. Therefore, infinity is generated as the Z value in all seat motions of the background image and added to the R, G, B information.
The background image stored in the storage device 11 is generated using the same photographing parameters (distance for focusing the lens and the F value and the focus) designated when the background image is generated according to the first embodiment. The flow after this is slightly different depending on the case and the case where the pan focus is used.

[When Background Image is Generated with the Same Shooting Parameter] All outputs of the RGBZ conversion processing unit 13 are transferred to the background image buffer 15 via the switch 14.
The format of the background image buffer 15 is shown in FIG.

After being transferred to the background image buffer 15, the image generation according to the first embodiment is started. After the multi-plane buffer processing is completed for all objects, the processing in the output image creation processing unit 9 is different from that of the first embodiment. Instead of equations (13) to (15), the output image is created by the following equation in the second embodiment.

[Equation 13] Where R (x, y) _back , G (x, y) _back , B
(X, y) _back is the R, G, B value at the coordinates (x, y) on the background image buffer 15.

[When Background Image is Generated by Pan Focus] The output of the RGBZ conversion processing unit 13 is transferred to the Z buffer 3 through the switch 14. After completion, the same defocus processing as in the first embodiment is performed on the entire background image transferred to the Z buffer 3. In the second embodiment, the alpha value A is not necessary for the background image, so the alpha value A according to the equation (12) is not calculated. The result of the defocus processing is moved to the background image buffer 15 instead of the defocus buffer 6.

After the defocusing process for the background image is finished and the result is stored in the background image buffer 15, the image generation according to the first embodiment is started. After the multi-plane buffer processing is completed for all the objects, the processing in the output image creation processing unit 9 is performed. This is performed according to the equations (17) to (19) as in the case where the background image is generated with the same shooting parameter.

In the present embodiment, the image processing apparatus is provided with the background image buffer 15, and the output image creation processing unit 9 uses the data in the background image buffer 15 and the data in the multi-plane buffer 8 to obtain equations (17) to (17). 19) to create an output image. However, Z value = ∞ and alpha value A = 1.0 are further added to the output data to the background processing buffer 15 and the data is transferred to the defocus buffer 6, and the multi-plane buffer processing unit 7
By storing the defocused background image in the multi-plane buffer 8, the processing of the output image processing unit 9 is the same as the processing described in the first embodiment (1
It can be processed by the equations (3) to (15). In addition, this makes it possible to omit the background image buffer 15.

(Third Embodiment) FIG. 14 is a block diagram of an image generating apparatus according to a third embodiment of the present invention. In the present embodiment, the foreground image reading processing unit 16, the chroma key area extraction processing unit 17, and the RGBZ conversion processing unit 18 are performed.
Is added to the second embodiment (FIG. 12).

The third embodiment may be executed before the object model data read processing of the object in the first embodiment, or may be executed after the multi-plane buffer processing of an object is finished and the object model data read of the next object. It may be performed at any timing before the start of the process, or after the end of the multi-plane buffer process of the last object and before the output image creation process.

Foreground image read processing unit 16
Reads from the storage device 11 an image captured using a blue background or the like (hereinafter referred to as a foreground image). If the image file format stored in the storage device 11 is not the RGB format, the image file format is first converted to the RGB format here. The conversion method is the same as a general method, and therefore its explanation is omitted.
If it is the same as the image size x _max , y _max to be generated, the image size is passed to the RGBZ conversion processing unit 18 as it is, and if not, a process of adjusting to x _max , y _max is performed here. Since the processing method at that time is the same as a general method, the description thereof will be omitted.

The chroma key area extraction processing section 17 extracts a chroma key area from the read foreground image. Since the processing at that time is the same as general processing, detailed description will be omitted. The outline of the processing is, for example, threshold values R _min , R for R, G, B respectively.
_The user is given _max , G _min , G _max , B _min , and B _max , and the chroma key processing is performed by regarding the pixels included in each range of R, G, and B from the foreground image as the chroma key area.

The RGBZ conversion processing section 18 includes a storage device 11
And the size is x _max , y _max and the format is RGB in the foreground image read processing unit 16.
First foreground image converted to format
It is written in the Z buffer 3 described in the description of the embodiment. At this time, the coordinates of the pixels in the chroma key area extracted by the chroma key area extraction processing unit 17 (R,
G, B) = (0, 0, 0) and give Z value infinity. For the coordinates of the other areas, give Z value given by the user with R, G, B values unchanged, and give to Z buffer 3. Write. This is shown in FIG. After the defocusing process after the end, the same process as the process for a certain object described in the first embodiment is performed.

(Fourth Embodiment) FIG. 16 is a block diagram of an image generating apparatus according to a fourth embodiment of the present invention. In the present embodiment, the foreground image reading processing unit 16, the chroma key area extraction processing unit 17, and the RGBAZ conversion processing unit 1
8'is added to the second embodiment (FIG. 12).

The fourth embodiment may be executed before the object model data read processing of the object in the second embodiment, or may be executed after the multi-plane buffer processing of an object is finished and the object model data read of the next object. It may be performed at any timing before the start of the processing, or after the end of the multi-plane buffer processing of the last object and before the output image creation processing.

Foreground image read processing unit 16
Reads from the storage device 11 an image captured using a blue background or the like (hereinafter referred to as a foreground image). If the image file format stored in the storage device 11 is not the RGB format, the image file format is first converted to the RGB format here. The conversion method is the same as a general method, and therefore its explanation is omitted.
Image size x _max trying to generate, passing the same if the size as it RGBAZ conversion processing unit 18 'and y _max, performed here a process to match the x _max, y _max otherwise. Since the processing method at that time is the same as a general method, the description thereof will be omitted.

The chroma key area extraction processing section 17 extracts the chroma key area from the read foreground image and calculates the A value. Here, the chroma key area extraction processing unit 17 sets 0 in the chroma key area and 1 in areas other than the chroma key area, and calculates an A value that changes gently in the boundary area with the area other than the chroma key area that is in contact with the chroma key area. Note that the chroma key area extraction and A value calculation processing are the same as general ones, and thus detailed description thereof is omitted. Here, the outline extraction process of the chroma key region, for example, R, G, B For each threshold _{R mi n, R max, G} min, G max, B min,
B _max is given by the user, and the chroma key processing is performed by regarding all the R, G, and B pixels included in the range from the foreground image as the chroma key area.

[0066] RGBAZ conversion processing unit 18 'is read from the storage device 11, the foreground image reading processing unit 16 with a size of x _max, y _max, format the foreground image converted to RGB format first embodiment It is written in the Z buffer 3 described in the description of the form. At this time, the coordinates of the pixels in the chroma key area extracted by the chroma key area extraction processing unit 17 are set to (R, G, B, A) = (0, 0, 0, 0), and the Z value is set to infinity. For the coordinates of the region of R,
Chroma key area extraction processing unit 17 with G and B values unchanged
Fig. 1 shows the A value calculated by and the Z value given by the user.
The data is written in the Z buffer 3 having the structure shown in FIG. This is shown in FIG.
Shown in.

After the defocusing process after the end, the same process as the process for a certain object described in the first embodiment is performed.

(Fifth Embodiment) In the fifth embodiment of the present invention, the foreground image read-out processing unit 1 of the third embodiment (FIG. 14) is added to the first embodiment (FIG. 1).
6, a chroma key area extraction processing unit 17, and an RGBZ conversion processing unit 18 are added. The explanation is shown in FIG. 1 and FIG.
Since it can be easily inferred from the description of, the description is omitted.

(Sixth Embodiment) The sixth embodiment of the present invention is the same as the first embodiment (FIG. 1) except that the foreground image read-out processing unit 1 of the fourth embodiment (FIG. 16) is used.
6, a chroma key area extraction processing unit 17, and an RGBAZ conversion processing unit 18 'are added. The description thereof will be omitted because it can be easily inferred from the description of FIGS. 1 and 16.

(Seventh Embodiment) FIG. 18 is a block diagram of an image generating apparatus according to a seventh embodiment of the present invention. In FIG. 18, in the storage device 11, an object for which a front-back relation with respect to a viewpoint is expressed by using the depth of field, which is a range in which the focus is practically focused before and after the focus, is used as object model data. It is data that has been subjected to at least coordinate conversion, hidden surface processing, and day focus processing for each object model data, and has R, G, and
A plurality of defocus data having values of B, A and Z are stored.

Defocus data read processing unit 31
Sequentially reads the defocus data stored in the storage device 11 and passes it to the multi-plane buffer processing unit 7. The multi-plane buffer processing unit 7 uses a multi-plane buffer 8 that can store R, G, B, A, and Z values as pixel values and that is composed of a plurality of planes, and then passes the defocus data that has been passed to the distance from the viewpoint. The data are stored in the multi-plane buffer 8 in the order close to.

The disk write processing unit 32 writes the data stored in the multi-plane buffer 8 in the storage device after all the defocus data to be processed has been processed.

As described above, an image of a large blurred object existing in front of a plurality of defocus data depending on a focused distance is blurred, and at the same time, an image having a natural depth of field can be seen through an edge at high speed. It becomes possible to generate.

In FIG. 18, parts corresponding to those in FIG. 1 are designated by the same reference numerals.

The object referred to here may include a background image generated by pan focus and having an infinity as a Z value. Also, the object extracts a chroma key area, which is an area that is not used at the time of image composition, by using a natural image with a threshold value given by the user, and outputs Z value and color information at infinity to the extracted chroma key area. It is also possible to include a foreground image in which 0 is given to each of R, G, and B, R, G, and B values are left unchanged for other regions, and Z values given by the user are given.

In FIG. 18, the operation of the multi-play buffer processing section 8 is as described in the first embodiment. That is, when the number of planes of the multi-plane buffer 8 is n, the Z value Z _d (X, Y) at the pixel position (X, Y) on the defocus data corresponds to the n-th multi-plane buffer 8. Z of pixel position
The values Z _m (X, Y) _#n are compared. And Z _d (X,
If Y) ≦ Z _m (X, Y) _#n , Z value Z _m (X, Y) _{1 of} the image corresponding to each plane of the multi-plane buffer 8
From Z _m (X, Y) _#n to Z _d (X, Y),
Find the Z value of the corresponding image position of the N-th plane that satisfies Z _d (X, Y) <Z _m (X, Y) _#N .

At the time when the Z value at the corresponding image position of the N-th plane satisfying the condition is found, the pixel values of the corresponding pixels on the N-plane and subsequent planes on the multi-plane buffer 8 are set to the corresponding pixels on the subsequent plane. The pixel value of the defocus data is shifted to be the pixel value of the N-th plane of the multi-buffer 8. As described above, a process of storing a plurality of defocus data in the multi-plane buffer 8 in the order of decreasing distance from the viewpoint is performed. It should be noted that the multi-plane buffer 8 has a minimum required number of planes in the initial stage, and the multi-plane buffer processing unit 7 performs a process of storing a plurality of defocus data in the multi-plane buffer in ascending order of distance from the viewpoint. At the time of execution, a necessary plane may be newly secured every time the number of planes in the multi-plane buffer 8 becomes insufficient.

The image generating apparatus of FIG. 18 may further include the defocus buffer 6 and the defocus processing section 5 described with reference to FIG. At this time, in the storage device 11, coordinate conversion, hidden surface processing, and shading processing are performed on the object model data,
Shading data having R, G, B, and Z values as pixel values is stored, and processing for the shading data is performed. The multi-plane buffer processing unit 7 defocuses the data stored in the defocus buffer 6. It will be processed as data.

In addition to this, the coordinate conversion and hidden surface removal processing unit 2 and the Z buffer described in FIG. 1 may be further provided, and a shading / mapping processing unit and the like may be provided if necessary. At this time, the data to be processed stored in the storage device is as described in the first embodiment.

The image forming apparatus shown in FIG. 18 further includes a processing section for processing the background image and the foreground image described in the second to sixth embodiments. May be.

(Eighth Embodiment) FIG. 19 is a block diagram of an image generating apparatus according to an eighth embodiment of the present invention. In the present embodiment, as compared with FIG. 18, the image generation apparatus of the present embodiment is provided with an output image creation processing unit 9, and the disc writing processing unit 32 of FIG. 18 is the disc writing processing described in the first embodiment. Part 10 is different.
Here, in FIG. 19, parts corresponding to the respective parts of FIG. 18 are designated by the same reference numerals, and description thereof will be omitted. The operations of the output image creation processing unit 9 and the disk writing processing unit 10 are the same as those described in the first embodiment, and the description thereof is omitted. From the above, it is possible to rapidly generate an image with a natural depth of field that is visible from the edge while blurring the image of a large blurred object that exists in the foreground depending on the focused distance from multiple defocus data. Is possible.

(Ninth Embodiment) FIG. 20 is a block diagram of an image generating apparatus according to a ninth embodiment of the present invention. In the image generating apparatus of FIG. 20, the storage device 11 performs coordinate conversion, hidden surface processing, and shading processing on the model data to be defocused, and outputs R, P as pixel values.
Shading data having G, B, and Z values is stored. Then, the shading data read processing unit 41 reads the shading data to be processed from the storage device 11 and delivers it to the defocus processing unit 5.
The defocus processing unit 5 defocuses the defocused image from the passed shading data according to a function consisting of the same focal length and aperture value of the lens of the camera and the distance to the subject.
Generate using. The disk writing processing unit 42
The defocus image generated in the defocus buffer 6 is written in the storage device 11. As described above, a defocused image can be generated. Note that, in FIG. 18, portions corresponding to the respective portions in FIG. 1 are designated by the same reference numerals.

The defocus processing unit 6 is as described in the first embodiment, but its operation will be described below again. Scan the passed shading data. Then, the diameter Φ of the point light source image when the point light source is blurred is calculated. The intensity ratio of the point light source image in the circle of the obtained diameter is obtained, and the intensity ratio coefficient M is obtained so that the sum of the intensity ratios in the circle of the diameter Φ becomes 1.0. Then, using the intensity ratio coefficient M, R in the circle of diameter Φ,
The G and B values and the alpha value A are calculated and added to the values stored in the defocus buffer 6. At this time, the Z value at the pixel position (a, b) within the circle of the diameter Φ on the defocus buffer 6 and the Z value at the same pixel position (X, Y) on the shading data are compared, and the Z value of the shading data is compared. If the value is smaller, the Z value at the pixel position (X, Y) on the shading data is set to the pixel position (a, b) within the circle of the diameter Φ on the defocus buffer 6.
To the Z value at. As described above, the defocus image is generated. Note that the defocus processing unit 5 can perform high-speed calculation by obtaining the intensity ratio of the point light source image within the circle of diameter Φ by approximate calculation using the cosine.

(Tenth Embodiment) FIG. 21 is a block diagram of an image generating apparatus according to a tenth embodiment of the present invention. In this embodiment, as compared with FIG. 20, the image generation apparatus of this embodiment additionally includes a coordinate conversion and hidden surface removal processing unit 2, a Z buffer 3, and a shading / mapping processing unit 4 as necessary. The points are different. Here, FIG.
In FIG. 20, parts corresponding to the respective parts in FIG. 20 are designated by the same reference numerals, and the description thereof will be omitted. The model data read processing unit 43 corresponds to the object model data read processing unit 1 in FIG. The coordinate conversion / hidden surface removal processing unit 2, the Z buffer 3, and the shading / mapping processing unit 4 are the same as those described in the first embodiment, and a description thereof will be omitted.

The image generated by the image generating apparatus described in the first to tenth embodiments is stored in the storage device 11
However, the present invention is not limited to this. For example, it may be sequentially displayed on the display device. Further, the functions of the respective processing units described in the first to tenth embodiments are provided as programs even if the functions are realized by dedicated hardware, and are provided as a memory in the image generating apparatus (not shown). Use CP
The function may be realized by being executed by U (central processing unit).

Incidentally, FIG. 1, FIG. 7, FIG. 10, FIG.
The procedure shown in FIG. 4, FIG. 16, FIG. 18, FIG. 19, FIG. 20, and FIG. 21 is executed by a program, and the program is provided by being recorded in a computer-readable recording medium such as a floppy disk, a CDROM, a ROM tape. The computer system executes the program and generates a screen. The “computer system” mentioned here includes an OS and hardware such as peripheral devices. Further, the "computer-readable recording medium" includes not only the above-mentioned portable medium such as a floppy disk and a CD-ROM but also a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" means to hold a program dynamically for a short time like a communication line when transmitting the program through a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system that serves as a server or a client in that case holds a program for a certain period of time. Further, the program may be for realizing a part of the functions described above,
Further, the functions described above may be realized by combining with the program already recorded in the computer system.

As described above, according to the present invention, an image having a natural depth of field similar to that of a real image can be generated at high speed by the same method as that of an actual camera, and the transparency /
It can handle translucent objects. Further, an image generated separately in advance such as a real shot can be naturally combined and used as a background in the depth of field, and a transparent / translucent object can also be handled.
Also, by giving an A value, a natural image such as a person image taken in a blue background or the like can be naturally combined in the depth of field, and a transparent / semi-transparent object can be handled.

[0088]

As described above, according to the image generating apparatus and method and the recording medium recording the program thereof according to the present invention, the following effects can be obtained. R as the pixel <br/> value, G, B, A, utilizing Maruchipure over Nbaffa comprising a plurality planes can store the values of Z, at least the coordinate transformation for each O Buje <br/> transfected model data, hidden surface processing,
A plurality of defocus data def Okasu process, are stored in the multi-plane buffer sequentially close in distance from the viewpoint. As a result, the image of a large blurred object existing in the foreground is blurred at the same time as it is seen through the edge depending on the focus distance from multiple defocus data, and images with a natural depth of field can be displayed at high speed. Can be generated. Also, from the shading data that has been subjected to coordinate conversion and hidden surface processing on the model data to be defocused, the focus is defocused according to the same function as the focal length of the actual camera, the aperture value, and the distance to the subject. The defocused image is generated using the defocus buffer. As a result, the same designation method (focal length of lens, F value,
An image with a natural depth of field similar to the photographed by distance) of Ru focus the subject, the earthenware pots by can be controlled by the same specified manner the actual camera.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an image generation apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a concept of an object.

FIG. 3 is a diagram showing a format and initial values of a Z buffer 3.

FIG. 4 is a diagram showing a format and initial values of a defocus buffer 6.

FIG. 5 is an explanatory diagram of defocus processing by geometrical optics.

FIG. 6 is a diagram showing a functional form of defocus processing by wave optics.

FIG. 7 is a flowchart showing the processing of the defocus processing unit 5.

FIG. 8 is a diagram showing a format and an initial value of a multi-plane buffer 8.

FIG. 9 is a diagram showing the concept of multi-plane buffer processing.

FIG. 10 is a flowchart showing the processing of the multi-plane buffer processing unit 7.

FIG. 11 is a diagram showing a format and initial values of the Z buffer 3 when reading R, G, B, A, and Z values from the Z buffer 3 to generate a defocused image.

FIG. 12 is a configuration diagram of an image generation apparatus according to a second embodiment of the present invention.

13 is a diagram showing the format and initial values of the background image buffer 15 in FIG.

FIG. 14 is a configuration diagram of an image generation apparatus according to a third embodiment of the present invention.

FIG. 15 is a diagram illustrating an outline of chroma key area extraction processing and RGBZ conversion processing according to the third embodiment.

FIG. 16 is a configuration diagram of an image forming apparatus according to a fourth embodiment of the present invention.

FIG. 17 is a diagram showing an outline of chroma key region extraction processing and RGBAZ conversion processing according to the fourth embodiment.

FIG. 18 is a configuration diagram of an image generation apparatus according to a seventh embodiment of the present invention.

FIG. 19 is a configuration diagram of an image generation apparatus according to an eighth embodiment of the present invention.

FIG. 20 is a configuration diagram of an image generation apparatus according to a ninth embodiment of the present invention.

FIG. 21 is a configuration diagram of an image generation apparatus according to a tenth embodiment of the present invention.

[Explanation of symbols]

1 Object model data read processing unit 2 Coordinate conversion and hidden surface removal processing section 3 Z buffer 4 Shading / mapping processing section 5 Defocus processing section 6 defocus buffer 7 Multi-plane buffer processing unit 8 multi-plane buffer 9 Output image creation processing unit 10 Disc writing processor 11 Storage device 12 Background image read processing unit 13 RGBZ conversion processing unit 14 switch 15 Background image buffer 16 Foreground image readout processor 17 Chromakey region extraction processing unit 18 RGBZ conversion processing unit 18 'RGBAZ conversion processing unit 31 Defocus data read processing unit 32, 42 Disc write processing unit 41 Shading data read processing unit 43 Model data read processing unit 21-27, 31-38 steps

Continuation of the front page (56) Reference JP-A 63-259778 (JP, A) JP-A 8-202869 (JP, A) JP-A 6-231273 (JP, A) JP-A 6-119462 (JP , A) JP 6-89090 (JP, A) JP 6-83977 (JP, A) JP 6-83955 (JP, A) JP 4-372079 (JP, A) JP 4-367083 (JP, A) Japanese Patent Laid-Open No. 4-362794 (JP, A) Special Table 9-507935 (JP, A) Naoki Asada et al., Analysis of optical properties of shield edge using back projection blur model, The Institute of Electronics, Information and Communication Engineers, Japan, The Institute of Electronics, Information and Communication Engineers, February 25, 1995, Vol. J78-DII No. 2, p. 248-262 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06T 15/40 200 G06T 15/00 100 JISST file (JOIS)

Claims (42)

(57) [Claims] 1. A plurality of objects, each of which is intended to represent a front-back relationship with respect to a viewpoint by utilizing a depth of field, which is a range in which the focus is practically focused before and after the focus, are used as object model data. An image generating apparatus for generating an image by inputting a plurality of defocus data that has undergone at least coordinate conversion, hidden surface processing of an object itself, and defocus processing for each pixel of model data, and has R, G, B, and A multi-plane buffer that can store A and Z values and consists of multiple planes, and multiple defocus data that has R, G, B, A, and Z values for each pixel, in order of decreasing distance from the viewpoint for each pixel. The multi-plane buffer processing unit that rearranges the data so that it is stored in the multi-plane buffer, and all the defocus data. After the processing by the multi-plane buffer processing unit is finished, the contents of the planes of the multi-plane buffer are combined to generate a target image, and an output image creation processing unit is provided. Image generating device. 2. A plurality of objects for which a front-back relationship with respect to a viewpoint is expressed by using the depth of field, which is a range in which the focus is practically in focus before and after the focus, are used as object model data. An image generation device for inputting object model data to generate an image, comprising a defocus buffer capable of storing R, G, B, A, and Z values as pixel values, and R, G, B, and A as pixel values. , Z
The multi-plane buffer consisting of multiple planes that can store the value of and the coordinate conversion for each pixel of one object model data,
The hidden surface processing and shading processing of the object itself are performed, and from the shading data having the R, G, B, and Z values as pixel values, the function consisting of the same focal length of the lens as the actual camera, the aperture value, and the distance to the object. A defocus processing unit that generates a plurality of defocus data having R, G, B, A, and Z values for each pixel for expressing a defocused image in accordance with the above, and stores the defocus data in the defocus buffer. And a multi-plane buffer processing unit that reads the defocus data from the defocus buffer and stores the defocus data in the multiplane buffer while rearranging the pixels so that the distance from the viewpoint is closer to each pixel , and all the defocus data. After the processing by the multi-plane buffer processing unit is completed, The contents of each plane of lane buffers synthesized to generate the image of interest, the image generating apparatus being characterized in that an output image creation processing unit for outputting. 3. The multi-plane buffer processing unit, where n is the number of planes of the multi-plane buffer, Z value Z _d (X, Y) and n at a pixel position (X, Y) on defocus data. Z value Z _m (X, Y) of the corresponding pixel position on the multi-plane buffer of the plane
_#N is compared, and if the former is smaller, the Z value Z of the image corresponding to each plane of the multi-plane buffer
Z _d from _m (X, Y) _{# 1} to Z _m (X, Y) _#n
Compared with (X, Y), Z _d (X, Y) <Z _m (X, Y)
_When the Z value of the corresponding image position of the N-th plane that satisfies _#N is found, the pixel values of the corresponding pixels on the N-plane and later on the multi-plane buffer are shifted one by one to the corresponding pixels on the subsequent plane, The pixel value of the defocus data is set to the pixel value of the N-th plane of the multi-plane buffer, thereby performing a process of storing the plurality of defocus data in the multi-plane buffer in ascending order of distance from the viewpoint. The image generation apparatus according to claim 1, wherein 4. The multi-plane buffer includes a minimum number of planes required in an initial stage, and the multi-plane buffer processing unit stores the plurality of defocus data in the multi-plane buffer in ascending order of distance from a viewpoint. 3. The image generation apparatus according to claim 1, wherein a necessary plane is newly secured every time the plane of the multi-plane buffer runs out when the storing process is performed. 5. The image generating apparatus according to claim 2 , wherein the function includes A as a parameter. 6. The defocus processing unit scans the shading data, calculates a diameter Φ of the point light source image when the point light source is blurred, and calculates an intensity ratio of the point light source image within a circle of the diameter. The strength ratio coefficient M is calculated so that the sum of the strength ratios in the circle of the diameter becomes 1.0, and the values of R, G, B in the circle of the diameter Φ are utilized by using the strength ratio coefficient M. , The value of the alpha value A is added to the value stored in the defocus buffer, and the Z value at the pixel position (a, b) in the circle of the diameter Φ on the defocus buffer is equal to the shading data. The Z values at the pixel position (X, Y) are compared, and if the Z value of the shading data is smaller, the Z value at the pixel position (X, Y) of the shading data is set to the diameter Φ on the defocus buffer. Defocus by moving the Z value in the circle within the pixel position (a, b)
The image generating apparatus according to claim 2, characterized in that for generating the data. 7. The image generation apparatus according to claim 6, wherein the defocus processing unit obtains the intensity ratio of the point light source image within the circle of the diameter by approximate calculation using cosine. 8. The image generating device comprises a Z buffer capable of storing R, G, B, and Z values as pixel values, coordinate conversion into the object model data, hidden surface processing,
It further comprises a coordinate conversion and hidden surface removal processing unit that performs shading processing and stores the result in the Z buffer, and the defocus processing unit processes the data stored in the Z buffer as the shading data. The image generation apparatus according to claim 2 . 9. The image generating device includes a Z buffer capable of storing R, G, B, A, and Z values as pixel values, coordinate conversion into the object model data, hidden surface processing,
It further comprises a coordinate conversion and hidden surface removal processing unit that performs shading processing and stores the result in the Z buffer, and the defocus processing unit processes the data stored in the Z buffer as the shading data. The image generation apparatus according to claim 2 . Wherein said object is generated by the pan-focus, also claim 1, characterized in that it comprises a background image infinity is given as Z value
Is an image generating apparatus described in 2 . 11. The object extracts a chroma key area, which is an area not used during image composition, with a threshold value given by a user, using a natural image, and extracts an infinite Z value in the extracted chroma key area. And 0 is given to each of R, G, and B as color information, and the R, G, and B values remain unchanged for other regions, and a foreground image given a Z value given by the user is included. The image generation device according to item 1 or 2 . 12. The image generation apparatus comprises a background image buffer capable of storing R, G, B, and Z values as pixel values, and an infinite distance as a Z value for the background image.
When the BZ conversion processing is performed and the background image is generated with the same shooting parameter as the object, the result of the RGBZ conversion processing is written in the background image buffer, and when the background image is generated by pan focus,
The output image creation processing unit further includes: a processing unit that writes the result of the RGBZ conversion processing to the Z buffer, and then writes the data that has been defocused by the defocus processing unit, to the background image buffer. The image generation apparatus according to claim 8 or 9, wherein the values of the background image buffer and the multi-plane buffer are read to generate an image, and the image is output. 13. The image generation device includes a chroma key area processing unit that extracts a chroma key area, which is an area that is not used during composition using a natural image, with a threshold value provided by a user, and an infinite number of extracted chroma key areas. 0 is given to each of R, G, and B as the far Z value and color information, and the R, G, and B values are left unchanged for the other areas, and the Z value given by the user is given and written to the Z buffer. The image generation apparatus according to claim 8, further comprising an RGBZ conversion processing unit. 14. The image generation apparatus extracts a chroma key area, which is an area not used at the time of composition, using a natural image, with a threshold value given by a user, and a boundary between the extracted area and another area in contact with the extracted area. Chroma key region processing unit that calculates A that changes gently in the region, and Z value at infinity to the extracted chroma key region and 0 is given to each of R, G, B, A as color information, and for other regions 10. An R, G, B value as it is, further comprising an RGBAZ conversion processing unit for giving the calculated A value and a Z value specified by a user and writing the Z value in the Z buffer. The image generation device according to 1. 15. A plurality of objects, each of which is desired to represent a front-back relationship with respect to a viewpoint by using a depth of field, which is a range in which the focus is practically in focus before and after the focus, are set as object model data, and one object is used. An image generation method for generating an image by inputting a plurality of defocus data subjected to at least coordinate conversion, hidden surface processing of an object itself, and defocus processing for each pixel of model data. A plurality of defocus data having A and Z values are rearranged so that the distance from the viewpoint is closer to each pixel, and R, G, B, A, and Z values can be stored as pixel values. Multi-plane buffer processing step to store in the multi-plane buffer and the multi-plane buffer for all defocus data After treatment with Ffa process step has been completed, the synthesized contents of each plane of multi-plane buffer to generate an image of interest, the image generation method characterized by comprising an output image creation processing step of outputting. 16. A plurality of objects for which a front-back relationship with respect to a viewpoint is expressed by using the depth of field, which is a range in which the focus is practically in focus before and after the focus, are used as object model data. An image generation method for inputting object model data to generate an image, in which coordinate conversion is performed for each pixel of one object model data,
The hidden surface processing and shading processing of the object itself are performed, and from the shading data having R, G, B, and Z values as pixel values, a function consisting of the same lens focal length, aperture value, and object distance as the actual camera. According to the above, a plurality of defocus data having values of R, G, B, A, and Z for expressing a defocused defocused image is generated, and R, G, B, A, and P are set as pixel values.
A defocus processing step of storing the value of Z in a defocus buffer, reading the defocus data from the defocus buffer, and rearranging the pixels so that the distance from the viewpoint is closer to each pixel value. As R, G, B, A,
A multi-plane buffer processing step of storing the value of Z in a multi-plane buffer composed of a plurality of planes, and after the processing by the multi-plane buffer processing step for all defocus data is completed, An image generation method comprising: an output image creation processing step of synthesizing contents to generate a target image and outputting the image. 17. In the multi-plane buffer processing step, when the number of planes in the multi-plane buffer is n, Z values Z _d (X, Y) and n at a pixel position (X, Y) on the defocus data are set. Z value Z _m (X, Y) of the corresponding pixel position on the multi-plane buffer of the plane
_#N is compared, and if the former is smaller, the Z value Z of the image corresponding to each plane of the multi-plane buffer
Z _d from _m (X, Y) _{# 1} to Z _m (X, Y) _#n
Compared with (X, Y), Z _d (X, Y) <Z _m (X, Y)
_When the Z value of the corresponding image position of the N-th plane that satisfies _#N is found, the pixel values of the corresponding pixels on the N-plane and later on the multi-plane buffer are shifted one by one to the corresponding pixels on the subsequent plane, The pixel value of the defocus data is set to the pixel value of the N-th plane of the multi-plane buffer, thereby performing a process of storing the plurality of defocus data in the multi-plane buffer in ascending order of distance from the viewpoint. 16. The method according to claim 15,
Alternatively, the image generation method according to Item 16. 18. The multi-plane buffer comprises a minimum number of planes required in an initial stage, and the multi-plane buffer processing step stores the plurality of defocus data in the multi-plane buffer in ascending order of distance from a viewpoint. 17. The image generation method according to claim 15, wherein when performing the storing process, a necessary plane is newly secured every time the plane of the multi-plane buffer runs out. 19. The image generation method according to claim 16 , wherein the function includes A as a parameter. 20. The defocus processing step scans the shading data, calculates a diameter Φ of the point light source image when the point light source is blurred, and calculates an intensity ratio of the point light source image within a circle of the diameter. The strength ratio coefficient M is calculated so that the sum of the strength ratios in the circle of the diameter becomes 1.0, and the values of R, G, B in the circle of the diameter Φ are utilized by using the strength ratio coefficient M. , The value of the alpha value A is added to the value stored in the defocus buffer, and the Z value at the pixel position (a, b) within the circle of the diameter Φ on the defocus buffer is the same as the shading data. The Z value at the pixel position (X, Y) is compared, and if the Z value of the shading data is smaller, the Z value at the pixel position (X, Y) of the shading data is set to the previous value on the defocus buffer. Defocus by shifting to the Z value at the pixel position (a, b) within the circle of the diameter Φ
Image generating method according to claim 16, characterized in that for generating the data. 21. The image generation method according to claim 20, wherein in the defocusing step, the intensity ratio of the point light source image within the circle of the diameter is obtained by approximate calculation using cosine. 22. The image generating method, the object model data, coordinate conversion, hidden surface processing,
Shading processing is performed, and the result is used as pixel values for R, G,
The method further comprises a coordinate conversion and hidden surface removal processing step of storing values of B and Z in a Z buffer, wherein the defocus processing step processes the data stored in the Z buffer as the shading data. The image generation method according to claim 16 . 23. The image generating method, the object model data, coordinate conversion, hidden surface processing,
Shading processing is performed, and the result is used as pixel values for R, G,
The method further comprises coordinate conversion and hidden surface removal processing step of storing the values of B, A, and Z in a Z buffer, wherein the defocus processing step processes the data stored in the Z buffer as the shading data. The image generation method according to claim 16 , wherein: 24. The object is generated by the pan-focus, claim 15, characterized in that it comprises a background image infinity is given as the Z value or
Or the image generation method described in 16 . 25. Using the natural image, the object extracts a chroma key region, which is a region not used at the time of image composition, with a threshold value given by the user, and the Z value at infinity in the extracted chroma key region. And 0 is given to each of R, G, and B as color information, and the R, G, and B values remain unchanged for other regions, and a foreground image given a Z value given by the user is included. Item 15. The image generation method according to Item 15 or 16 . 26. The image generation method according to claim 1, wherein the background image is given an infinity as a Z value,
BZ conversion processing step, and when the background image is generated with the same shooting parameters as the object, a background image buffer that can store R, G, B, and Z values with the result of the RGBZ conversion processing as pixel values Writing step, and if the background image is generated by pan focus, the result of the RGBZ conversion processing is written in the Z buffer, and then the data defocused in the defocus processing step is written in the background image buffer. 24. A processing step, further comprising: wherein the output image creation processing step reads values of the background image buffer and the multi-plane buffer, generates an image, and outputs the image. Image generation method. 27. The image generation method comprises a chroma key region processing step of extracting a chroma key region, which is a region that is not used at the time of composition, using a natural image, with a threshold value given by a user, and the extracted chroma key region is infinite. 0 is given to each of R, G, and B as the far Z value and color information, and the R, G, and B values are left unchanged for the other areas, and the Z value given by the user is given and written to the Z buffer. The image generation method according to claim 22, further comprising an RGBZ conversion processing step. 28. The image generating method extracts a chroma key area, which is an area not used at the time of composition, using a natural image, with a threshold value given by a user, and a boundary between the extracted area and another area in contact with the area. Chromakey region processing step to calculate A that changes gently in the region, and give Z value at infinity to the extracted chromakey region and give 0 to each of R, G, B and A as color information, and for other regions 24. The method according to claim 23, further comprising an RGBAZ conversion processing step of giving the calculated A value and the Z value specified by a user and writing the Z value in the Z buffer while keeping the R, G, and B values as they are. The described image generation method. 29. A plurality of objects for which a front-back relationship with respect to a viewpoint is expressed by using the depth of field, which is a range in which the focus is practically in focus before and after the focus, are object model data, and one object is used. A computer-readable recording medium that records an image generation program that generates an image by inputting a plurality of defocus data that has undergone at least coordinate conversion for each pixel of model data, hidden surface processing of the object itself, and defocus processing. A plurality of defocus data having R, G, B, A, and Z values for each pixel are rearranged in order of decreasing distance from the viewpoint for each pixel, and R, G, B, A, and A multi-plane buffer processing step of storing the value of Z in a multi-plane buffer composed of a plurality of planes; After the processing by the multi-plane buffer processing step for all the defocus data is completed, the contents of each plane of the multi-plane buffer are combined to generate a target image, and an output image creation processing step is output. A recording medium storing an image generation program, which is realized by 30. A plurality of objects for which a front-back relationship with respect to a viewpoint is expressed by using the depth of field, which is a range in which the focus is practically in focus before and after the focus, are set as object model data, and the plurality of objects are set. A computer-readable recording medium recording an image generation program for inputting object model data and generating an image, in which coordinate conversion is performed for each pixel of one object model data,
The hidden surface processing and shading processing of the object itself are performed, and from the shading data having R, G, B, and Z values as pixel values, a function consisting of the same lens focal length, aperture value, and object distance as the actual camera. According to the above, a plurality of defocus data having values of R, G, B, A, and Z for expressing a defocused defocused image is generated, and R, G, B, A, and P are set as pixel values.
A defocus processing step of storing the value of Z in a defocus buffer, reading the defocus data from the defocus buffer, and rearranging the pixels so that the distance from the viewpoint is closer to each pixel value. As R, G, B, A,
A multi-plane buffer processing step of storing the value of Z in a multi-plane buffer composed of a plurality of planes, and after the processing by the multi-plane buffer processing step for all defocus data is completed, A recording medium recording an image generation program, which causes a computer to realize an output image generation processing step of synthesizing contents to generate a target image and outputting the image. 31. In the multi-plane buffer processing step, when the number of planes in the multi-plane buffer is n, Z values Z _d (X, Y) and n at a pixel position (X, Y) on the defocus data are set. Z value Z _m (X, Y) of the corresponding pixel position on the multi-plane buffer of the plane
_#N is compared, and if the former is smaller, the Z value Z of the image corresponding to each plane of the multi-plane buffer
Z _d from _m (X, Y) _{# 1} to Z _m (X, Y) _#n
Compared with (X, Y), Z _d (X, Y) <Z _m (X, Y)
_When the Z value of the corresponding image position of the N-th plane that satisfies _#N is found, the pixel values of the corresponding pixels on the N-plane and later on the multi-plane buffer are shifted one by one to the corresponding pixels on the subsequent plane, The pixel value of the defocus data is set to the pixel value of the N-th plane of the multi-plane buffer, thereby performing a process of storing the plurality of defocus data in the multi-plane buffer in ascending order of distance from the viewpoint. 30. The method according to claim 29,
A recording medium having the image generation program described in 30 recorded therein. 32. The multi-plane buffer includes a minimum number of planes required in an initial stage, and the multi-plane buffer processing step stores the plurality of defocus data in the multi-plane buffer in ascending order of distance from a viewpoint. The recording medium recording the image generation program according to claim 29 or 30, wherein a necessary plane is newly secured each time the plane of the multi-plane buffer runs out when the processing of storing is performed. 33. The function contains the A as a parameter
Recording medium recording an image generating program according to claim 30, wherein no possible. 34. The defocus processing step scans the shading data, calculates a diameter Φ of the point light source image when the point light source is blurred, and calculates an intensity ratio of the point light source image within a circle of the diameter. The strength ratio coefficient M is calculated so that the sum of the strength ratios in the circle of the diameter becomes 1.0, and the values of R, G, B in the circle of the diameter Φ are utilized by using the strength ratio coefficient M. , The value of the alpha value A is added to the value stored in the defocus buffer, and the Z value at the pixel position (a, b) within the circle of the diameter Φ on the defocus buffer is the same as the shading data. The Z value at the pixel position (X, Y) is compared, and if the Z value of the shading data is smaller, the Z value at the pixel position (X, Y) of the shading data is set to the previous value on the defocus buffer. Defocus by shifting to the Z value at the pixel position (a, b) within the circle of the diameter Φ
The recording medium recording the image generation program according to claim 30 , wherein the recording medium records data. 35. The image generation program according to claim 34, wherein in the defocus processing step, an intensity ratio of a point light source image within a circle of the diameter is obtained by an approximate calculation using a cosine. recoding media. 36. The image generation program, the object model data, coordinate conversion, hidden surface processing,
Shading processing is performed, and the result is used as pixel values for R, G,
The method further comprises a coordinate conversion and hidden surface removal processing step of storing values of B and Z in a Z buffer, wherein the defocus processing step processes the data stored in the Z buffer as the shading data. A recording medium recording the image generation program according to claim 30 . 37. The image generation program, the object model data, coordinate conversion, hidden surface processing,
Shading processing is performed, and the result is used as pixel values for R, G,
The method further comprises coordinate conversion and hidden surface removal processing step of storing the values of B, A, and Z in a Z buffer, wherein the defocus processing step processes the data stored in the Z buffer as the shading data. A recording medium recording the image generation program according to claim 30 . 38. The object is generated by the pan-focus, claim 29, characterized in that it comprises a background image infinity is given as the Z value or
Or a recording medium recording the image generation program described in 30 . 39. The object extracts a chroma key area, which is an area not used at the time of image composition, with a threshold value given by a user, using a natural image, and extracts a Z value at infinity from the extracted chroma key area. And 0 is given to each of R, G, and B as color information, and the R, G, and B values remain unchanged for other regions, and a foreground image given a Z value given by the user is included. A recording medium recording the image generation program according to Item 29 or 30 . 40. The image generation program sets the background image to RG by giving infinity as a Z value.
BZ conversion processing step, and when the background image is generated with the same shooting parameters as the object, a background image buffer that can store R, G, B, and Z values with the result of the RGBZ conversion processing as pixel values Writing step, and if the background image is generated by pan focus, the result of the RGBZ conversion processing is written in the Z buffer, and then the data defocused in the defocus processing step is written in the background image buffer. 38. The output image creation processing step further includes a processing step, wherein the output image creation processing step reads values of the background image buffer and the multi-plane buffer, generates an image, and outputs the image. Recording medium recording the image generation program of. 41. The image generation program includes a chroma key region processing step of extracting a chroma key region, which is a region not used at the time of composition, using a natural image, with a threshold value given by a user, and the extracted chroma key region is infinite. 0 is given to each of R, G, and B as the far Z value and color information, and the R, G, and B values are left unchanged for the other areas, and the Z value given by the user is given and written to the Z buffer. The recording medium storing the image generation program according to claim 36, further comprising: a RGBZ conversion processing step. 42. The image generation program extracts a chroma key area, which is an area that is not used at the time of composition, by using a natural image, with a threshold value given by a user, and a boundary between the extracted area and another area in contact with the area. Chromakey region processing step to calculate A that changes gently in the region, and give Z value at infinity to the extracted chromakey region and give 0 to each of R, G, B and A as color information, and for other regions The computer is further allowed to implement the RGBAZ conversion processing step of giving the calculated A value and the Z value specified by the user and writing the Z value in the Z buffer while keeping the R, G, and B values as they are. A recording medium recording the image generation program according to Item 37.