JP5321385B2 - Image processing program and computer-readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate a reflection image that shows the surroundings reflected on a wet road surface, etc., in the rain, by simple processing. <P>SOLUTION: A computer functions as: a coordinate converting means for projecting onto a projection plane an object which is arranged inside a view frustum based on a virtual point of view; a holding means for holding color information and depth information of the object in a memory; a calculating means for calculating the normal line value and coordinate value of a reflection object corresponding to a pixel when the pixel on the reflection object is processed; a reflection line segment calculating means for determining in a virtual space, a reflection line segment beginning with the coordinate value; an intersection determining means for determining whether there is an intersection coordinate where the reflection segment intersects the object in the view frustum; and a pixel color reflecting means for reflecting the held color information of a reflection pixel on the color information of the pixel when there is the intersection coordinate, and reflecting predetermined color information on the color information of the pixel when there is no intersection coordinate. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a display image generation technique in a video game, a CG (Computer Graphics) video, or the like.

  In video games, CG videos, and the like, there are often scenes that require an image in which the surrounding scenery is reflected by a road surface wet with rain, a water surface, or a mirror surface.

  Patent Document 1 discloses a technique for realizing real-time glitter (glitter) on the water surface caused by reflection of light from the sun with a small processing load. However, it only represents the sparkle of the water surface and does not represent an image of the reflection of the surrounding scenery on the water surface.

  On the other hand, Patent Document 2 discloses a technique for generating a specular reflection image. That is, after the first three-dimensional image data is generated, second three-dimensional image data obtained by specularly reflecting the first three-dimensional image data is generated, and the second three-dimensional image data is displayed in a cutout manner. Set the area. Then, after the first and second 3D image data is converted into the first and second 2D image data, the second 2D image data in the cut-out display area is added to the first 2D image data. draw.

Japanese Patent No. 4001556 JP 2007-259942 A

  According to the technique disclosed in Patent Document 2 described above, a specular reflection image can be displayed on the screen. However, the first three-dimensional image data is separated from the basic first three-dimensional image data. There is a problem in that the second three-dimensional image data subjected to specular reflection must be generated and converted to the second two-dimensional image data, resulting in an excessive processing load.

  In addition, although only the image in the cutout display area is finally displayed in the second 2D image data, other overall 3D image data and 2D image data are generated. Therefore, there is a problem that the processing contents are wasteful.

  Furthermore, in generating the second 3D image data obtained by specularly reflecting the first 3D image data, the object is uniformly arranged at a position to be mirrored with respect to a flat mirror surface such as a floor surface. In the case where there are reflectors whose inclination changes in the middle or a plurality of reflectors having different slopes, a specular reflection image must be generated individually, which causes a problem that the processing becomes too complicated.

  The present invention has been proposed in view of the above-described conventional problems, and an object of the present invention is to generate an image in which the surrounding scenery is reflected by a road surface wet with rain by a simple process. An image processing program and a computer-readable recording medium are provided.

  In order to solve the above problems, according to the present invention, as described in claim 1, a computer sets a virtual viewpoint for a virtual space in which an object including at least a reflective object is arranged, and Coordinate conversion means for projecting an object arranged inside the viewing frustum with respect to the virtual viewpoint onto the projection plane, and color information and depth information of the object in the virtual space are held in the memory for each pixel of the projection plane Holding means, calculation means for calculating a normal value and a coordinate value of the reflection object corresponding to the pixel in processing the pixel on the reflection object, the calculated coordinate value and normal value, and the virtual viewpoint Based on the position of the reflection line segment calculation means for obtaining the reflection line segment starting from the coordinate value in the virtual space, in the viewing frustum, An intersection determination means for determining whether or not there is an intersection coordinate where the reflection line segment intersects with the object. When it is determined that there is the intersection coordinate, a reflected pixel corresponding to the closest intersection coordinate is calculated from the coordinate value. And pixel color reflecting means for reflecting the color information of the reflected pixel held in the color information of the pixel and reflecting predetermined color information in the color information of the pixel when it is determined that there is no intersection coordinate. The gist of the image processing program is to function as

  Moreover, as described in claim 2, in the image processing program according to claim 1, the intersection determination unit obtains an intersection between the reflection line segment and the surface of the viewing frustum, and A plurality of sample points are set between the start point of the reflection line segment and the intersection point, two adjacent sample points are determined from the start point side of the reflection line segment, and on the projection plane corresponding to the sample point Two sample pixels are calculated, the depth values of the two sample pixels are obtained from the memory, the coordinate values in the virtual space of the object corresponding to the two sample pixels are calculated from the depth values, and the 2 It is determined whether or not a line segment connecting two coordinate values intersects with the reflection line segment. When there is an intersection with the reflection line segment, the determination is performed with the coordinates of the intersected point as the intersection coordinate. If there is no intersection with the reflection line segment, the next two sample points are determined and the determination is repeated until the reflection line segment is determined by the determination based on all the set sample points. If there is no intersection, it can be determined that there is no intersection coordinate and the determination can be terminated.

  According to a third aspect of the present invention, in the image processing program according to the first or second aspect, the computer is a normal map for changing normal information of the reflection object. And a means for changing the normal value of the pixel based on the normal value of the pixel corresponding to the reflective object and the normal value of the normal map corresponding to the pixel. Can be.

  Also, as described in claim 4, in the image processing program according to any one of claims 1 to 3, the pixel color reflecting means is based on a normal value of the reflective object. The luminance in the color information of the pixel of the reflective object can be changed.

  In addition, as described in claim 5, the image processing program according to any one of claims 1 to 4 can be configured as a computer-readable recording medium.

  With the image processing program and computer-readable recording medium of the present invention, it is possible to generate an image in which the surrounding scenery is reflected by a road surface wet with rain or the like by simple processing.

It is a figure which shows the structural example of the image processing apparatus concerning one Embodiment of this invention. It is a flowchart which shows the example of a production | generation process of a rainy road surface height texture. It is a figure which shows the example of a rainy road surface height texture. It is a flowchart which shows the example of the production | generation process of a rain road surface normal texture. It is a figure which shows the example of a rain road surface normal texture. It is a flowchart which shows the example of polygon drawing processes, such as a wet road surface. It is a figure which shows the example of view space and projection space. It is a figure (the 1) which shows a processing example. It is FIG. (2) which shows a processing example. It is FIG. (3) which shows a process example. It is a figure (the 1) which shows the example of a production | generation image. It is a figure (the 2) which shows the example of a generated image. FIG. 11 is a diagram (part 3) illustrating a generated image example;

  Hereinafter, preferred embodiments of the present invention will be described.

<Configuration>
FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to an embodiment of the present invention.

  In FIG. 1, the image processing apparatus includes a 3D application (3D application program) 2 including a drawing command of a 3D (3 Dimension) object such as a game application, and a 3D such as OpenGL or Direct3D that receives a drawing command or the like from the 3D application 2. An API (Application Program Interface) 3 and a GPU (Graphics Processing Unit) 4 that executes drawing processing are provided. Although not shown, the execution environment of the 3D application 2 and 3D-API 3 is a general computer CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), HDD ( Hardware resources such as Hard Disk Drive) are provided.

  The GPU 4 receives a GPU command and a data stream from the 3D-API 3, performs a coordinate transformation (matrix transposition) and the like by receiving 3D vertex data from the GPU front end 41 and projecting it to the 2D screen space. A programmable vertex processor (VS) 42 and a basic assembly unit 43 that assembles a vertex index stream supplied from the GPU front-end unit 41 and vertex data coordinate-converted by the programmable vertex processor 42 are provided. The parts of the programmable vertex processor 42 and the basic assembly unit 43 are called vertex shaders.

  In addition, the GPU 4 performs rasterization / interpolation unit 44 that performs rasterization and interpolation from the polygon, line, and point data assembled by the basic assembly unit 43, and rasterized pixel (fragment) data from the rasterization / interpolation unit 44. A raster processing (pixel data of pixel data) is received from a programmable pixel processor (PS: Pixel Processor) 45 that receives and performs texture mapping, a pixel position stream given from the rasterizing / interpolating unit 44, and texture mapped data given from the programmable pixel processor 45. And a raster calculation unit 46 that performs arrangement on the screen. The portions of the rasterizing / interpolating unit 44, the programmable pixel processor 45, and the raster computing unit 46 are called pixel shaders.

  Further, in the GPU 4, the rendering contents are written / updated by the raster calculation unit 46, and the rain road surface height texture 471, the rain road surface normal texture 472, and the depth information texture 473 that can be referred to / updated from each unit in the GPU 4. And a color information texture 474 and a frame buffer 48 into which drawing contents are written / updated by the raster computing unit 46.

  The rain road surface height texture 471, the rain road surface normal texture 472, the depth information texture 473, and the color information texture 474 are used in drawing processing of a wet road surface or the like. The rain road surface height texture 471 and the rain road surface normal texture 472 will be described later, including generation processing thereof.

  The depth information texture 473 is a texture that holds depth information (z-direction distance from the viewpoint) of each pixel of the drawing frame. The color information texture 474 is a texture that holds color information of each pixel of the drawing frame. The texture is map data that holds values corresponding to the U coordinate (generally 0 ≦ U ≦ 1) and the V coordinate (generally 0 ≦ V ≦ 1) of the texture space. Since the texture generally has a format for holding color information data, when information other than the color information is held, it is converted into the color information format and stored.

  The content written in the frame buffer 48 is periodically read out and converted into a video signal and displayed by a monitor device or the like.

  Data necessary for processing is held in the data holding unit 5. Main data includes model data, model posture, viewpoint, viewing direction, angle of view, and external color. The model data includes materials, vertex coordinates, normals, textures, and vertex colors.

  The model data is polygon data representing the form of the model in the virtual three-dimensional space in the standard posture. A model that is instantiated and placed in a virtual three-dimensional space may be referred to as an object. The material is information for distinguishing the type of model, and indicates whether it is a road (road surface) that reflects when it gets wet due to rain or a mirror surface. The vertex coordinates are coordinates (x coordinate, y coordinate, z coordinate) of the polygon of the model. The normal is a vector value indicating the direction in which each vertex faces. The texture is image data that is pasted on the surface of the polygon. The vertex color is a value such as RGB (Red Green Blue) indicating the color of each vertex.

  The model posture is a value indicating the position and rotation angle of each model in the virtual three-dimensional space. The viewpoint, the viewing direction, and the angle of view are the three-dimensional position, the line-of-sight center direction, and the viewing angle of the virtual camera (virtual viewpoint) when the model in the virtual three-dimensional space is drawn as a two-dimensional image on the projection plane. The external color is a sky color or the like, and is used as a color of reflected light when there is no reflection from another model.

  Although the configuration using the GPU 4 for the image processing apparatus 1 has been described, it goes without saying that the present invention can also be applied to a case where a graphic engine is realized by software on general-purpose computer hardware. In this case, the function of the GPU 4 is realized by software like the 3D application 2 and the 3D-API 3, and the textures 471 to 474 and the frame buffer 48 are arranged on a memory (RAM).

<Operation>
FIG. 2 is a flowchart showing an example of the generation process of the rain road surface height texture 471. The rain road surface height texture 471 is a texture indicating the thickness (height) of the water film in order to give the road surface or the like of the road fluctuation due to the water film. The generation process of the rain road surface height texture 471 is mainly performed by a pixel shader.

  In FIG. 2, when processing is started every frame or every several frames (step S101), noise indicating a water pattern is drawn on the entire rain road surface height texture 471 (step S102).

  Next, an arbitrary number of ripples due to rainfall are drawn on the rain road surface height texture 471 (step S103), and the process is terminated (step S104).

  FIG. 3 is a diagram showing an example of a rainy road surface height texture 471, which shows the rainy road surface height by color. In the figure, the white part is a part with a high height, and the black part is a part with a low height.

  FIG. 4 is a flowchart showing an example of the generation process of the rain road surface normal texture 472. The rain road surface normal texture 472 is a texture that indicates the normal of the surface of the water film in order to give fluctuation to the road surface of the road due to the water film. The rain road surface normal texture 472 is generated based on the rain road surface height texture 471 described above. The generation process of the rain road surface normal texture 472 is mainly performed by a pixel shader.

  In FIG. 4, when processing is started every frame or every several frames (step S111), a 2D inclination vector in the texture space is calculated from the value of the rain road surface height texture 471 around each pixel for each individual pixel (step S111). Step S112).

  Next, a normal vector is calculated from the 2D inclination vector on the texture space, and is output as a color value to the rain road surface normal texture 472 (step S113), and the process is terminated (step S114).

  FIG. 5 is a diagram illustrating an example of the rain road surface normal texture 472, and the rain road surface normal is indicated by a color. In the figure, the white portion is a portion where the normal is perpendicular to the road surface and the black portion is the portion where the normal is parallel to the road surface.

  FIG. 6 is a flowchart showing an example of polygon drawing processing for a wet road surface or the like. The polygons of other models are subjected to normal drawing processing and synthesized on the frame buffer 48 with the drawing processing results of polygons such as wet road surfaces.

  In FIG. 6, when the processing is started every frame or every several frames (step S201), the vertex shader is converted into a view space and a projection space, rasterized, etc. as processing by the vertex shader (step S202).

  FIG. 7A shows an example of a view space. The view space is a coordinate space based on the viewpoint (virtual camera) VP, and the horizontal axis is the x axis, the vertical axis is the y axis, and the depth is the z axis with respect to the viewing direction from the viewpoint VP. A near clip plane CP1 is set on the side closer to the viewpoint VP, and a far clip plane CP2 is set on the far side within the range where the angle of view is swung in the x axis direction and the y axis direction with respect to the viewing direction of the viewpoint VP. Is done. The hexahedron surrounded by the near clip plane CP1 and the far clip plane CP2 and the plane showing the angle of view is called a view frustum, and a model (object) arranged inside is a drawing target. In the case of a video game, the position of the viewing frustum is determined so as to include a player character operated by an operation means by a player (player). FIG. 7B shows an example of a projection space, and the view space shown in FIG. 7A is converted into ranges of −1 ≦ x ≦ 1, −1 ≦ y ≦ 1, and 0 ≦ z ≦ 1. It is a thing.

  Next, returning to FIG. 6, processing for each pixel by the pixel shader is performed (step S203).

  First, a post-change processing point normal is calculated from the processing point normal (the normal value of the object at the pixel position to be processed) and the rain road surface normal texture 472 (step S204). That is, a post-change processing point method in which a fluctuation due to a water film is given by adding the normal value corresponding to the rain road surface normal texture 472 in the UV space to the original normal value (vector value) of the object. Find a line. FIG. 8A conceptually shows the processing contents. If the original normal is N for the processing point PX of the road object OB, the post-change processing point is changed by the change using the rain road surface normal texture 472. A normal line N ′ is obtained.

  Next, returning to FIG. 6, the reflection direction vector is calculated from the processing point coordinates, the post-change processing point normal, and the line-of-sight vector (step S205). FIG. 8B conceptually shows the processing contents, and the reflection direction θ is the same reflection angle θ as the incident angle θ formed by the line-of-sight vector VV from the viewpoint VP toward the processing point PX and the post-change processing point normal N ′. The vector RV is obtained from the geometric relationship.

  Next, returning to FIG. 6, the farthest reflection coordinate which is the intersection of the reflection line segment determined from the processing point coordinates and the reflection direction vector and the view frustum is calculated (step S206). FIG. 8C conceptually shows the processing contents, and the intersection of the reflection line segment RL obtained by extending the reflection direction vector RV and the view frustum is obtained from the geometric relationship as the farthest reflection coordinate MP.

  Next, returning to FIG. 6, n 3D sample points (S (1), S (2),..., S (n)) from the processing point coordinates to the farthest reflection coordinates are extracted from the processing point coordinates. They are set in order of closeness (step S207). FIG. 9A conceptually shows the processing content, and the 3D sample points S (1), S (2), S between the processing point PX and the farthest reflection coordinate MP along the reflection line segment RL. The state where (3),... Are set is shown.

  Next, returning to FIG. 6, "1" is set to the variable i (step S208).

  Next, texture space 2D coordinates (similar to projection plane coordinates) UV (i) and UV (i + 1) of 3D sample points S (i) and S (i + 1) are calculated, and depth values D (i) and D of the positions are calculated. (I + 1) is acquired from the depth information texture 473 (step S209). FIG. 9B conceptually shows the processing contents and shows a state in which the texture space 2D coordinates UV (1) and UV (2) corresponding to the 3D sample points S (1) and S (2) are obtained. ing. FIG. 9C shows a state where the depth values D (1) and D (2) are obtained from the depth information texture 473 based on the texture space 2D coordinates UV (1) and UV (2). Depth values D (1) and D (2) indicate the surface positions of the objects existing behind 3D sample points S (1) and S (2) as viewed from the viewpoint.

  Next, returning to FIG. 6, the 3D coordinate A (i) is calculated from the texture space 2D coordinate UV (i) and the depth value D (i), and similarly, the texture space 2D coordinate UV (i + 1) and the depth value D (i + 1). 3D coordinates A (i + 1) are calculated from (Step S210). FIG. 10A conceptually shows the processing contents, which overlap with 3D sample points S (1) and S (2) when viewed from the viewpoint VP, and depth values D (1) and D (2) from the viewpoint VP, respectively. The 3D coordinates A (1) and A (2) in FIG.

  Next, returning to FIG. 6, the intersection coordinates of the line segment composed of the 3D coordinates A (i) and A (i + 1) and the reflection line segment are calculated (step S211). FIG. 10B conceptually shows the processing contents, and shows a case where the line segment AL composed of the 3D coordinates A (1) and A (2) and the reflection line segment RL do not intersect. When intersecting, the intersecting coordinates are coordinates between two 3D sample points projected on the surface of the reflected object.

  Next, returning to FIG. 6, it is determined whether or not crossing coordinates exist (step S212). Whether or not an intersection coordinate exists can be determined by a well-known mathematical formula that determines whether or not an intersection exists between two line segments.

  If the intersection coordinates exist (Yes in step S212), the texture space 2D coordinates UV of the intersection coordinates are calculated, the color information is acquired from the color information texture 474, and is set as the output color to the frame buffer 48 (step S213). .

  On the other hand, when there is no intersection coordinate (No in step S212), 1 is added to the variable i (step S214).

  Next, it is determined whether or not the variable i has reached the maximum number n of 3D sample points (step S215).

  When the variable i is not equal to or greater than the maximum number n of 3D sample points (No in step S215), calculation of the texture space 2D coordinates UV (i) and UV (i + 1) and depth values D (i) and D (i + 1) Return to (step S209).

  If the variable i is equal to or greater than the maximum number n of 3D sample points (Yes in step S215), the external color is set as the output color to the frame buffer 48 (step S216). This is because when there is no reflected object, the output color should reflect a color such as sky.

  After setting the output color (steps S213 and S216), the luminance of the output color is changed according to the value of the post-change processing point normal (step S217). This is to eliminate the inconvenience that the output color changes greatly when the incident angle / reflection angle θ is small due to the value of the post-change processing point normal, for example, the display of the reflected image under the feet of the person becomes unstable. It is.

  After performing the above processing for all the pixels, the processing is terminated (step S218).

  11 to 13 are diagrams showing examples of generated images. FIG. 11 is a scene in which a planar road exists from the bottom to the top of the screen, and images of surrounding objects are naturally reflected on the road surface wet with rain. FIG. 12 shows a scene in which a planar road exists from the lower left to the upper right, and an inclined tread is placed at the end of the road in the center of the screen. In this case, a reflected image tilted according to the tilt is displayed on the tilted footboard. FIG. 13 shows a scene where there is a road that slopes to the upper right near the center from a flat road on the left side. Also in this case, a reflection image inclined according to the inclination is displayed on the road surface of the inclined road.

<Summary>
As described above, the present embodiment has the following advantages.
(1) It is not necessary to set a new object for the reflected image, and a drawing process for that purpose is not required, so the processing load is small.
(2) In the processing of individual pixels of a reflective object such as a wet road surface, the color information of the reflected object is obtained by tracing the reflection vector, and only the minimum processing necessary for display is performed. There is no waste.
(3) In order to identify the object to be reflected in consideration of the normal corresponding to each pixel of the reflective object such as a wet road surface, there are reflectors whose slope changes or multiple reflectors with different slopes. Even in this case, the reflection image can be drawn by uniform processing.
(4) A natural reflection on a wet road surface or the like can be expressed by giving a fluctuation by a water film to the normal line of the reflective object.
(5) By controlling the brightness of the reflected image in accordance with the normal value of the reflective object, it is possible to eliminate the unstable display of the reflected image when the incident angle / reflective angle is small.
(6) As a result, it is possible to generate an image in which the surrounding scenery is reflected by a road surface wet with rain by a simple process.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments, various modifications and changes may be made to the embodiments without departing from the broad spirit and scope of the invention as defined in the claims. Obviously you can. In other words, the present invention should not be construed as being limited by the details of the specific examples and the accompanying drawings.

1 Image processing device 2 3D application 3 3D-API
4 GPU
41 GPU front end unit 42 Programmable vertex processor 43 Basic assembly unit 44 Rasterization / interpolation unit 45 Programmable pixel processor 46 Raster operation unit 471 Rain road surface height texture 472 Rain road surface normal texture 473 Depth information texture 474 Color information texture 48 Frame buffer 5 Data holding part

Claims (5)

Computer
Coordinate conversion means for setting a virtual viewpoint for a virtual space in which an object including at least a reflection object is arranged, and projecting the object arranged inside the viewing frustum based on the virtual viewpoint onto a projection plane;
Holding means for holding, in a memory, color information and depth information of the object in the virtual space for each pixel of the projection plane;
Calculation means for calculating a normal value and a coordinate value of the reflective object corresponding to the pixel in processing the pixel on the reflective object;
Reflected line segment calculation means for obtaining a reflected line segment starting from the coordinate value in a virtual space based on the calculated coordinate value and normal value and the position of the virtual viewpoint,
An intersection determination means for determining whether or not there is an intersection coordinate where the reflection line segment intersects the object in the viewing frustum;
When it is determined that there is the intersection coordinate, the reflected pixel corresponding to the closest intersection coordinate from the coordinate value is calculated, and the color information of the reflected pixel that is held is reflected in the color information of the pixel, An image processing program for functioning as pixel color reflecting means for reflecting predetermined color information in the color information of the pixel when it is determined that there is no intersection coordinate. An image processing program according to claim 1,
The intersection determination means includes
Finding the intersection of the reflection line segment and the surface of the viewing frustum,
A plurality of sample points are set between the start point of the reflection line segment and the intersection point,
Two adjacent sample points are determined from the start point side of the reflection line segment, and two sample pixels on the projection plane corresponding to the sample points are calculated,
A line connecting the two coordinate values is obtained from the memory by obtaining the depth values of the two sample pixels, calculating coordinate values in the virtual space of the object corresponding to the two sample pixels from the depth values. Determine whether the minute intersects the reflection line,
When there is an intersection with the reflected line segment, the determination is terminated with the coordinates of the intersected point as the intersection coordinates,
If there is no intersection with the reflection line segment, determine the next two adjacent sample points and repeat the determination,
An image processing program for ending a determination that there is no intersection coordinate when there is no intersection with the reflection line segment by the determination based on all the set sample points. An image processing program according to any one of claims 1 and 2,
Computer
Means for creating a normal map for changing normal information of the reflection object;
An image processing program that functions as means for changing a normal value of the pixel based on a normal value of the pixel corresponding to the reflective object and a normal value of the normal map corresponding to the pixel. An image processing program according to any one of claims 1 to 3,
The pixel color reflecting means includes
An image processing program for changing brightness in color information of the pixel of the reflective object based on a normal value of the reflective object.   The computer-readable recording medium which records the image processing program as described in any one of Claims 1 thru | or 4.