KR20100075351A - Method and system for rendering mobile computer graphic - Google Patents

Abstract

PURPOSE: A method and a system for rendering a mobile computer graphic are provided to use a fixed point arithmetic processor to efficiently perform rendering at low power consumption. CONSTITUTION: A rendering server(210) determines the attributes of each apex of a geometric model for a 3 dimensional object through vertex shaping in a world coordinate system, and generates vertex data which indicates the attributes of each vertex. Plural rendering clients(220) perform by using the generated vertex and the pre-stored image. The plural rendering clients generate new images.

Description

Method and system for rendering mobile computer graphics {Method and System for Rendering Mobile Computer Graphic}

Geometry based rendering (hereafter referred to as 'GBR') is a camera that takes a wire frame mesh model created based on a geometric model of a three-dimensional graphic object. It is a technique of synthesizing as if made. In general, three-dimensional computer graphics refer to GBR, and realistic GBR requires a lot of computation and memory access depending on the complexity of the geometric model and the level of detail displayed.

Modern computer graphics include GBR as well as Image Based Rendering (hereinafter referred to as IBR). IBR is a technique that uses a number of pre-captured images to create images based on the location, time and environment of a new camera. At this time, when compositing an image suitable for a rendered scene by using captured images, image warping, image morphing, and perspective transformation are considered in consideration of coordinate shift, zoom, and rotation. (Visually transformed image) is most commonly used.

IBR and GBR are increasingly used for faster and more realistic rendering. In the case of animation, in particular, when an object or camera moves, it is more efficient to update a frame that was created in the past than to render a new frame each time.

When rendering an object (Geometric Object, Triangle, Polygon) in GBR, it is often used to attach a pre-captured image to the surface of the object to be rendered, rather than to synthesize the image according to the material properties of the object. . This technique is called texture mapping, and the image must be perspective corrected according to a given time. In this respect, texture mapping is very similar to IBR.

Meanwhile, in order to render computer graphics at a high speed, a graphics processing pipeline, which is a hardware-based accelerator such as FIG. 1, is conventionally used.

Referring to FIG. 1, the conventional graphics processing pipeline converts three vertices of an object to be rendered, for example, triangles, into coordinates on a canonical view volume based on a 4D homogeneous coordinate with the camera as the origin (S110). After the process of clipping (S120) to cut out the portion outside the view volume from the images indicated by these coordinates, the image is projected on the screen (S130), and the three vertices of the images projected on the screen are assigned. Color is sent to the screen by rasterization (S140) in the image by linear interpolation (Linear Interpolation) using the color.

Existing graphics processing pipeline transforms all geometric data into Canonical Eye Space with camera or eye origin in order to reduce hardware complexity, and project objects from 3D space to 2D space. In this case, we use homogeneous space that extends three-dimensional space into four dimensions. Using homogeneous space has the advantage of combining all the coordinate transformations required for Perspective Projection and multiplying them into a single 4 x 4 matrix. On the other hand, due to the nature of Homogeneous Projection, in order to prevent the object behind the camera from being flipped before rendering, a view frustum, which is a rendered space, is defined, and an object outside the space is defined. View Frustum Clipping must be preceded. In homogeneous spaces, clipping can be easily done with a hardware-based accelerator.

In a three-dimensional space, a wire frame is made up of numerous triangular pieces whose surface has different properties (such as color, brightness, texture, transparency, and reflectivity) for each vertex of each wire frame, depending on the characteristics of the object. Is given. Here, the given property is changed in color, brightness, etc. according to the environment of the object, the light source characteristics, the number of light sources. The existing graphics processing pipeline uses a relatively simple illumination model, which supplements the primary rendering using shaders in several operations as needed.

In the rasterization step, for each pixel coming into the triangle projected into the two-dimensional space, the appropriate color is determined using the attribute data defined at the three vertices of the triangle, and then written to the screen buffer. This step uses a hardware-based accelerator and computes the color of the three vertices by linear interpolation using a simple operator. The calculation by linear interpolation requires perspective correction interpolation to solve this problem, because the perspective distortion is severe when the object is near and distributed in a wide view. This distortion includes especially the case of texture mapping, which gives the user a sense of rejection.

Meanwhile, with the development of semiconductor technology, the computational speed and performance of the graphics processing unit (GPU) are rapidly increased, and the rendering technology also includes a program shader having a single instruction multiple data (SIMD) structure. In general, the ray tracing technique that uses many operations for more realistic rendering is becoming a trend.

Light-tracking technique sends out a ray of light that goes straight through the screen from the camera to the object to find the intersection with the object, and the light intensity and color that reaches the pixel by reflecting light reaching the intersection from various light sources. Calculate and sample

Light tracking techniques also apply when the object is a transparent, translucent and reflector. When a ray meets these objects, it uses the law of refraction or reflection to determine the path and direction of the new ray to emit a new ray, find the intersection of this new ray with another object, and repeat this process. You can get the final color. In addition, when calculating the color at the intersection, the contrast and color are adjusted according to the degree of light source visibility from the light source.

However, since the aliasing can occur by using the value of the point sampling, it can be solved by the super sampling which uses the pixel average value by exporting several rays per pixel. ) Is often used as an antialiasing method.

According to the present invention, in a mobile device environment requiring low power consumption, IBR and GBR are efficiently performed on a single platform to provide perspective correction and antialiasing using a true area average. An algorithm for rendering an image of a computer, a method and system for rendering a computer graphics for a mobile device to implement a hardware software architecture.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Communication Research and Development. [Task management number: 2007-S-026-02, Task name: Multi-format multimedia SoC based on MPCore platform] ]

As contents for multimedia terminals are advanced and diversified, mobile multimedia system on chip (SoC) requires high performance to produce visually faithful image quality in a short time with low power consumption. As a field of application of high quality multimedia terminals, games, animations and navigation are transmitted to the multimedia terminal in the form of compressed 3D graphic data in order to reduce the transmission bandwidth, and the multimedia terminal implements the image using the compressed data.

Graphic rendering, on the other hand, is the composition of scenes as seen by a particular camera. In this case, the pixel data of the synthesized image should use the light intensity reaching each pixel and the average value of each pixel area in consideration of the light sensitivity characteristics of the camera, not the center point sample of each pixel.

If the light intensity changes rapidly in a space less than the image sampling point interval, the image composed of the point samples may not visually exhibit the effect of the fast change and may be visually disturbed.

Aliasing can be reduced by sufficient sampling, but in point sampling, aliasing can occur at any time due to changes in the space smaller than the pixel size of the image. Severe distortion occurs in the area, especially pixel blinking in animation. To exclude this perspective distortion, area samples, not point samples, should be used.

In order to eliminate this aliasing, the fixed function graphics pipeline can be rasterized and processed using a post processing filter or supersampling by taking multiple samples per pixel to average. ) Method is used.

However, supersampling can also occur in some cases, and in the rasterization phase, when linear interpolation using vertex data is used to determine the color and brightness within a triangle, the object is near and distributed to a wider perspective. In this case, perspective distortion may not be represented properly, and perspective correction mapping may be used to solve the perspective. In addition, as IBR techniques are becoming more and more popular, they require perspective correction calculations.

Therefore, conventionally, a lot of operations are required to render in graphics processing, and a lot of data access, which is a problem that leads to power consumption and performance degradation.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object thereof is to provide a computer graphics rendering method and system for mobile to reduce the amount of computation when rendering computer graphics.

Another object of the present invention is to reduce memory access by rendering the computer graphics as much as possible to reuse the already accessed data, and to reduce the memory access contention by reducing the data correlation between each operator when using multiple parallel operators Computer graphics rendering method and system.

It is another object of the present invention to provide a computer graphics rendering method and system for mobile for rendering high quality images by efficiently performing IBR and GBR on one platform in a mobile device environment requiring low power consumption.

According to an aspect of the present invention, there is provided a rendering server configured to determine attributes of each vertex of a geometric model of an object displayed in three dimensions through vertex shading on a global coordinate system, and generate vertex data representing the attributes of each vertex; And a plurality of rendering clients performing rendering using the generated vertex data and the pre-stored image and generating a new image.

The present invention comprises the steps of determining whether all vertices of a given pixel on a global coordinate system are inside an image projected on the pixel grid plane; A rasterization step of calculating color values of each vertex of the pixel piece when the pixel spans a plurality of images by interpolation using an attribute of each vertex of the pixel piece; And dividing the sum of the color values of the vertices of the pixel fragment by the area of the pixel fragment * the number of vertices of the pixel fragment to obtain the color of the pixel.

As described above, according to the present invention, by providing a computer graphics rendering method and system for mobile that renders IBR and GBR on one platform, there is an effect of efficiently processing high quality images.

In addition, by using a fixed-point arithmetic processor when rendering computer graphics, rendering can be efficiently performed at low power consumption.

In addition, when the color of each vertex is used to interpolate differently depending on the distance of the object, it is possible to reduce the perspective distortion, thereby efficiently processing an application such as animation or game.

In addition, by dividing the two-dimensional space region arbitrarily without overlapping, and performing the GBR and IBR using the parallel vector operator for each divided region, there is an effect of reducing the memory access contention by reducing the data correlation between each operator.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in describing the present invention, if it is determined that the detailed description of the related well-known configuration or function may obscure the gist of the present invention, the detailed description thereof will be omitted.

Example

2 is a diagram illustrating a computer graphics rendering system for mobile according to one embodiment of the present invention.

As shown in FIG. 2, the computer graphics rendering system for mobile 200 according to the present invention is divided into a rendering server 210 and a rendering client 220.

The rendering client 220 according to the present invention may generate a new image using the compressed data and the pre-stored data, that is, the image, which are given from the rendering server 210, and may include some functions of the rendering server 210.

The rendering client 220 uses a fixed point arithmetic processor to efficiently perform rendering at low power consumption.

The rendering server 210 according to the present invention interfaces with an application program such as an animation or a game through an application program interface, and processes the geometric model in advance so that the rendering client 220 can render the rendering client 220. The compressed data may be compressed and transmitted, and the data required by the rendering client 220 may be transmitted in advance according to the rendering operation of the rendering client 220.

The computer graphics rendering system for mobile 200 according to the present invention defines a rendering server 210 as a geometry engine, a rendering client 220 as a rendering engine, and defines at least one rendering client. In addition, the rendering client may perform some functions of the rendering server 210.

In the rendering server 210 and the rendering client 220 according to the present invention, one rendering server 210 may have a plurality of rendering clients each using a given scene at several different viewpoints, different times, and different screen regions. .

The rendering server 210 determines an attribute including color, brightness, texture, transparency, and the like at each vertex of the given data in consideration of the lighting model and the surrounding environment on the World Coordinate System (WCS). This can be seen as a combination of vertex shaders and shadowing.

The rendering server 210 finishes this vertex shading and sends vertex data from which hidden surfaces have been removed for each rendering client 220.

The rendering client 220 according to the present invention determines a sampling grid based on camera data in the rendering server 210 and performs rendering based on vertex data transmitted by the geometric engine. A detailed method of performing rendering using vertex data in the rendering client 220 will be described later with reference to FIGS. 3 to 5.

FIG. 3 is a diagram for describing a method of obtaining an intersection of a ray passing through a pixel and a projected image on a global coordinate system.

3 uses the vertices and camera coordinates of the object 310 represented in three dimensions on the global coordinate system, and illustrates the relationship in which light passing through the pixels of the camera is reflected from the object and projected onto the pixels of the camera. That is, it indicates a point projected on the display screen 320 when light starting at an arbitrary point in the object 310 proceeds to the camera.

Referring to FIG. 3, a unit vector e _z is a unit vector defining a direction of a camera's line of sight, e _y is a unit vector defining an upward direction (y-axis) of camera space, and e _x is a horizontal axis (x of camera space). Axis, a unit vector that defines three unit base vectors in camera space.

When unit vectors of the camera space are expressed in coordinates of the global coordinate system, and the sample point x-axis and y-axis index values on the screen 320 are kx and ky, S (kx, ky) is a screen sample point expressed in the global coordinate system. (Screen Sample Point). Light tracking refers to a method of calculating a color of a ray-triangle intersection where the line of sight that passes from the camera origin E and passes through the pixels kx and ky meets an object. This intersection is calculated by Equation 1 using a vector cross product and a dot product.

The color at this intersection is obtained by linear interpolation of the values given at the vertices. Also, if there is no object where the rays kx and ky meet, the background color of the scene to be rendered is used as it is. In this way the color at all sample points is determined.

4 is a graph showing the relationship between the length and area of a pixel piece of the present invention.

Referring to FIG. 4, when a plurality of images span a given pixel, the length (λ) and area (a) of the pixel fragment are represented by a distance d 'between the straight line 410 and the pixel center 420. The length function 430 and the area function 440 of the fragment change differently according to the slope m of the straight line 410. The function is calculated in advance to make a look up table, and the color of the pixel. It is used to find.

FIG. 5 is a diagram for describing a method of calculating an intersection divided by a pixel on a global coordinate system.

Referring to FIG. 5, it is shown that when a projected image boundary of an object passes inside a pixel, several pixel pieces are generated, and the coordinates of all the vertices of the pixel pieces are calculated.

The color at the vertices of every pixel fragment can also be computed using the vertex values of the original image. It shows how to calculate the intersection with the plane that the line defined in 3D space expressed in global coordinate system is spatially divided by the x-axis pixel grid of the screen. In the same way, it is calculated in the x-screen direction, y-screen direction of every pixel grid, and is calculated correctly visually. Nx is a unit normal vector of the kx pixel grid plane, RA and RB are three-dimensional coordinates of the global coordinate system, and E represents a camera position on the global coordinate system. Color calculation at the intersection computes the color of points A and B by linear interpolation.

6 is an algorithm illustrating an anti-aliasing light tracking based rasterization method according to an embodiment of the present invention.

Referring to FIG. 6, the present invention proposes a rasterization based on a light tracking technique using pixel corner sampling (Pixel Coner Sampling). That is, when a pixel is divided by a plurality of projected images, the color values of each vertex are calculated by interpolating the color and brightness of each divided pixel fragment, and the pixel values are calculated using the color values of all the vertices and the area of the pixel fragments. The color area average value of is calculated.

7 is a flowchart illustrating a method for rendering computer graphics for mobile according to an embodiment of the present invention.

The computer graphics rendering method for mobile according to the present invention does not use a conventional Homogeneous Coordinate System, but uses light displayed in a global coordinate system (hereinafter referred to as 'WCS') to perform light tracking. By using the tracing technique and the advantages of the existing rasterization, a high quality screen is rendered at a high speed. Because WCS is used, it does not pre-clip the object behind the camera and uses the 3D vector operation rather than the 4D operation to perform the calculation required for rendering.

That is, the computer graphics rendering method for mobile according to the present invention divides the client display screen area, that is, the two-dimensional space area, without randomly overlapping several pieces, and uses geometric vector rendering by using a parallel vector operator for each divided area. Hereinafter, a rendering process is performed by performing 'GBR') and image based rendering (hereinafter referred to as 'IBR').

Referring to FIG. 7, it is determined whether four vertices of a given pixel are present in the projected image on the World Coordinate System (WCS) in consideration of the illumination model and the surrounding environment (S710). If four vertices exist within the projected image, the color values of each vertex of the pixel are calculated by interpolation using the properties of each vertex of the pixel (S712), and the average value of the color values of each vertex of the pixel is calculated. The color of the pixel is obtained (S714).

If the pixel given in step S710 spans multiple images, the color value of each vertex of the pixel piece is calculated by interpolation using the attributes of each vertex of the pixel piece (S720). In this case, when the perspective correction mode (Perspective Correct Mode) in which the object is close and distributed in a wide view according to the distance, depth width, and visual distribution with respect to the preset object is satisfied, Perspective Correct Interpolation is used. Compute the color values of each vertex of the pixel fragment and, if the object satisfies the Weak Perspective Mode, which is distributed far and narrowly, use linear interpolation to determine each vertex of the pixel fragment. Calculate the color value. That is, when the depth deviation between the projected image and the object (average deviation between the projected image area and the object area at various points of view) is 1/10 or more, Perspective Correct Interpolation, and the depth deviation between the projected imager and the object. Is less than 1/10, the linear interpolation (linear interpolation) to obtain the color value of each vertex of the pixel fragment.

Subsequently, the sum of the color values of all the vertices of the pixel fragment is divided by (area of the pixel fragment * the number of vertices of the pixel fragment) to obtain an average value of the color area of the pixel, that is, the color (S730). In other words, when a pixel is spread over multiple images, the pixel fragments for each image are always triangles or rectangles, and when the pixels are rectangles, the pixel pieces may be divided into triangles and parallelograms (diamonds, rectangles, etc.). Therefore, the color and brightness of all the vertices of the triangle and the rhombus can be calculated by interpolation, and the average value of the color area by the area of the triangle and the parallelogram can be calculated accurately. In detail, the color area average value of the triangle can be obtained by dividing the color and brightness of three vertices by three times the area of the triangle, and the color area average value of the square, rectangle, and rhombus is divided by four times the area. You can get it. In this way, the color area of each pixel can be calculated to calculate the color of the pixel.

The computer graphics rendering method for mobile according to the present invention is compared with the case where the number of samples to be calculated is sampling the center, the number of horizontal pixels) * (vertical pixels) to (horizontal pixels + 1) * (vertical pixels) +1), which increases with a probability of 1 / (horizontal pixels) + 1 / (vertical pixels), which is negligible as the total number of pixels increases.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

1 is a view showing a process of a conventional graphics processing pipeline;

2 is a diagram illustrating a computer graphics rendering system for mobile according to one embodiment of the present invention;

3 is a view for explaining a method for obtaining an intersection point of a ray passing through a pixel and a projected triangle on a global coordinate system;

4 is a graph showing the relationship between the length and area of a pixel piece of the present invention;

5 is a diagram for describing a method of calculating an intersection divided by a pixel on a global coordinate system;

6 is an algorithm illustrating a rasterization method based on anti-aliasing light tracking according to an embodiment of the present invention;

7 is a flowchart illustrating a method for rendering computer graphics for mobile according to an embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

210: rendering server 220: a number of rendering clients

Claims (8)

A rendering server that determines attributes of each vertex of the geometric model of the object displayed in three dimensions through vertex shading on a global coordinate system, and generates vertex data representing the attributes of each vertex; And A plurality of rendering clients performing rendering by using the generated vertex data and the pre-stored image and generating a new image; Mobile computer graphics rendering system comprising a. The method of claim 1, And said plurality of rendering clients uses a fixed point arithmetic processor. The method of claim 1, And the plurality of rendering clients are selectively used according to a viewpoint, time and screen area of a given scene, respectively. The method of claim 1, Wherein each attribute of the vertex comprises at least one of color, brightness, texture, and transparency. Determining whether all vertices of a given pixel on the global coordinate system are inside an image projected on the pixel grid plane; A rasterization step of calculating color values of each vertex of the pixel piece when the pixel spans a plurality of images by interpolation using an attribute of each vertex of the pixel piece; And Obtaining the color of the pixel by dividing the sum of the color values of each vertex of the pixel piece by the area of the pixel piece * the number of vertices of the pixel piece; Mobile computer graphics rendering method comprising a. The method of claim 5, A rasterization step of calculating a color value of each vertex of the pixel by interpolation using an attribute of each vertex of the pixel when all vertices of the pixel exist in the projected image; And Obtaining a color of the pixel by calculating an average value of color values of each vertex of the pixel A computer graphics rendering method for mobile further comprising. The method of claim 5 or 6, wherein in the rasterization step, When the Perspective Correct Mode is satisfied according to the distance to the preset object, the depth width, and the visual distribution. Perspective Correct Interpolation is used. When the Perspective Correct Interpolation is satisfied. A method for rendering computer graphics for mobiles comprising performing rasterization using linear interpolation. The method of claim 7, wherein The perspective correction interpolation is expressed as

Interpolation is performed taking into account the intersection point t with the object calculated by d, and d is the distance from the origin E to the pixel grid plane,

Is a normal vector, S (kx, ky) is a screen sample point expressed in a global coordinate system.

Images