KR100900822B1 - System and method for real-time relighting system - Google Patents

Abstract

The present invention relates to a real-time re-illumination rendering method and system, and more particularly, to a software system that supports the lighting operation of each scene or character in the process of 3D computer animation production, and updates the rendered image in real time according to the change of the light source in the scene The present invention relates to a real-time re-illumination rendering method and system that allows an operator to easily check the change of visual effects. To this end, the present invention includes the steps of calculating and storing an illumination independent component which is not affected by changes in the light source; Selecting a specific light source among the light sources and modifying the selected light source using animation software; Performing rendering for each light source; It provides a real-time re-illumination rendering method comprising the step of synthesizing the rendered image.

Relighting, caching, lighting, 3D animation.

Description

Real-time relighting rendering system and method {SYSTEM AND METHOD FOR REAL-TIME RELIGHTING SYSTEM}

The present invention relates to a real-time re-illumination rendering method and system, and more particularly, to a software system that supports the lighting operation of each scene or character in the process of 3D computer animation production, and updates the rendered image in real time according to the change of the light source in the scene The present invention relates to a real-time re-illumination rendering method and system that allows an operator to easily check the change of visual effects.

The present invention is derived from the research conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. .

The lighting work is to create a screen atmosphere to maximize content transfer by appropriately determining the arrangement, direction, intensity, and color of the light source in a given situation of all geometric elements and surface materials in the scene. In this work, lighting includes both the entire background and the lighting limited to each character in the scene.

The overall mood of the screen is greatly affected by some modifications to some of the properties of the light sources. Therefore, in order to correctly identify the visual change of the screen according to the light source change, an accurate feedback method is required. Existing animation software provides preview rendering function that can preview the work result, which is faster than the final rendering, but it is less than real time and the image quality is much lower. This problem is a waste of time and money for lighting work.

It is an object of the present invention to provide a tool for contributing to the productivity improvement of animation makers by solving the above problems of the present invention, and to provide a convenient interface that allows the user to modify the light source properties while viewing the intermediate result of the lighting operation in real time. have.

In order to achieve the above object, the present invention comprises the steps of pre-calculating and storing an illumination independent component that is not affected by changes in the light source; Selecting a specific light source among the light sources and modifying the selected light source using animation software; Performing rendering for each light source; It provides a real-time re-illumination rendering method comprising the step of synthesizing the rendered image.

In addition, to achieve the above object, the present invention provides a lighting independent component cached module for calculating the lighting independent component is not affected by changes in the light source; Animation software to modify a particular light source selected from the light sources; A rendering module for each light source to perform rendering for each light source; And an image synthesizing module for synthesizing the rendered image, wherein the rendering is performed independently with respect to the selected light source and the unselected light source.

The present invention is intended to implement re-illumination rendering in real time by updating the rendering result due to the change of the light source through caching of independent components, rendering by the light source using a hardware graphic accelerator, and reusing the rendering result for the unchanging light sources. The method for implementing the relighting system of the present invention can be easily integrated and developed into the existing animation production pipeline since the software renderer of the existing animation system can be used as it is.

Since the present invention enables the re-lighting result to be confirmed in real time, the production time and cost can be reduced by eliminating the inconvenience that the animation worker has to wait for checking the intermediate result unnecessarily. In addition, real-time feedback can help improve the quality of animation work by eliminating the psychological burden on the worker so that they can focus on more creative parts.

The terms used in the present invention are selected as general terms that are widely used at present, but in some cases, the term is arbitrarily selected by the applicant, in which case the meaning is described in the detailed description of the invention, The present invention should be understood as meanings of terms rather than names.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described. Although the present invention encompasses all media signals, the following description focuses on audio signals for convenience of description.

The overall configuration of the present invention is illustrated in FIG. The user modifies the light source through the animation system (application system unit 10). When the user starts the lighting operation, the caching module of the lighting independent component values is performed to calculate necessary component values for the current scene and to store them on the disk in the form of an image file. Next, when the user modifies the property, position, or direction of one of the light sources, the rendering module for each light source is executed to calculate the rendering result by the individual light sources (relighting processor 20). Images rendered for each light source are sent to the image synthesis module to be combined into one image, and finally, the image is output to the image output device (image output module) and output to the screen (result output unit 30).

The animation software (animation module) is software that allows the user to read scene data, model various elements of the animation, such as characters, backgrounds, and light sources, specify movement over time, and even render finally. . This software is commonly available commercially available animation software. In the present invention, the animation software provides an interface for the user to position and orient the light source, modify the properties of the light source's intensity, color, and the like, and to generate a cache of lighting independent component values.

The lighting independent component value caching module calculates a component which is not affected by the light source change in the formula of calculating the color of each pixel of the image and stores it as a texture object on the hard disk and the computer memory in the form of an image file. The equation for calculating each pixel color in 3D rendering is determined by the shading model. Commonly used shading models are Lambert, Phong, Blinn, and so on, depending on the materials of objects in the scene. With these three models, we will illustrate lighting independent components.

The Lambert model represents a matte surface with no gloss or highlights. It is the most basic shading model, and other models include Lambert's properties by default. The Lambert model is expressed as Equation 1 below.

은 픽셀의 색상이고,

는 객체 표면이 가지는 색상,

는 주변광의 색상,

는 물체 표면에서 산란되는 빛의 색상,

은 현재 픽셀에 투영되는 물체 표면 점에서의 수직 벡터,

은 물체 표면 점에서 광원으로 향하는 벡터, 그리고

는 광원으로부터 오는 빛의 세기이다. 여기서,

과

를 제외한 나머지 성분들은 광원 변화에 관계없이 일정한 값을 갖기 때문에 조명독립적 성분이 된다.here,

Is the color of the pixel,

Is the color of the object's surface,

Is the color of the ambient light,

Is the color of light scattered from the surface of the object,

Is the vertical vector at the object surface point projected on the current pixel,

Is the vector from the object surface point to the light source, and

Is the intensity of light coming from the light source. here,

and

The other components except for become constant values independently because they have a constant value regardless of light source change.

The Phong model is useful for expressing the intense gloss and highlights of glass or glass surfaces (ex: cars, phones, bathroom interiors, etc.). The Phong model is represented by Equation 2 below.

는 빛이 전반사될 때의 표면 색상,

는 현재 픽셀에 투영되는 물체 표면점에서 카메라로 향하는 벡터, specularity는 물체 표면에서 하이라이트가 퍼지는 정도 또는 난반사율, 그리고

은 광원으로부터 온 빛이 물체 표면점에서 반사되어 나가는 방향이 된다. 여기서

와

, specularity는 광원의 변화에 영향을 받지 않는 성분들이다. 반면에

은 광원으로부터 오는 빛의 방향이 바뀌면 따라서 바뀌기 때문에 조명독립적이라고 할 수 없다.Looking at the above equation it can be seen that the expression of the Lambert model is included. Therefore, the light-independent component of Lambert model is also applied to Phong model. In the additional paragraph

Is the color of the surface when light is totally reflected

Is the vector from the object surface point projected on the current pixel to the camera, specularity is the extent to which the highlight spreads or diffuse reflectivity from the object surface, and

Is the direction in which light from the light source is reflected at the object surface point. here

Wow

However, specularity is the component that is not affected by the change of light source. On the other hand

It is not lighting independent because it changes as the direction of light coming from the light source changes.

The Blinn model is a useful material for representing metal surfaces (such as copper and aluminum) and has a soft luster and highlights. The Blinn model is expressed by Equation 3 below.

는 Fresnel 반사율로 입사광의 방향과 표면 재질의 특성에 따라 결정되는 함수이다. 본 발명에서는 D(), G(),

의 값이 광원에 의존적이므로 캐쉬화하지 않고 렌더링 시에 계산하도록 하였는데, D()와

에서 내부적으로 이심률(eccentricity)와 롤오프(roll-off)값을 계산에 사용하고 있다. 따라서, Blinn 모델에서 캐쉬화할 광원독립적 성분들은

,

, 이심률, 롤오프 값들을 선택한다. 구현에 따라 D(), G(),

의 계산식에서 광원 독립적인 부분의 계산값을 추가로 캐쉬화할 수도 있지만, 이 경우 캐쉬 저장 공간이 더 많이 필요하게 된다.As with the Phong model, the terms for ambient and scattered light are taken directly from the Lambert model, while the overall picture is different from the Phong model. The function D () is a direction distribution function of the micro facet constituting the object surface. The function G () is a geometric attenuation factor that represents the extent to which the microsurfaces cover each other. function

Is a Fresnel reflectance that is determined by the direction of incident light and the characteristics of the surface material. In the present invention, D (), G (),

Since the value of is dependent on the light source, we calculated it at rendering time without caching.

Internally, eccentricity and roll-off are used in the calculations. Thus, the light-independent components to cache in the Blinn model

,

Select the eccentricity, rolloff values. Depending on the implementation, D (), G (),

Although the calculation of the light-independent part of the equation can be further cached, this requires more cache storage space.

), 쉐이더 모델 종류, 주변광에 대한 표면 색상(

), 산란광에 대한 표면 색상(

), 전반사광에 대한 표면 색상(

), 표면의 법선벡터(

), 픽셀의 깊이값, 그리고 난반사율(specularity), 이심률, 롤오프 값들을 하나로 묶은 전반사속성값으로 여덟 가지가 된다. 도2에서는 예제 장면에 대해 계산한 캐쉬값들을 이미지 형태로 예시하고 있다.This way, once you've collected all the cache types you need, the object surface color (

), Shader model type, surface color for ambient light (

), The surface color for scattered light (

), The surface color for total reflection

), The normal vector of the surface (

), Total depth attribute of the pixel, and specularity , eccentricity, and rolloff values. 2 illustrates cache values calculated for the example scene in the form of an image.

The process of calculating and caching lighting independent component values in advance may use a hardware renderer or a software renderer. Using a hardware renderer yields very fast calculations, but because sophisticated sampling techniques are not available, the calculations are not clean and visual aliasing defects can appear on surface textures. Using a software renderer takes relatively long computations, but can reduce aliasing defects even more. In the present invention, the software renderer is selected because the accuracy of the cache value is more important than the time required to generate the cache.

인 결과를 얻게 된다. 만일

로 설정하면, 렌더링 결과는 주변광에 의한 표면 색상(

)만이 얻어지게 된다. 따라서, 우리가 얻고자 하는 성분값을

로 연결하고 주변광 렌더링을 수행하면 렌더링된 영상의 각 픽셀에는 투영된 3D 표면점의 성분값이 저장되게 된다. 주변광 렌더링을 이용하는 이유는 기존 소프트웨어 렌더러를 수정없이 쓸 수 있도록 하기 위함이고, 결과적으로 본 발명은 어떤 소프트웨어 렌더러든지 사용할 수 있게 된다.In the present invention, the calculation of the lighting independent component value using the software renderer used ambient rendering. This is done by turning off all light sources in the scene and setting only ambient light to brightness 1 (reference value). This way, as you can see from the shading model

Results in if

When set to, the rendering result is the color of the surface due to

) Is obtained. Therefore, we want to get the component value

When the ambient light is rendered and each pixel of the rendered image is stored, the component values of the projected 3D surface points are stored. The reason for using ambient light rendering is to allow the existing software renderer to be used without modification, and as a result, the present invention can use any software renderer.

속성을 임시 저장 공간에 백업한다. 4)캐쉬화할 각 조명독립적 성분들에 대해 차례로 각 표면 재질의

속성으로 연결한다. 5)장면을 렌더링하여 결과를 파일에 저장한다. 6)원래의

값을 회복시킨다. 7)원래 장면 내의 광원들을 켠다.Using this algorithm simply as pseudocode: 1) Turn off all light sources in the scene. 2) Activate the ambient light with brightness 1.0. 3) all surface quality

Back up attributes to temporary storage. 4) each surface material in turn for each lighting independent component to be cached.

Link to an attribute. 5) Render the scene and save the result to a file. 6) Original

Restore the value. 7) Turn on the lights in the original scene.

Next, the rendering module for each light source calculates the degree to which each light source contributes to the image, which is actually like rendering each light source on. The present invention implements a rendering module for each light source by using cache images calculated and stored in the lighting independent component value caching module and a fragment shader program of a hardware graphics accelerator. A fragment shader is a hardware unit that performs calculations that determine the color of each pixel to be drawn on the screen, and a fragment shader program is programmed for the computation process.

The light source-specific rendering module of the present invention is implemented as follows using the OpenGL API. First, the images that store the lighting-independent component values calculated above are read into each texture object. Initialize the OpenGL accumulation buffer. Next, we initialize OpenGL's color buffer for each light source and bind OpenGL a fragment shader program for the current light type. Pass the properties of the current light source as arguments to the fragment shader. We then draw a rectangle that completely fills the entire image and accumulate the result in a cumulative buffer. Perform rendering for the next light source.

과 광원의 세기

을 구한다. 4)현재 픽셀의 쉐이딩 모델식을 계산한다. 5)계산 결과를 출력한다. 상기에서 광원 종류에 따라 달라지는 부분은 단계 3이다. 단방향(directional) 광원, 점(point)광원, 스팟(spot) 광원에 따라 광원 방향과 광원 세기를 구하는 것이 달라진다.The pseudo code of the fragment shader program for rendering for each light source of the present invention is as follows. 1) Read the lighting independent component values from the texture. 2) The 3D position is obtained from the depth value of the current pixel through the reverse camera projection. 3) light source direction

And the intensity of the light source

Obtain 4) Calculate the shading model formula of the current pixel. 5) Output the calculation result. The part which depends on the kind of light source in the above is step 3. The direction of the light source and the intensity of the light source vary depending on the directional light source, the point light source, and the spot light source.

3 is an exemplary view showing a process of synthesizing a result of rendering for each light source when re-illumination rendering according to an embodiment of the present invention. Referring to FIG. 3, the present invention completes the final image by sequentially adding images rendered for each light source to the accumulation buffer. However, in general, the number of light sources used in a scene of a commercial animation ranges from tens to hundreds, so it is necessary to repeat the rendering for each light source as many times. This amount of computation depends on the performance of the hardware graphics accelerator, but may not be complete in real time. As a method for solving such a performance degradation, the present invention uses a method of rendering only a light source selected by a user to be corrected.

When the user selects a light source for the first time, only light sources that are not selected are rendered for each light source, and the result is accumulated in the accumulation buffer. The result thus obtained is transferred to a separate auxiliary buffer and kept there. After this, whenever the user modifies the light source, the preliminary buffer image is copied to the cumulative buffer, and then only the modified light source is rendered and accumulated in the cumulative buffer. The result thus obtained is the same as the result obtained by rendering for all light sources. On the other hand, the rendering time is averaged only as long as the rendering time for one light source regardless of the total number of light sources except for the first light source selection time.

4 is a flowchart illustrating a real-time relighting rendering method according to an embodiment of the present invention. Referring to FIG. 4, the lighting independent component value caching module first stores components that are not affected by the change in the light source (S401). Next, the user selects a light source to be modified, and modifies the selected light source using animation software (S402). Next, the rendering module for each light source performs rendering for each light source. If the light source is not selected by the user (S403), that is, the rendering results are accumulated and reused for light sources that do not change (S404). The accumulated result is stored in the preliminary buffer (S405), and the image of the preliminary buffer is copied into the preliminary buffer every time the light source is modified. If it is the light source selected by the user (S403), the rendering is performed for each light source (S406) and then accumulated in the accumulation buffer (S407).

Although the present invention has been described in detail with reference to some embodiments, the above embodiments are presented for the purpose of understanding the present invention, and the scope of the present invention is not limited to the above embodiments. Those skilled in the art will understand that various modifications are possible without departing from the scope of the technical idea of the present invention, and the scope of the present invention should be interpreted by the appended claims.

1 is an exemplary configuration of a software system according to an embodiment of the present invention.

2 is an exemplary view showing an example in which a light-independent component value for a given 3D scene is generated as a cache image when re-rendering a rendering according to an embodiment of the present invention.

3 is an exemplary view showing a process of synthesizing a result of rendering for each light source when re-illumination rendering according to an embodiment of the present invention.

4 is a flowchart illustrating a real-time relighting rendering method according to an embodiment of the present invention.

* Description of the symbols for the main parts of the drawings *

10: Application system part

20: relight processing unit

30: result output

Claims (8)

Calculating and storing an illumination independent component of the light source in advance; Correcting the specific light source in response to a correction value input through animation software in response to a specific light source being corrected; Rendering the modified specific light source; And And synthesizing the image of the rendered specific light source and the reuse image of the light source except the specific light source. delete The method of claim 1, wherein the synthesizing the image of the specific light source and the reuse image comprises: Maintaining a light source excluding the specific light source in a preliminary buffer; Copying an image of the preliminary buffer into a cumulative buffer; And And accumulating and combining the image rendered for the specific light source in the accumulation buffer. The method of claim 1, And the illumination independent component is extracted from an equation for calculating the color of each pixel. The method of claim 1, And the illumination independent component is calculated by turning off all light sources and rendering only ambient light as a reference value. An illumination independent component cached module for calculating an illumination independent component of the light source; An animation module for modifying the specific light source in response to a correction value input through animation software in response to a specific light source being requested to be corrected; A light source-specific rendering module for rendering the modified specific light source; And And an image synthesizing module for synthesizing the image of the rendered specific light source and the reuse image of the light source except the specific light source. 7. The system of claim 6, wherein the real-time re-illumination rendering system is And a video output module for outputting the synthesized video. The method of claim 6, And the illumination independent component caching module calculates the illumination independent component by turning off all light sources and rendering only ambient light as a reference value.

Images