KR20160077559A - Real time interal imaging method - Google Patents

Abstract

The present invention relates to a method for generating a real-time integral image by using a real-time integral generation system for generating an integral image in real time at high speed in a computer graphics environment. The method for generating a real-time integral image by using a real-time integral generation system for generating an integral image in real time at high speed in a computer graphics environment uses shader programming together with a GPGPU, thereby displaying an integral image on a monitor at high speed without transferring data between RAM and a GPU. Accordingly, the method for generating a real-time integral image according to the present invention allows the GPU to process overall calculation required to generate an integral image while maintaining an existing rendering pipeline structure, thereby having a significant effect of generating an integral image without a delay time required for physical data transmission between the RAM and the GPU.

Description

[0001] The present invention relates to a real-time integrated imaging method,

The present invention relates to a real-time integrated image generation method using a real-time integrated generation system for generating integrated images at high speed in a computer graphics environment.

Computer Graphics (CG) refers to a technique for creating a new image of a room using a computer.

In recent years, 3D computer graphics is often referred to as computer graphics.

The present invention relates to a method for generating an integrated image at high speed in real time in a computer graphics environment.

In the conventional computer graphics environment, the method of generating the integrated image at high speed is a method of reducing the calculation amount using the GPGPU.

As a conventional technique for generating an integrated image at high speed in such a computer graphics environment, a virtual lens composed of N x N lenses is provided in Patent Document 1275588, and a virtual lens information input unit, a user requirement input unit, A data input unit configured with a volume video data input unit; A virtual lens information calculation unit that receives information input to the virtual lens information input unit and calculates information of the virtual lens; and a virtual lens information calculation unit that receives information input to the user requirement input unit and the 3D volume image data input unit, A calculation unit configured to calculate a virtual camera setting information; When the values obtained from the virtual lens information calculation unit and the virtual camera setting information calculation unit are sent to an OpenCL (OpenCL) kernel program, the pixel values of the integrated image to be rendered in the multiclass threads in the OpenCL kernel program are subjected to parallel processing An integrated image generation unit for generating an integrated image in real time; And a display unit for displaying the generated integrated image. A high-speed integrated image generation system for three-dimensional volume data is disclosed.

Another conventional technique includes a setting unit for generating polygon descriptions as a graphics processor unit (GPU) in claim 10 of Patent Publication No. 0902974; A rasterizing unit coupled to the setting unit for rasterizing the polygon techniques; A rasterization unit, provided in the rasterization unit, for rasterizing graphics primitives to a first level of precision to produce a plurality of tiles of pixels; And a precision raster unit for rasterizing the graphics primitive to a second level precision to produce covered pixels for rendering operations at a subsequent stage of the graphics processor, The system of claim 18, further comprising: a system memory; A central processing unit coupled to the system memory; A graphics processor unit communicably connected to the central processing unit; A setting unit, provided in the graphics processor unit, for generating polygon techniques; A rasterizer unit coupled to the setting unit within the graphics processor unit for rasterizing the polygon techniques; And a generic raster section provided in the rasterizer unit, the generic raster section receiving a graphics primitive for rasterization at a raster end of the graphics processor, and rendering the graphics primitive to generate a plurality of tiles of pixels Rasterizing the tiles to a second level of precision to rasterize the tiles to a level of precision and creating the covered pixels, and wherein the covered pixels are printed by a computer system that is output for rendering operations at a subsequent stage of the graphics processor, It is public.

The problem with existing methods is that when generating Multi-View Image, unnecessary delay is generated by repeating pipeline M × N times, and when transmitting / receiving data between RAM and GPU for Sub-Image conversion, There is a limit to the transmission speed.

And it can not make full use of the computer graphics pipeline, so it can not use existing contents right away.

In order to solve the above-described problems, the present invention has been made in order to improve the speed of the M × N rendering pipeline by using Shader Programming in the existing Multi-View Image generation and to read / write data between the RAM and the GPU In order to overcome the limit of transmission speed in physical (PCle Slot), it is necessary to process all the operations required for the integrated image generation in the GPU while maintaining the existing rendering pipeline structure so that the delay time for the physical data transfer between the RAM and the GPU So that an integrated image can be generated.

A real-time integrated image generation method of the present invention uses a real-time integrated generation system that generates high-speed integrated images in real time in a computer graphics environment. By using a shader programming and a GPGPU, The image is displayed on the monitor.

The method of generating a real-time integrated image according to the present invention can process all the operations required for generating an integrated image in the GPU while maintaining the existing rendering pipeline structure, thereby generating an integrated image without delay time required for physical data transmission between the RAM and the GPU And the like.

1 is a conceptual diagram of a conventional rendering system;
2 is a conceptual view of a single GPU rendering system;
3 is a flowchart in a single GPU;
4 is a conceptual diagram of a multiple GPU rendering system;
5 is a flowchart in multiple GPUs;
Figure 6 is a rendering pipeline flowchart operating as a computer program.
7 is a 2D processing flow chart.
8 is a flowchart at a back-point.
Figure 9 is a rendering result image.
10 is an outline view showing a Local Area Multi-View Image.

A real-time integrated image generation method of the present invention uses a real-time integrated generation system that generates high-speed integrated images in real time in a computer graphics environment. By using a shader programming and a GPGPU, The image is displayed on the monitor.

When the 3D data is read from the RAM to the GPU, a Multi-View Image is generated using a shader programming language and a Multi-View algorithm in the Graphics Rendering Pipeline, and the generated Multi-View Image is converted into a sub- And generates an element image using a Sub-Image Transformation algorithm to display a 3D image through an integral imaging display system.

In addition, by using a memory distribution manager (Memory Distribute Manager) composed of a plurality of CPUs, the data processing speed is improved by dispersing and transmitting data through each GPU pipeline connected to correspond one-to-one with 3D data to a plurality of CPUs.

When the 3D data is read from the RAM to the memory distribution manager, the read information is appropriately distributed according to the number of GPUs, and data is transferred to each GPU. In each Graphics Rendering Pipeline, shading And the local area multi-view image is created using the Multi-View algorithm and the local area multi-view image is combined into one multi-view image. Then, the sub-image conversion is performed using the GPGPU (Sub-Image Transformation) algorithm, and displays the 3D image through an integrated image display system (Integral Imaging Display System).

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a real-time integrated image generation system according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of a conventional rendering system.

As shown in FIG. 1, in the conventional rendering system, the 3D data information is repeated M × N times to generate a multi-view image, and the generated multi-view image is read from the frame buffer to the RAM again.

The read Multi-View Image makes an elemental image using sub-image conversion algorithm.

The larger the resolution of the image in the glReadPixel () and glDrawPixel () parts of the RAM and the GPU's framebuffer, which is repeated M × N times in the existing method, is physically time consuming.

Most of the methods for generating integrated images at high speed in the conventional computer graphics environment are using GPGPU to reduce the calculation amount.

However, the problem with existing methods is that when transferring data between RAM and GPU (Read / Write), the transfer speed is limited by physical (using PCIe3 slot).

And because the computer graphics pipeline can not be utilized to the full extent, it can not use existing content directly.

Referring to FIG. 1, the Fixed Rendering Pipeline constituted after the CPU RAM is a graphics card processing area, the other pair of RAM, and the sub-image transformation is a CPU area.

The frame buffer is a memory space (Viedeo Memory) for storing information about the entire screen to be drawn on the screen, that is, a storage device for storing an image in a computer. The signal of the display data from the central processing unit (CPU) And then displayed on the display screen.

Often, the resolution contains information about the screen to be drawn at that resolution, such as 1024x768 or 800x480.

GPGPU is a general purpose GPU, abbreviation of General Purpose computing on Graphics Processing Units. It is a technology that performs calculations of applications that the CPU has traditionally processed using a GPU that only handles calculations for computer graphics.

This is achieved by connecting programmable layer and precision operations to the graphics pipeline, which allows software developers to use stream processing for non-graphics data.

2 is a conceptual view of a single GPU rendering system.

2, in the present invention, in order to overcome the limitation of transmission speed by using a physical (PCIe3 Slot) when transmitting / receiving data between an existing RAM and a GPU, a conventional rendering pipeline structure is maintained GPU handles all the operations required to generate integrated images.

Thus, the integrated image is generated without the delay time for the physical data transfer between the RAM and the GPU.

Therefore, by using Shader Programming and GPGPU as shown in FIG. 2, integrated images are displayed on the monitor at high speed without transferring data between GPUs in the RAM.

In particular, the geometry shader executes the program-defined shader code that receives the vertex as input and can generate the vertex as output.

Unlike the vertex shader, which works only for one vertex, the geometry shader receives all the vertices that constitute one complete primitive as input.

The geometry shader can also fetch the vertices of adjacent primitives as input.

Therefore, Multy-View is completed at one time without repeating until one view is generated and M × N views are generated.

Shader Programming refers to creating functions for vertex shaders and pixel shaders one by one.

A shader refers to a function that computes the position and color of a pixel to be displayed on the screen, which is a compound term of Shade er, which means to give a color shade, hue and contrast effect.

For reference, a vertex shader transforms the position of 3D object vertices into screen coordinates, which is executed as many times as the number of vertices of a 3D object, and the output value of a vertex shader is a position value of a vertex.

The pixel shader calculates the final color to be displayed on the screen. The pixel shader calculates the number of pixels to be found by the rasterizer (collects three vertex shader output values to form a triangle to find the number of pixels) To calculate what color to paint.

Figure 3 is a flow chart in a single GPU.

As shown in FIG. 3, when the real-time integrated image system operates, data (Resource (3D Data)) is read from the RAM to the GPU.

Graphics Rendering Create multi-view images using Shading Programming language and Multi-View algorithm in Pipeline.

The generated Multi-View Image is generated by the Sub-Image Transformation algorithm using GPGPU.

And displays the generated element image through the integrated image display system (Integral Imaging Display System).

4 is a conceptual diagram of a multi-GPU rendering system.

There is a limitation in creating a multi-view image in a conventional single GPU rendering system.

This is because there is a processing amount that can be processed by one GPU.

If you have a large amount of 3D data and a large number of multi-views when creating Multi-View Image, it is difficult to create real-time because of overflow of computation amount that can be processed in one GPU.

In this case, real-time processing can be performed using multiple GPUs for real-time processing.

As shown in FIG. 4, 3D data is distributed to each GPU by using a memory distribution manager.

Each Single GPU Pipeline creates a Multi-View Image with 3D data and parameter values and combines the created Local Area Multi-View Image into one.

Local Area Multi-View Image refers to multi-view area of partial area in Multi-View Image.

The combined Multi-View Image is transformed into an Elemental Image using Sub-Image Transformation algorithm.

As shown in FIG. 4, the memory distribution manager is composed of a plurality of CPUs, and each CPU is connected to a single GPU pipeline.

That is, the data processing speed of distributing and transmitting data to each GPU connected to correspond to one-to-one correspondence of 3D data in a parallel structure is increased by using a memory distribution manager composed of a plurality of CPUs.

Each GPU pipeline is selectively enabled to distribute the data.

5 is a flowchart in multiple GPUs.

As shown in FIG. 5, when a real-time integrated image system is operated, Resource (3D Data) is read from the RAM to the GPU.

We then distribute the information to each GPU by distributing the information appropriately according to the number of GPUs.

In each Graphics Rendering Pipeline, a local area multi-view image is created using the shading programming language and the multi-view algorithm.

Combine the generated Local Area Multi-View Image with one Multi-View Image, and generate the element image using Sub-Image Transformation algorithm using GPGPU.

And displays the generated element image through the integrated image display system (Integral Imaging Display System).

Figure 6 is a rendering pipeline flowchart that operates with a computer program.

Figure 6 is an OpenGL pipeline.

OpenGLdms A two-dimensional and three-dimensional graphics standard API standard created by Silicon Graphics, Inc. supports cross-application programming between platforms.

This API can generate complex 3D scenes from simple geometric shapes using about 250 function calls.

6, the multi-view algorithm can be applied to obtain a multi-view image.

Figure 7 is a 2D processing flow chart.

As shown in Fig. 7, the 2D processing flow chart

Copy 2D information in Frame Buffer to Pixel Buffer.

Calculate the size and number of pixels of the single view image in the copied pixel buffer.

Parallel processing is performed using GPGPU with the calculated information.

And transforms it into an elemental image using a sub-image transformation algorithm.

Copy the transformed element image to Frame Buffer.

Fig. 8 is a flowchart at a more point.

Programmable Rendering The flowchart of the Pipleline corresponds to ⓒ.

Calculate the number of M × N of Multi-View.

3-D movement matrix setting, 3-D movement movement setting, and Multi-View with the set value.

Based on the input parameter information (basic information required for Multi-View generation), new 3D information is generated at different points of view through each 3D matrix calculation.

The geometry shader moves, rotates, and creates a new vertex, and computes the texture mapping and clipping process to match it.

When one view is created, it is repeated in the vertex shader until M × N views are generated.

9 is a laddering result image.

As shown in FIG. 9, (a) is a single view image, which is a result of rendering information such as a 3D vertex, an edge, a face, and a texture image as a single view. (B) Is a multi-view image generated by the user.

(C) is a final result image obtained by converting a Multi-View Image as an Elemental Image into an Elemental Image using a Sub-Image Transformation algorithm.

Accordingly, as described above, the real-time integrated image generation method of the present invention can process all the operations required for the integrated image generation in the GPU while maintaining the existing rendering pipeline structure, It is possible to generate an image.

Claims (4)

Using Shader Programming and GPGPU, the integrated image can be displayed on the monitor at high speed without transferring data between RAM and GPU by using real-time integrated generation system that generates high-speed integrated image in real time in computer graphics environment. Time integrated image generation method.
The method according to claim 1,
When the real-time integrated generation system is composed of a single GPU pipeline, when the 3D data is read from the RAM into the GPU, a multi-view image is generated using the shader programming language and the multi-view algorithm in the Graphics Rendering Pipeline The 3D image is displayed through the integrated image display system by generating the element image by using the Sub-Image Transformation algorithm using the generated Multi-View Image using the GPGPU. Integrated image generation method.
The method according to claim 1,
When the real-time integrated generation system is constituted by a plurality of GPU pipelines, it is possible to use a memory distribution device (Memory Distribute Manager) composed of a plurality of CPUs to connect the 3D data to each of the plurality of CPUs through respective GPU pipelines Wherein the data processing speed is increased by dispersing and transmitting data.
The method of claim 3,
When the 3D data is read from the RAM to the memory distribution manager, the read information is distributed according to the number of GPUs, and data is transferred to each GPU. In each Graphics Rendering Pipeline, a shading programming language View image using the Multi-View algorithm, combine the generated Local Area Multi-View Image into a Multi-View Image, and use the GPGPU to convert the sub- And a 3D image is displayed through an integrated image display system (an integrated image display system).

Images