KR100893855B1 - Method for combination both of two-dimensional background and three-dimensional foreground and engine for three-dimensional application - Google Patents

Abstract

An interactive application engine providing visualization of an automatic three-dimensional full-parallax three-dimensional image of application objects in combination with a known two-dimensional background. The engine is based on the rendering technique of 3D images using a limited set of temporary rendered textures corresponding to the key points of object localization as source data for 3D visualization of application objects at any position in the application space. The 3D visualization is based on integral photography technology.

3D visualization, rendering

Description

Method for combination both of two-dimensional background and three-dimensional foreground and engine for three-dimensional application}

1 is a functional diagram of an application engine of interaction using a temporary rendered set of projected images for 3D object visualization.

FIG. 2 is a functional diagram of the rendering module of FIG. 1 designed for use of a temporary rendered projected set for 3D object visualization.

The present invention generally provides a personal computer (PC) -based interactive application engine that provides visualization of three-dimensional (3D) images of application objects, and a three-dimensional foreground and two-dimensional background combination with the application engine. It is about a method. In the present invention, the visualization of the 3D image is based on an integral photography technique.

Integral photography is generally known as an optical technique for capturing and displaying continuous parallax 3D images.                         

The technique is based on recording / reproducing 3D images using a two-dimensional spherical microlens array (ie, a fly's eye lens).

During recording, a number of very small pictures appear on the focal plane of each micro lens. Each picture corresponds to a portion of the registered 3D object and completes a set of those pictures called an projectionogram.

During playback, the projected image is displayed on the projected image carrier screen, and each recovered micro image is integrated with another image so that a true-scale 3D copy of the original objects appears. As any integrated device, the same micro lens array applied in the recording step is used.

Ando et al. Have patented an imaging system consisting of recording / transfer / playback of 3D images formed by a typical real-time integral photography capture application and micro lens array. Alternatively, Brown proposed a system similar to the above for cathode ray tubes. Doudnikou applied this method in the form of a real-time projection system.

Recently, most real-time applications that provide 3D recognition are based on real-time computer rendering of the data required for 3D image visualization.

Such kind of rendering may be provided by using the result of real-time capture of an application object as source information for computer processing, for example in a Visual Papilla Recognition (VISPAR) system.

As a source of information about application objects, those virtual models (eg, polygonal meshes) created using special purpose software for real-time 3D image rendering can be used. Such techniques apply to Horizontal Parallax Only (HPO) systems such as parallax panogramograms.

To realize a full parallax system such as integral photography, a great deal of computational level is required to provide on-line rendering of the data required for 3D image visualization.

To avoid such difficulties, preliminary rendered data should be used for the visualization of 3D images.

Because of that fact, only 3D images corresponding to the temporary rendered data can be played back, resulting in an animated movie application (Nakajima et al.).

An object of the present invention is an application that visualizes a 3D image using temporary rendered sets of micro-images, called projection images of the 3D object, as source data for visualization of application objects confined to any position dependent on the logic of the application. It is to provide a three-dimensional foreground and two-dimensional background combining method having an engine and the application engine.

In order to achieve the above object, the present invention recovers the automatic three-dimensional full-parallel three-dimensional image of the application objects using a PC-based interactive real-time application, the three-dimensional foreground and two with such an application It is characterized by combining the dimensional background.

Preferably, the restoration of the 3D image of the application objects is characterized by using an integral photography technique.

Preferably, a polygon mesh representation of an application object surface is used to provide a two-dimensional background of the application.

Preferably, a temporary rendered set of micro images is used as source data for visualization of an autostereoscopic three-dimensional image of the application object, wherein only a temporary rendered micro image corresponding to a key localization of the application object is used. It is characterized by.

Preferably, the polygon mesh representation of the surface of the application object is replaced with a simple rectangle to be visualized as the three-dimensional application object.

Preferably, the temporary rendered set of micro images is applied as a texture bitmap for a simple rectangle that replaces the polygon mesh of the application object.

Preferably, as a source for forming a single rectangle corresponding to any application object position, a set of rectangles corresponding to a key position of an application object closest to any current object position is generated.

Preferably, the set of rectangles is superimposed to convert a polygon mesh representation of the surface of the application object into a single rectangle corresponding to an arbitrary application object position.

Preferably several textures corresponding to the key position of the application object closest to any current object position, so as to form a micro image set corresponding to any non-temporary rendered position of the object to be visualized as a 3D application object. It is characterized by blending.

Preferably, the blended micro-image set is applied with a texture bitmap for a rectangle corresponding to the position of the intermediate application object, for visualization of the 3D image of the application object restricted to any position.

Preferably a 3D display based on integral imaging or any other technique that provides automatic stereoscopic 3D image recovery is used as a visualization device for interactive, real time applications based on temporary rendered sets of micro images. It is characterized by.

A PC-based interactive real-time application engine according to the present invention is characterized by providing for the recovery of an autostereoscopic full-parallax 3D image of application objects.

Preferably the application engine replaces the polygon mesh representation of the surface of the application object to a simple rectangle so that it is visualized as a three-dimensional application object and temporarily renders a micro image of the application object as a texture bitmap for the simple rectangle. It is characterized by comprising a rendering module to apply the set.

Preferably, the rendering module generates a set of rectangles corresponding to a key position of an application object closest to any current object position as one source for forming a single rectangle corresponding to any application object position. It is done.

Preferably, the rendering module further comprises an overlapping module for generating a single rectangle corresponding to an arbitrary application object localization.

Preferably, the rendering module further comprises a blending module for forming a micro image set corresponding to an arbitrary application object position.

Preferably, the rendering module applies the blended micro image set as a texture bitmap for a rectangle corresponding to a position of an intermediate application object to visualize a 3D image of an application object restricted to an arbitrary position.

Other objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings.

Hereinafter, the configuration and operation of the embodiments of the present invention with reference to the accompanying drawings, the configuration and operation of the invention shown in the drawings and described by it will be described as at least one embodiment, which By the technical spirit of the present invention described above and its core configuration and operation is not limited.

1 is a functional diagram of a 3D game engine according to the present invention. As a source for the game, 3D environment objects created using 3D Studio MAX (or other compatible software) are used. These objects consist of a set of source objects.

All objects are then converted to the internal game format using the Export plug-in. The Export plug-in, in addition to others, forms an internal object representation in the engine kernel that is typically used for visualization of objects.

 The internal object representation is a polygon mesh corresponding to a surface of a three-dimensional application object having a corresponding texture.

The internal object representation of the application object is linked with internal object controllers, which have the functionality of game logic, and external object controllers, which allow the user to intervene interactively during game processing. The internal object representation is under the influence of the internal and external object controllers.

In addition, in order to obtain a set of projection bitmaps that form an projection database, the objects represented as 3D images are temporarily processed using an projection generation toolkit.

Each projection bitmap in the set of projection bitmaps of the object corresponds in part to the key position of the object in application space. The projected bitmap will be used as a texture to provide 3D visualization of application objects.

The set of internal game objects is used to form the "Live" application scene. The "Live" application scene uses the game logic provided by the internal object controller and is under the influence of the engine kernel, taking into account the user's intervention provided by the external (user) object controller.

The formed "Live" application scene is input information for the rendering module.

To visualize objects represented as a 2D background, a rendering process is typically used that maps a polygon mesh corresponding to the surface of the rendered object to the texture of the object.

In the case of an object to be represented as a 3D image, the polygon mesh is converted into a simple rectangle.

The appropriate projected image bitmap output from the projected image database is used as the texture for that rectangle to provide a representation of the 3D image on the 3D display.

To avoid any of the distortion dimensions mentioned above, the rectangle must correspond exactly to the projected bitmap size.

To maintain proper game rate values, hardware accelerators are used at the final stage of rendering.                     

FIG. 2 is a functional diagram of the rendering module of FIG. 1.

2, for each object in the "Live" application scene, it is determined whether the object is to be visualized as a 3D object.

If the object is not visualized using a 3D representation, 2D background rendering is typically applied to map the polygon mesh of the object to the texture of the object to visualize the object as a 2D background.

On the other hand, if the object is visualized using a 3D representation, 3D rendering must be performed.

At this time, to represent the application object as an autostereoscopic 3D image, the process becomes much more complicated.

First, the object state is input from the "Live application scene" created in the engine kernel. Then, it is determined whether the object position is one key position.

If it is determined that the condition is correct, that is, if the object position is one key position, the polygon mesh of the object is ignored, and a single rectangle is provided for the key position aspectogram texture provided by the projection database. Applied to and rendered.

If the condition is not met, a set of rectangles corresponding to the nearest key localization of the object is created. And, by a superimposing module, a single rectangle of the set of rectangles is created at the desired position.

At the same time, the projected image bitmap textures corresponding to the selected key position are output from the projected image database and blended by the blending module.

At this time, by using the blending module, it is possible to smooth the running of 3D objects reproduced between different key positions.

The texture blended by the blending module is the texture to be mapped to a single rectangle created by the overlapping module. The results are displayed on the 3D display.

As such, the present invention limits the amount of temporary rendered data to only the projected images corresponding to the key position of the application object, and the rendering module of the present invention is used for rendering the objects before visualization.

At this time, both a combination of a two-dimensional (2D) background object (providing an application environment) and a foreground object (visualized as a three-dimensional image) is used for that kind of application. And, during the rendering process, objects used as 2D backgrounds are rendered in a conventional manner (i.e., mapping a polygon mesh representing the surface of the application object to a texture). At the same time, the polygon mesh representation of the object to be visualized as a 3D object is converted into a simple rectangle, which is then textured using the temporary rendered projection image and the visualization at the appropriate location in the 3D space.

The visualization is based on integral photography technology and the visualization also requires a 3D display based on integral photography technology.

The invention is applicable to games, CAD / CAE visualization systems, simulation systems, training systems, medical and other imaging systems with specific purposes.

As described above, according to the application engine of the present invention, the amount of temporary rendered data, that is, the amount of source data can be reduced by using only a limited set of temporary rendered projection images corresponding to the key position of the object for visualizing the 3D image. have.

In addition, by using a blending module, it is possible to smooth the running of 3D objects reproduced between different key positions.

And by using additional acceleration provided by the PC hardware, the application rate can be improved.

Those skilled in the art will appreciate that various changes and modifications can be made without departing from the spirit of the present invention.

Therefore, the technical scope of the present invention should not be limited to the contents described in the embodiments, but should be defined by the claims.

Claims (23)

In a method of combining three-dimensional foreground and two-dimensional background with a PC-based interactive real-time application that provides for the recovery of autostereoscopic full-parallax three-dimensional images of application objects, Receiving a live application scene; Determining whether to visualize an object in the live application scene as a 3D object; As a result of the determination, if the object is not visualized as a 3D object, the polygon mesh of the object is mapped to the texture of the object and rendered, and if the object is visualized as a 3D object, the source data of the pre-rendered micro image set is rendered. And converting the polygon mesh of the object into a single rectangle, mapping the single rectangle to the texture of the object, and rendering the object. The method of claim 1, wherein the restoration of the 3D image of the application objects is performed. A method of combining, which uses integral photography techniques with interactive real-time applications. delete delete The method of claim 1, wherein the mapping of the simple rectangle to the texture of the object comprises: If the object is visualized as a 3D object, determining whether the position of the object is one key position; As a result of the determination, if the position of the object is one key position, the polygon mesh of the object is converted into a single rectangle using source data of the pre-rendered micro image set, and the single rectangle is converted into the texture of the object. Mapping method comprising the step of mapping and rendering. delete delete The method of claim 5, wherein As a result of the determination, if the position of the object is not one key position, as a source for forming a single rectangle corresponding to the position of the current object, it corresponds to the key position of the application object closest to the position of the current object Generating a set of rectangles; Overlapping the set of generated rectangles to generate a single rectangle corresponding to the current object position; And converting the polygon mesh of the object into the generated single rectangle. delete The method of claim 5, wherein Replacing the polygon mesh of the object with a single rectangle, Blending the texture corresponding to the key position of the application object closest to any current object position to form a micro-image set corresponding to any non-temporary rendered position of the object to be visualized as a 3D application object. Joining method. The method of claim 10, And apply the blended micro-image set as a texture bitmap for a rectangle corresponding to the position of the intermediate application object to visualize a 3D image of the application object constrained to the arbitrary position. delete A computer-readable medium for recording a PC-based interactive real-time application that provides for the recovery of autostereoscopic full-parallax 3D images of application objects, Receiving a live application scene, Determining whether to visualize an object in the live application scene as a 3D object; As a result of the determination, if the object is not visualized as a 3D object, the polygon mesh of the object is mapped to the texture of the object and rendered, and if the object is visualized as a 3D object, the source data of the pre-rendered micro image set is rendered. And converting the polygon mesh of the object into a single rectangle, mapping the single rectangle to a texture of the object, and rendering the object. The method of claim 13, Providing recovery of a three-dimensional image of application objects is a computer readable medium characterized by using integral photography techniques. delete The method of claim 13, The step of mapping the simple rectangle to the texture of the object to render If the object is visualized as a 3D object, determining whether the position of the object is one key position; As a result of the determination, if the position of the object is one key position, the polygon mesh of the object is converted into a single rectangle using source data of the pre-rendered micro image set, and the single rectangle is converted into the texture of the object. A computer-readable medium, comprising: mapping and rendering. delete The method of claim 16, As a result of the determination, if the position of the object is not one key position, as a source for forming a single rectangle corresponding to the position of the current object, it corresponds to the key position of the application object closest to the position of the current object Creating a set of rectangles, Overlapping the set of generated rectangles to generate a single rectangle corresponding to the current object position; And converting the polygon mesh of the object into the generated single rectangle. delete delete delete The method of claim 16, wherein replacing the polygon mesh of the object with a single rectangle includes: Blending a texture corresponding to a key position of an application object closest to the current arbitrary object position to form a micro image set corresponding to any non-temporary rendered position of the object to be visualized as a 3D application object. Computer-readable media. delete

Images