CN103957400A - Naked eye 3D display system based on Unity3D game engine - Google Patents

Abstract

The invention provides a naked eye 3D display system based on a Unity3D game engine. A Shader operating on a GPU is written according to a calculated viewpoint sub-pixel mapping matrix, the Shader comprises a peak Shader and a fragment Shader. Multiple stereo cameras are established in Unity3D software and are arranged according to certain structural positions. Finally, sampling and merging are performed on rendering maps output by all the stereo cameras and the Shader, obtained composite images are output into a naked eye 3D display screen, and 3D display is achieved.

Description

A kind of bore hole 3D display system based on Unity3D game engine

Technical field

The present invention relates to bore hole 3D and show field, relate in particular to a kind of 3D display system based on Unity3D game engine.

Background technology

It is a kind of Display Technique that does not need to wear auxiliary equipment (as red indigo plant, polarization, shutter glasses or the liquid crystal helmet etc.) that bore hole 3D shows, its background that has a wide range of applications in fields such as Aero-Space, military affairs, medical science, advertisement design and entertainment interactives.

At present, the bore hole 3D Display Technique of main flow mainly realizes based on slit grating and column mirror grating, slit grating is made up of printing opacity and the gratings strips being in the light, by the effect of blocking to light, the space of realizing different points of view image separates, and column mirror grating is to utilize cylindrical lens to interception of rays, and light is deflected in space, thereby realize the effect similar to slit grating, these two kinds of modes all belong to the bore hole 3D Display Technique of spatial reuse.

Conventionally, in order to obtain better stereoeffect, often adopt several anaglyphs to synthesize, as long as beholder sees that in different viewing location two width images wherein just can produce stereoscopic vision in cerebral cortex, and along with moving horizontally of beholder, can see the different angles side of object, and then produce motion parallax.

And Unity3D is a 3D game engine of Unity Technologies company exploitation, this engine is because of its powerful cross-platform exploitation characteristic, gorgeous 3D rendering effect and freely abundant human-computer interaction function is well-known outstanding.Its cross-platform exploitation characteristic can be saved the development time, accelerate development of games progress, because differing greatly between platform, such as the difference of screen size, mode of operation, hardware condition etc. all can cause huge trouble to developer, and the cross-platform characteristic of Unity3D be able to be saved the transplanting development work of developer between different platforms.Unity3D is a software that is used for specially making scene of game, architectural design, animated show, and it is only embedded the most basic three-dimensional body, spheroid, plane, cylinder etc. aspect model, in practical application, need to import from the external world model of making.Unity3D is for the Art Design software development of main flow model out, as 3ds Max, Maya etc., nearly all can be compatible, and Unity3D is its freely abundant human-computer interaction function compared to the advantage of 3ds Max, Maya etc.

Realize bore hole 3D Presentation Function if utilize Unity3D game engine, just can greatly expand the two application of Unity3D game engine software and bore hole 3D Display Technique, and the existing game based on Unity3D just can be able to be realized to bore hole 3D through a little amendment shows, thereby reduce to a great extent the video source cost of manufacture that bore hole 3D shows, and interactive function abundant Unity3D can be transplanted to bore hole 3D Display Technique, and enrich displaying contents and the material of bore hole 3D by means of the powerful model compatibility of Unity3D.

Summary of the invention

Human eye is in the time watching natural forms, and two is to watch from different perspectives object, thereby forms slightly discrepant picture on right and left eyes retina, and then merges and form stereoscopic vision through brain analysis.The procedure for displaying of bore hole 3D is exactly the stereotactic procedures of simulation human eye.

Patent of the present invention being divided into two large divisions below makes an explanation: the firstth, and the method for designing of bore hole 3D display unit and parameters feature, bore hole 3D display unit is mainly point light action based on grating, the image information of different points of view is realized and being separated in space, thereby ensure that the information that enters human eye right and left eyes adheres to different points of view separately, and within parallax amount between the viewpoint scope that can merge at human brain.Conventionally grating can be slit grating and column mirror grating.The secondth, Unity3D game engine is developed, by setting up many virtual three-dimensional cameras, and by means of the coloration program of writing the upper operation of GPU, make the image of Unity3D final rendering output meet the requirement of bore hole 3D display unit.

One, bore hole 3D display unit method for designing

The grating of bore hole 3D display unit can be slit grating, also can be column mirror grating, slit grating is made up of printing opacity and the gratings strips being in the light, by the effect of blocking to light, the space of realizing different points of view image separates, and column mirror grating is to utilize cylindrical lens to interception of rays, and light is deflected in space, thereby realize the effect similar to slit grating, these two kinds of modes all belong to the bore hole 3D Display Technique of spatial reuse.

What deserves to be explained is, be not that all 2D display screens all can show for bore hole 3D, bore hole 3D display unit requires high especially to the sub-pixel structure of 2D display screen, a complete pixel is made up of 1R1G1B tri-sub pixel lamps, sub-pixel arrangement mode and should meet following rule: arrange by red, green, blue or indigo plant, green, red circulation in the horizontal direction; Be the same of the same colour of red, green, blue in vertical direction, each sub-pixel spacing in the horizontal and vertical directions all should keep equal.

The method for designing of given first slit grating major parameter below:

1. first, grating constant, the computing formula in grating cycle is as follows:

$d=\frac{\mathrm{Ksp}}{p+s}---\left(1\right)$

Wherein, K is viewpoint number, for the bore hole 3D display that has K viewpoint, the anaglyph information of K viewpoint need be synthesized to a width composograph and is presented on 2D display screen.The resolution of 2D display screen is fixed, and so, the resolution of single width anaglyph can only account for the 1/K of 2D display resolution, and corresponding 3D resolution also drops to the 1/K of 2D display resolution.Therefore it should be noted that and adopt many viewpoint technique can improve the stereos copic viewing degree of freedom, viewpoint number is more, the stereos copic viewing degree of freedom is larger, but makes 3D resolution loss more serious simultaneously, therefore, need choose reasonable viewpoint number, average out between the two in the stereos copic viewing degree of freedom and three-dimensional resolution.Conventionally, the viewpoint number of bore hole 3D display is 8.

S in formula (1) is adjacent viewpoint spacing, is determined by viewpoint compound mode, and here taking 8 viewpoints as example, compound mode can be 1-4,2-5,3-6,4-7,5-8 or 1-2,2-3, other combinations such as 3-4....In order to allow right and left eyes receive respectively two anaglyph information of every a pair of viewpoint combination, must make every a pair of viewpoint be combined in and watch the spacing of plane and the interpupillary distance e of human eye to equate, adjacent viewpoint spacing is

$s=\frac{e}{\mathrm{\ΔK}}---\left(2\right)$

Wherein, the vision area number of Δ K for comprising between every a pair of viewpoint combination, eyes interpupillary distance is got e=65mm conventionally.

2. the spacing of grating and 2D display screen, screen apart from the computing formula of l is:

$l=\frac{\mathrm{Hp}}{p+s}---\left(3\right)$

Wherein, H is viewing distance, and namely beholder is apart from the distance of 2D display screen.

In addition, provide the method for designing of column mirror grating major parameter:

Post grid Research Thinking is design and optimize Lenticular screen according to 2D display screen parameter and stereos copic viewing condition.The basic parameter of Lenticular screen comprises that refractive index n, thickness t, focal distance f, radius of curvature r, grating constant and screen are apart from l.Wherein, refractive index n is to be determined by post grid material, and the refractive index of common PS (claiming again polystyrene) is 1.6.The design key of column mirror grating is the design of focusing f, radius of curvature r, thickness t and grating constant, itself and concrete 2D display screen is matched and obtain the Lenticular screen 3D display of function admirable.What deserves to be explained is, herein using screen apart from as known quantity, and screen distance and post grid thickness any all can be used as known quantity, this will see actual processing and assembling situation.

Though column mirror grating is different with the light-dividing principle of slit grating, but basically identical in the thinking that realizes bore hole 3D demonstration, the physical quantity title in column mirror grating computing formula is all consistent with slit grating, therefore no longer repeat specification here.

1. first, grating constant, the computing formula in grating cycle is as follows:

$d=\frac{\mathrm{Ksp}}{s+p}---\left(4\right)$

2. the computing formula of the thickness t of column mirror grating is as follows:

$t=\frac{\mathrm{Hnp}-\mathrm{nl}(s+p)}{s+\mathrm{np}}---\left(5\right)$

3. the computing formula of the radius of curvature of column mirror grating is as follows:

$r=\frac{p(n-1)[H+l(n-1)]}{s+\mathrm{np}}---\left(6\right)$

Design Lenticular screen designs its focal distance f, radius of curvature r, thickness t and grating constant exactly, itself and concrete 2D display screen are matched and obtains the Lenticular screen 3D display of function admirable.

Two, the specific implementation thinking of the bore hole 3D Display Technique based on Unity3D game engine:

1, set up scene, put many stereoscopic cameras

The most important condition that realizes stereoscopic vision is to provide slightly discrepant anaglyph to beholder's right and left eyes.In Unity3D software, can pass through many virtual three-dimensional cameras of rational deployment, same scene is carried out to synchronous shooting and obtain several anaglyphs.Stereoscopic camera need put photographed scene to obtain rational anaglyph by a fixed structure, and its placing structure mainly contains run-in index, convergence type, from axle run-in index and 4 kinds of structures of arc form.

In the target pinup picture of every camera, add and play up pinup picture Render Texture, playing up pinup picture is a kind of special texture type, can in the time of operation, produce and upgrade.Use and play up pinup picture, first will create a new texture of playing up, and specify a video camera to play up it, then just can be for texture be played up in material interpolation as conventional texture.

2, calculate viewpoint mapping matrix

First, determine that RGB sub-pixel given on 2D display screen should take from the RGB component of which viewpoint, formula (7) has provided the computing formula of many viewpoints sub-pixel mapping matrix:

$Q=\frac{(k+k_{\mathrm{off}}-3l\tan \mathrm{\α})\mathrm{mod}X}{X}{N}_{\mathrm{tot}}---\left(7\right)$

Wherein, (k, l) is the coordinate position of RGB sub-pixel, k _offthe horizontal displacement that represents 2D display screen left edge and raster unit marginal point, α is the slanted angle of grating axle with respect to LCD display vertical axis, X is the number that a grating cycle covers in the horizontal direction RGB sub-pixel, N _totrepresent total viewpoint number, namely participate in synthetic anaglyph quantity.

The gray value that can calculate the each sub-pixel on 2D display screen according to above formula should be taken from the gray value of the corresponding coordinate position of which width anaglyph.

3, Shader processes and plays up pinup picture

Viewpoint mapping matrix is programmed in Shader, and multiple that utilize Shader to play up camera are played up pinup picture and are carried out other sampling processing of sub-pixel-level, and multiple anaglyphs through sampling processing are added, and obtain final composograph.

Wherein, Shader is also referred to as tinter, is the program operating on GPU, is used for three-dimensional body to carry out that painted processing, shadow calculating and texture color present etc.Along with the lifting of GPU hardware performance, the programming mode of Shader has experienced from initial fixing rendering pipeline, to the development of rendering pipeline able to programme.Fixing rendering pipeline is a kind of pattern of supporting old video card, and the Shader that it writes out is based on summit illumination, and compared to the lighting system by pixel, the illumination computational speed of this mode is faster, but effect can be weaker.Fixed pipelines is how much, illumination pipeline of standard, and it is being controlled the world, is looking projective transformation and fixed-illumination control and texture blend, and substantially all video cards can normally move.Rendering pipeline able to programme carries out respectively programmed process to the summit computing in rendering pipeline and Pixel calcualting.Shader has summit Shader (Vertex Shader) and fragment Shader (Fragment Shader, be called again Pixel Shader) two kinds of fundamental types, summit Shader can process and convert the vertex position that is rendered into the mesh on screen, and the Output rusults of summit Shader will pass to next step of streamline.The grid of solid is after the rasterizing of hardware, fragment Shader on streamline can be performed, fragment Shader can carry out various tests to a fragment, and the successful fragment of final test can be written in the output frame of playing up, thereby becomes a visible pixels on indicator screen.

4, composograph output, realizes bore hole 3D and shows

Composograph is written to and is played up in array, output on indicator screen.Then, utilize point light action of grating, the light of different anaglyphs is separated in space, when observer is at suitable viewing location place, right and left eyes can receive respectively the information from different points of view, thereby produces stereoscopic vision.

Brief description of the drawings

Below in conjunction with accompanying drawing, the present invention will be further described.

Fig. 1 is the flow chart the present invention relates to;

Fig. 2 is the Unity3D camera structure specific embodiment the present invention relates to;

Fig. 3 is the viewpoint sub-pixel mapping matrix specific embodiment that the present invention calculates;

Fig. 4 is the synthetic anaglyph specific embodiment of the present invention;

Fig. 5 is the concrete embodiment of bore hole 3D display the present invention relates to.

Embodiment

Below in conjunction with accompanying drawing and concrete execution mode, the invention will be further described.

Fig. 1 is thinking flow chart involved in the present invention.First, in Unity3D game engine software, set up scene, game object can be self-built in Unity3D, also can from other modelling softwares, import.Then, in scene, set up many virtual cameras, and put according to certain structural requirement, add with it and play up pinup picture at each camera, to interact with corresponding Shader.Secondly, according to the viewpoint sub-pixel mapping matrix calculating, write corresponding Shader, the pinup picture that every camera is played up carries out sampling processing, and obtains composograph.Finally, be written to the output frame of playing up of GPU, output to bore hole 3D display screen, obtain stereo-picture.

Fig. 2 is the Unity3D camera structure specific embodiment the present invention relates to, and given here is parallel convergence type stereoscopic camera structure, and taking 9 cameras as example.Wherein, the convergence plane of 1 optical axis 2 that is many cameras, also referred to as parallax free face.3 is the imaging system central point place plane of many cameras, and 4 is virtual camera.

Fig. 3 is the viewpoint sub-pixel mapping matrix specific embodiment that the present invention calculates.Wherein, the elongated grid in figure represents a sub-pixel, and the numeral in each elongated grid is the corresponding viewpoint number of this sub-pixel, the namely value in matrix Q.

Fig. 4 is the synthetic anaglyph specific embodiment of the present invention.Automobile in figure is the model of making at 3ds Max and exports to FBX form, import in Unity3D scene, add background and shadow, then according to many cameras of layout noted earlier, the pinup picture texture that compile script and Shader play up out to many cameras is processed, the final composograph obtaining as shown in Figure 4.By this composograph output display, to bore hole 3D display screen, beholder, at viewing ratio place, just can see the automobile of the screen that flies out.

Fig. 5 is the concrete embodiment of bore hole 3D display the present invention relates to, and given here is device vertical view.Wherein, 5 is the sub-pixel pitch p of 2D display screen, and 6 is 2D display screen, can be LED, can be also LCD, and 7 is slit grating, and 8 is the grating constant of slit grating 7, and 9 is the distance of slit grating 7 apart from 2D display screen 6, and 10 is viewing distance.What deserves to be explained is, the grating relating in patent of the present invention is not only confined to slit grating, is equally applicable to column mirror grating, is only here to carry out device explanation as an example of slit grating example.

The foregoing is only the preferred embodiment of the invention, not in order to limit the invention, any amendment of doing within all spirit in the invention and principle, be equal to and replace and improvement etc., within all should being included in the protection range of the invention.

Claims (3)

1. the bore hole 3D display system based on Unity3D game engine, it is characterized in that: the sub-pixel structure for the 2D display screen of bore hole 3D display unit should meet as follows: a complete pixel is made up of 1R1G1B tri-sub pixel lamps, arrange by red, green, blue or indigo plant, green, red circulation in the horizontal direction; Be the same of the same colour of red, green, blue in vertical direction, each sub-pixel spacing in the horizontal and vertical directions all keeps equal.

2. the bore hole 3D display system based on Unity3D game engine according to claim 1, is characterized in that: by slit grating or column mirror grating being assemblied in to front or the rear of 2D display screen, wherein, the parameter designing of slit grating meets following feature:

A) grating constant, the computing formula in grating cycle is:

Wherein, K is viewpoint number, and s is adjacent viewpoint spacing, is determined by viewpoint compound mode, and every a pair of viewpoint is combined in watches the spacing of plane and the interpupillary distance e of human eye to equate, adjacent viewpoint spacing is:

Wherein, the vision area number of Δ K for comprising between every a pair of viewpoint combination, eyes interpupillary distance is got e=65mm conventionally;

B) spacing of grating and 2D display screen, screen apart from the computing formula of l is:

Wherein, H is viewing distance, and namely beholder is apart from the distance of 2D display screen;

In addition, the parameter designing of column mirror grating meets following feature:

A) grating constant, the computing formula in grating cycle is:

B) computing formula of the thickness t of column mirror grating is:

Wherein, the refractive index that n is lens pillar, is determined by post grid material;

C) computing formula of the radius of curvature r of column mirror grating is:

。

3. the bore hole 3D display system based on Unity3D game engine according to claim 1, is characterized in that: utilize Unity3D game engine to realize the thinking of bore hole 3D Display Technique as follows:

A) in Unity3D software, set up scene, object model imports to Unity3D scene after by 3ds Max, Maya modelling software development, and in scene, puts many virtual stereoscopic cameras; Every virtual camera adds and plays up pinup picture Render Texture in target pinup picture position, real-time update in the time of operation; The parameters of each camera should be consistent, and keeps synchronous shooting; Between many virtual three-dimensional cameras, arrange according to certain structure, within guarantee enters the parallax amount Pan Nong district scope of two width anaglyphs of observer's right and left eyes, be that brain can be fused into anaglyphs different two width an object, there is ghost image and be unlikely, conventionally the placing structure of stereoscopic camera have run-in index, convergence type, from axle run-in index and 4 kinds of structures of arc form;

B) according to 2D display screen parameter and grating parameter, calculate viewpoint sub-pixel mapping mapping matrix, determine that RGB sub-pixel given on 2D display screen should take from the RGB component of which viewpoint, the computing formula of many viewpoints sub-pixel mapping matrix is:

Wherein, (k, l) is the coordinate position of RGB sub-pixel, k _offthe horizontal displacement that represents 2D display screen left edge and raster unit marginal point, α is the slanted angle of grating axle with respect to LCD display vertical axis, X is the number that a grating cycle covers in the horizontal direction RGB sub-pixel, N _totrepresent total viewpoint number, namely participate in synthetic anaglyph quantity;

C) the viewpoint sub-pixel mapping matrix obtaining is programmed in tinter Shader, multiple that utilize Shader on GPU, camera to be played up are played up pinup picture and are carried out other sampling processing of sub-pixel-level, and multiple anaglyphs after sampling processing are added, obtain final composograph; The Shader writing is divided into two types of summit Shader and fragment Shader, process and convert the vertex position that is rendered into the mesh on screen by summit Shader, the Output rusults of summit Shader passes to next step of streamline, the grid of solid is after the rasterizing of hardware, fragment Shader on streamline can be performed, fragment Shader can carry out various tests to a fragment, the successful fragment of final test can be written in the output frame of playing up, thereby becomes a visible pixels on indicator screen;

D) composograph is outputed on bore hole 3D indicator screen, point light action that utilizes grating, makes the light of different anaglyphs separate in space, when observer is at suitable viewing location place, right and left eyes can receive respectively the information from different points of view, thereby produces stereoscopic vision.