CN106933082A - For the method and apparatus of three-dimensional scenic of the coding comprising transparent substance in holophotal system - Google Patents

Abstract

The method for calculating the coding of the reconstruction of three-dimensional scenic (200,300) for the object (300) including partly absorbing light or sound.The method can be implemented in computing unit.It is independent at the origin of diffraction pattern to calculate diffraction pattern by the situation for considering to be absorbed in scene in order to truly rebuild three-dimensional scenic as far as possible.The method can be used to three-dimensional scenic is presented in holographic display device or body display.Further, it can be performed to realize the reconstruction of sound field in acoustic source array.

Description

For in holophotal system coding comprising transparent substance three-dimensional scenic method and Device

The application is that international filing date is on 04 01st, 2011, and international application no is PCT/EP2011/055144,2012 On December 03, in enters National Phase in China, Chinese Application No. 201180027446.8, and is obtained on 08 04th, 2016 Chinese invention patent mandate, it is entitled " for the method and dress of three-dimensional scenic of the coding comprising transparent substance in holophotal system Put " application for a patent for invention divisional application.

Background technology

The present invention relates to a kind of method of the object point values for three-dimensional scenic of the calculation code comprising transparent substance.It enters one Step is related to implement the computing unit of the method.

The method can be applied to the calculating of the hologram of the computer generation of holographic display system or hologram (is copied firmly Shellfish) generation.It further can be used together with other three-dimensional display systems (such as body display), and wherein object point can be in space In in an interleaved manner individually display.

Generally, the present invention can be used for other wave-length coverages outside limit of visible spectrum.With wherein have at least two all the time So that the antenna array that the electromagnetic wave of transmitting can be interfered is combined, it for example can be hoped individual antenna transmitting coherent radiation in radio For simulating and rebuilding electromagnetic spectrum in the case of the spatial analysis of the cosmic radiation that remote mirror is received.For this simulation or reconstruction Spectral region not necessarily must be corresponding with spectral region to be analyzed, but can be mapped on the former by conversion.

The present invention can further be applied to other media outside electromagnetic spectrum, for example, being used for sound wave.With wherein have all the time At least two sound-producing devices can be through controlling to launch coherent wave so that the sound-producing device array phase that can interfere of the sound wave launched With reference to it can be used to simulating and rebuilding three-dimensional sound field, wherein the present invention should not necessarily be limited by the audio frequency range heard.Sound field bag Include the sound value of the room and time change of the three-dimensional scenic containing the object with sound absorption attribute.The method and computing device also may be used To generate for being the anti-phase sound that noise is not only reduced in small space but also in big environment.

The method can also be used for the display and analysis of other spatial distributions for being alternatively non-optical properties.The three-dimensional of physics point Cloth and other specification are mapped to transparence value, three-dimensional body and light source (pseudo-colours imaging).It for example can preferably embody and divide Analyse the distribution of the mechanical stress of various chromatography imaging methods, 3D ultrasonic examinations or workpiece.

Holographic display system (being hereinafter also expressed simply as holophotal system) according to the application is a kind of for three The display device of object data is tieed up, wherein the three-dimensional body data of the scene to be presented are with the shape of the diffraction pattern of scene to be reconstructed Formula is encoded.Especially, the calculating of diffraction pattern will be encoded herein referred to as, and many such coding methods are known.

Coding can be completed by generating the set hologram of all object point information, however, this can be easy to cause a large amount of Calculated load, when particularly with high-resolution display system.

According to other method, to minimize calculated load, the single adjacent area that hologram is divided into same size is (complete Breath is plain (hogel)).Each region thus with spatial light modulator (spatial light modulator (SLM)) used Equal number of unit correspondence.Each holographic-essence carries a large amount of on object point and largely on the letter of the angle of diffraction (holographic-essence vector) Breath.When the holographic-essence is calculated, since it is expected that the diffraction pattern calculated can find from consult table (LUT), so as to realize simplifying.

Or, the calculating can independently be performed for each object point in the way of sub-hologram.Each sub-hologram is only write To the subregion of the modulator surface of the optic light modulator (or spatial light modulator, SLM) for rebuilding.Each sub-hologram The position that object point can be based on partially or even wholly overlaps in modulator surface.If hologram is only small visibility region compiled Code, the method can be applied particularly preferably, wherein providing at least one device for will distribute to one the one of observer's eyes Individual or multiple visibility regions trace into the motion of observer's eyes of one or more observers.Applicant is for example in file DE Such holographic display is described in 103 53 439 B4 and the A1 of file WO 2006/066919.The sub-hologram with make Expect that object point expects brightness or gathers the diffraction lens of modulator surface at desired distance with expectation brightness and color (diffraction lenses) is corresponding.The function of convex lens is for generating object point before modulator surface.It is recessed The function of mirror is for generating virtual object point behind modulator surface.Object point positioned at modulator surface is directly generated.Thoroughly The function of mirror also can precomputation, and storage in consult table.When coded diffraction figure, it may be considered that additional parameter, for example can be with Consider the optical component in conversion function, light source and other light paths of the modulator region of SLM used.This also includes and is intended to subtract The technology of few spot.

Because each pixel in most of displays is presented on two-dimentional SLM surface, the 2D images of pixelation or comprising The three-dimensional 3D of at least two different 2D images (3D shows) is presented can directly be represented on such displays, without too Many necessary adjustment effort.Necessary adjustment relates generally to the area zoom that will present to the resolution ratio of display panel and is aobvious Show brightness that the classification of plate carries out and the adjustment of color.In 3D displays, the method based on being used, the multiple that solid is presented View must temporally and/or spatially be encoded in modulator surface.2D vectograms image must be converted before it is shown It is raster graphics image.

Before three-dimensional scenic can be presented on 2D displays or 3D displays, or it can be for reconstruction in holographic display device Before coding, view must be generated from the three-dimensional data record of the object with its attribute description scene.This process is also claimed For image synthesizes or renders.The method for being largely used to this be it is known, wherein the species of scene description, the desired qualities of view and The mode for actually generating these views is different.

For example, 3D CAD models include the geometric description of object, it is included in three-dimensional coordinate system.Additionally, many is entered The physical attribute of one step can be defined to describe the material of object, including such as opaque article reflectivity and emissivity and Additionally optical properties such as refractive index and absorptivity of transparent substance.For the object of homogeneity, these parameters are only boundary face circle Surely it is enough.Usually, these attributes can not only spacial flex gradient, and they can be dependent on one or more other ginseng Number, such as wavelength and polarization.

Data can also exist in the form of volumetric pixel data in advance.For example it is typically such in medical applications.Work as 3D When scene is generated, it has been divided into independent spatial point or small area of space (voxel).

2D data generation of such as 3D scenes by combining the pixelation of depth map is also possible.Each pixel is to the plane of reference Distance stored in depth map.Such data form in 2D displays and other various 3D for example for that should show dress Put both upper video datas for presenting.It promotes a multiple for scene regarding map generalization.But must provide for excessive data With it can be considered that hidden objectses.

When image synthesizes beginning, it is necessary to be in it will be used to describe the three-dimensional system of coordinate of object position in the scene Generation each views selection position, the position with record scene view camera (virtual camera) position phase Correspondence.Further, it is necessary to virtual location and void in the scene on the active modulator surface for defining the SLM generated for image Intend size.The dimensions of virtual on active modulator surface may differ from its actual size, if for example, using scanning arrangement or thrown Shadow is arranged.Define the position and direction that observer's eyes discover scene in the position of virtual camera.The position can also be located at object Between or object in.Which section is the performance (such as focal length and visual angle) of virtual camera determine with much virtual times magnification digital displays Show.Virtual area by SLM and its position relative to virtual camera determine visual angle.Originate from virtual camera position and pass through The light for wearing the border of the virtual area of SLM defines the space for representing visibility region.Scene parts outside the cone can not show Show.It is that each observer's eye generates same view, therefore be only possible to be perspective view in 2D displays.By for each Individual observer's eyes synchronizing moving virtual camera, observer can virtually pass through scene in image sequence, without aobvious Moved before showing device.If a sensor senses that the movement in front of the display of observer's eyes, then may be based on the information control The movement of virtual camera processed.Further imaging device can be arranged between virtual modulation device surface and observer's eyes.This A little imaging devices can be included in virtual modulation device surface region, and/or are considered that in the attribute of virtual camera.

In holography display, real depth information can be generated by diffraction pattern.This causes that observer may pay close attention to warp The different depth face of the scene of reconstruction, without changing the reconstruction.Therefore, in the case of holography display, it is further suitable that Imaginary observer position, rather than virtual camera.

Further image synthesize during, determine scene which partly be located at visual field in and which partly in fact Border it is seen, for example, which will not be hidden in behind the other parts of the scene.This can be multistage calculating process, its midfield Scape is more complicated or expects to be presented more real, and the effort to be paid is bigger.The position of attribute and light source based on the material in scene, It is contemplated that reflection, diffraction, refraction and scattering, it and then can bring further visible dummy object, surface or point, and they pass through It can be seen that, it is hiding and/or outside visual field scene parts generation.

It is considered that the material properties of object (shelter) calculate the outward appearance on surface in scene.This is included for example by texture It is imaged onto the surface (texture mapping) of object.Because image synthesis is extremely complex process, the thing during pixel synthesizes The outward appearance of body, surface and each picture point can change repeatedly.

If scene includes structure light source, then their influence (illumination, screening can be considered by adjusting the outward appearance on surface Gear), wherein in order to minimize calculated load usually using simplified lighting model.The reflectivity on surface generally uses bidirectional reflectance Distribution function (BRDF) is calculated.

The reality of scene is generated usually using recurrence ray-tracing procedure (Recursive ray tracing method) Border view.It means that the path of each light defined by the position of display picture element and virtual camera is reversed tracking. First, the light passes through a little them to virtual photographing through in the non-concealed surface of the object (hit object) being mapped to The distance of machine determines and classifies.Then, it is considered to the outward appearance of the visible dots for being related to, generation integrated data is so that describe will be in phase Answer the viewpoint presented at the position of display picture element.When the integrated data is generated, consider to be related to one by one Brocken spectrum transparency property, and opaque point (if any) transparency property.For example, by considering to determine transparency And the material properties of the optical path length of light covering determine transparency property in material.Also contemplate for these material properties Spectrum and spatial distribution.

Such a method is described in the B2 of US 7,030,887.Delayed by the multiple depth using storage depth information Deposit, the transparent pixels that will be overlapped mutually are classified according to depth.This allows to find the picture closest to opaque pixel Element.Then, the transparency for calculating the pixel relative to the opaque pixel influences.Then, find out with the presence or absence of with before The adjacent transparent pixels of transparent pixels.The transparency for calculating the pixel relative to the transparency influence being computed influences.Repeat The process, the transparent pixels until considering all superpositions.The method has the disadvantage in that, i.e. be only capable of being corresponding on display panel Each relevant ray of pixel determine a brightness value or a brightness value and a color value.

Acoustics and geometry attribute (auralisation (auralisation)) in room is considered are in a room There is a problem of during simulation sound field similar.It is this to simulate for evading the extensive measurement in geometrical model.Can thus test The sound source of diverse location, movement, the correlation of polar mode and loudness and room acoustics.Except with a certain position and shape Formula, the object of each object or auditory scene in space also show wavelength and specifically absorb and spread.In multistage process The acoustic properties in middle discovery room, wherein also using recursive ray-tracing procedure.Furthermore, it may be considered that for example, by reflection, expanding Dissipate and deflect the virtual sound source for causing.The sense of hearing being computed typically is rendered by three-dimensional earphone at the position of virtual hearer, Wherein it is necessary for the true sense of hearing and considers head related transfer function (HRTF).One is only existed herein to be rendered by three-dimensional earphone Aggregate signal be shortcoming.It is touring in Virtual Space when the position of the change for hearer calculates the sense of hearing again (round tour) is still possible, but the true sense of hearing that voice signal is no longer calculated after head movement is impossible 's.

The content of the invention

It is logical according to the present invention when presentation includes the three-dimensional scenic of transparent substance in holographic display device or body display The method characteristic crossed in claim 1 overcomes these defects.For field of acoustics, according to the present invention by claim 8 Method characteristic overcome defect.

According to the present invention, the creative method can be implemented in the computing unit of the feature including claim 10.

Present invention further optimization embodiment and improvement are defined and defined in the dependent claims.

The method according to the invention can be particularly used for as at least one observer's eyes as the spatial light of holophotal system is adjusted Device (SLM) calculating holographic coding value processed, the wherein holophotal system are used for the reconstruction of three-dimensional scenic, and the three-dimensional scenic includes having thoroughly The object of bright attribute.The method is comprised the following steps：

A) three-dimensional scenic is divided into single object point and detects the coordinate of these object points.These coordinates are used, be can perform according to side The d of method) step classification, or these object points other purposes for can be used in the calculating of holographic encoding value of coordinate.

B) imaginary observer position is determined, the position corresponds to the observer's in selected obvious perception three-dimensional scenic The position of eyes.

C) determine all object points for not covered by other object points from the observation of the position of imaginary observer and treat it Coding.

D) by all visible points according to all visible object point to be encoded to imaginary observer position distance minute Class, all visible object points can be seen that from imaginary observer position in same angle.

E) intrinsic brilliance of each visible object point is determined, if possible, it is considered to which all true and virtual light sources of scene exist Position and the intensity of their angle are observed at the position of those object points from imaginary observer position, is included truly wherein being contemplated that With the physical attribute of the object of virtual light source.

F) to the visible object point of each to be encoded, it is contemplated that it physical location intrinsic brilliance, to virtual object point away from From and positioned at all objects between visible object point and imaginary observer position or object point transparency property, it is determined that by its The apparent brightness value of imaginary observer's location aware object point.

G) be separately encoded each object point with its respective brightness value so that its with the brightness value in holophotal system in its position Put and rebuild as far as possible so that opaque and transparent object point can be independently perceived at each of which position.

It will be noted that holographic encoding value is also referred to as hologram data or is simply referred as hologram.

If holographic display device is used or is configured so that small virtual observer's window, such as in WO 2006/ Holophotal system described in 066919 A1, one of imaginary observer's window can use to each eye of observer, should be extremely The calculating of few each execution holographic encoding value to imaginary observer's window, the position observation person in imaginary observer's window Eyes.If holophotal system includes the tracking function of imaginary observer's window of the physical location for observer's eyes, if The position of the interior perhaps imaginary observer window of scene is changed, it is only necessary to calculate the actual and apparent brightness of object point again Value.If however, the coding of holographic encoding value is adapted to the new position of imaginary observer's window, it is contemplated that for being observer The shortcoming and/or imaging device of the follow-up mechanism that the new position of eyes is encoded to object point and the local spy of optic light modulator Levy.

On imaginary observer's window as the above-mentioned, it is alternatively referred to as virtual observation window mouthful, it shall be noted that be not There are physical holes or physical frame or related other optical elements.Imaginary observer's window of the invention is a region, Wherein can be seen that the three-dimensional scenic that will be rebuild by holophotal system.Therefore, the eyes of observer are located on or near virtual observation Person's window.

If holophotal system includes big imaginary observer's window, the respectively list of the view in imaginary observer's window One direction, reality and apparent brightness value to each object point perform calculating.Normally, if view window has circle, see The person's of examining window has the size of diameter about 10-20mm.If view window has rectangle, imaginary observer's window is typically side The rectangle of a length of 10-20mm.If however, imaginary observer's window have more than stock size size, can be with single angle Fragment (single angular segments) scans big imaginary observer's window.It is in such a case each single angle Degree fragment performs the calculating of holographic encoding value.If necessary, median or inserted value can be calculated with known interpolation method.Once Calculating is performed through for each angle sections, then can respectively perform the coding of encoded radio.By comprising the maximum possible angle of diffraction The feature of optic light modulator determines the full-size of imaginary observer's window.Based on the full-size of imaginary observer's window, Necessary increment is generated from that.It is responsible for being depended on the resulting structure feature of the diffraction of the light of optic light modulator interaction The type of coding is (for example, Bookhardt coding (Burkhardt-encoding), 2- phase codes (2-phase- Encoding)) and the type of optic light modulator is depended on.The pixel quilt of a plurality of modulator units or optic light modulator Amplitude and phase value for encoding complicated encoded radio.These a plurality of modulator units define effective knot of optic light modulator Structure size and the diffraction at that.For example, performing 2- phase codes (2-phase-encoding), so phase is being embodied in Encoding phase value in two adjacent pixels of the optic light modulator of position modulator.Then, that two encoding phase value codings are multiple Miscellaneous encoded radio.If using diffraction structure (such as optically addressable spatial light modulator for not having grid or rule (OASLM) optic light modulator), then minimum resulting structure size depends on writing for programming unit.

The method according to the invention can also apply to body display device.If when the actual bit of follow-up mechanism tracing study person When putting, change position of the observation by the body Display Technique just observer of the three-dimensional scenic of display, then it is particularly true. Use such display device, it is impossible in different direction encoding difference brightness values, like this with holophotal system or holographic display Device is possible, wherein can encode an object point with the different or single directions that different diffraction figure is view.

In holophotal system, due to in a region or in the finite region of an optic light modulator or optical modulator it is complete An object point in the domain of main plot can produce the diffraction pattern of single object point, another outside effective grid or diffraction pattern of optics light regulator Grid or diffraction pattern can be used.

The method according to the invention can apply to generation such as 3-D wireless electrograph (computer tomography, CRT), visual Change and analyze, for the mesh of diagnosis wherein in the transparency and Absorption Characteristics of material (in particular for the biomaterial of x-ray) Investigation and display.

In substantial amounts of application, three-dimensional scenic is not only described by brightness value or intensity level, and three-dimensional scenic outward appearance Also rely on the transmitting of individual element or object in three-dimensional scenic, absorb and/or reflecting attribute.

In a preferred embodiment, it is contemplated that actual color value at each object point position to be encoded and positioned at visible The transparency property of all objects and object point between object point and imaginary observer, in addition to apparent brightness value, waits to compile for each Through determining, the object point with the apparent colour value can be perceived the apparent colour value of the object point of code at imaginary observer position. Each object point is separately encoded with its respective color value, to the greatest extent may be used at its reality or virtual location with being passed to this color value Can be rebuild far in holophotal system, therefore opaque and transparent object point can be respectively perceived at each of which position.

Method according to the embodiment can be applied to using color model and apply color depth, wherein can encode that weight is treated The scene built.Usual color model for example, can very well be adapted to the additional red of this place, green, blue model (RGB colour model).Use these color model, with launch or propagate light 1/3rd it is red, 1/3rd it is green Color and 1/3rd blue subpixels generation color.Stored in three gray value passages that color red, green, blue is presented respectively The brightness value and color value of object point.The numerical value of the maximum possible of gray value determines possible color depth.Generally, further The transparent feature of object point is stored in passage, it is also referred to as α passages.Other color model for example using for each color (form and aspect, Saturation degree and value (value), HSV models, or form and aspect, saturation degree and brightness, HSB models) color value, color saturation and bright Angle value.Further, also exist for the color model for being particularly shown device or form through adapting to, such as TV format The YUV models of NTSC or PAL.Especially, the printing technology or optical modulator in reflection model is worked in, use mould of losing lustre Type.Therefore example is blue-green, carmetta, yellow model or blue-green, carmetta, key (CMYK), and wherein key represents black part Point.Such model is adapted to the generation using the Hard copy holograms (hard-copy-hologram) of printing technology.

Because the additional virtual object point that reflection, alleviation or scattering can be generated virtually is computed, it is at least one The result of the interaction between the object point of the light of individual virtual or real light source and the object of the three-dimensional scenic that will be rebuild.Energy Consider the optical properties and virtual or real light sources of object.Be those as the virtual object point of true object point, calculating is virtually being observed Apparent brightness value or apparent brightness and color value at the position of person.Them are respectively encoded with their respective value.

Real light source is the light source for generating directional light apparently in three-dimensional scenic to be reconstructed.Virtual light source can example Such as it is employed by the reflection of the light generated by real light source on the surface of object.Such virtual light source can be generated into one The virtual light source of step, such as by multiple reflections.When also the method for computation holographic encoding value, multiple reflections can be examined Consider.Such multiple reflections generally need the multiple phase process for image synthesis.

In a preferred embodiment, the position of the object point is corrected relative to imaginary observer position, if necessary, due to Optical properties at the object or object point being positioned between imaginary observer position and object point to be modified (for example reflect, reflect Or diffraction).The position corrected for those and calculate apparent brightness value or apparent brightness and color value.Can be distinguished with respective value Encode the object point.

This can apply to include the object (such as mirror) (to complicated object, such as glass jar) with reflecting surface Three-dimensional scenic；Wherein in order to visualize the correct position of the object point in such glass jar (such as fish or stone), it is necessary to Using the law of refraction.

To those skilled in the art, the position with imaginary observer can be performed in the three-dimensional scenic for not including transparent substance The amendment of the position of associated object point is obvious.Therefore, the method according to the invention can be changed by this way, to neglect The method and step of the transparent substance in three-dimensional scenic to be reconstructed is slightly not present on (under special circumstances).

It is distributed by the object or the space transparency of object point that consider to be located between object point and imaginary observer position, is calculated The apparent brightness value or apparent brightness and color value of object point at imaginary observer position.Their spectral correlation performance is examined Consider.

The object of three-dimensional scenic can include unitary relative transparent angle value τ.This transparence value is the transparency T from material With the business of thickness D.Therefore the transparency of object is depended in object along the length of its light path propagated.Generally, object Transparence value τ can be the function of the function (color function) from position (luminance function) or brightness and color.Can pass through (in thing Internal light is along paths) space on relative transparency Distribution value τ relies on the integrated calculating transparency T of item.If object Spatial distribution including refractive index, then this light path can be nonlinear.

If transparent substance includes scattering signatures, such as diffusing screen or ground glass, only directly through the scatterer Light determine to be located at apparent brightness or apparent brightness and the apparent colour of object point most forward behind object point.Scattering light facilitates this The apparent brightness of the object of sample.Such object is also referred to as translucent object.

Preferably, transparent in random device and between object point and the position of imaginary observer and/or reflection and/or With the help of the probability of diffusion and/or the absorption of diffraction and/or the object or object point of refraction, scattering, reflection and diffraction, calculate The apparent brightness value or apparent brightness and color value of the object point at the position of imaginary observer.Their spectral correlations are examined Consider.

If performed further with the help of random device (such as Monte Carlo Method (Monte-Carlo-Method)) Parts of images synthesizes, and this is particularly helpful.Not over regression coefficient description such as absorption, reflection, diffraction or scattering etc. Physical effect, but described by carrying out the probability of these physical effects during self-application random device.For example, can follow the trail of or note Anticipate through the light path of multiple single photons of three-dimensional scenic.Real light sources can be passed through and consider that the probability of the radiation of these photons is given birth to simultaneously Into these photons.These parameters are considered if desired, random number can determine that the radiation feature of these light sources, such as in transmitting photon Position, the direction of light radiation, wavelength and, if necessary, just in the polarization and other photons or light wave of launching light The phase relation of radiation.Can determine whether to reflect in each optical interface by means of equally distributed random number, reflect, spread out Penetrate, scatter or absorb photon.Equally distributed random number is measured in interval generally between zero and one.Can be based on absorb, Propagate, scatter, reflecting and/or the probability of diffraction probability makes such a decision, wherein can consider from as wavelength or polarization Physical parameter correlation.The transparency distribution of the wavelength dependency with birefringent material or material can for example be considered is associated hair The effect of raw polarization.It is all that the photon of object point is directly originated from or at the surface of object to imaginary observer's window by transmitting The direction of mouth can be calculated by the photon that diffusing reflection sends, and these determinations are in the sum of the photon for all generations The intrinsic brilliance or intrinsic brilliance and color of object point after the appropriate measurement of amount.Reach the number of these photons of imaginary observer's window Value is also calculated, and these numerical value determine the apparent brightness of this object point after a measurement.

As the volume efficiency for passing through medium or simplify model in object surface intensity with incidence intensity Ratio transmission rank or light transmittance T can for example be interpreted transmission probability.When incident photon enters medium or thing During body, normalized random number Z (normalized random number Z) is generated.If Z is less than or equal to T, then photon Pass through medium or the surface for passing through object.If Z is more than T, photon is absorbed can't cause apparent brightness.

It is possible using more complicated model, wherein being for example contemplated that the reflection on body surface.If photon is incident Onto surface or optical interface, whether the probability for generating new random number and reflecting R occurs to depend on this random number.It is false Generation is such as reflected, photon changes the direction of its propagation according to reflection law.If generation is scattered or diffusing reflection, by means of one The other random number of group can determine that the new direction of propagation of photon.For example the general of photon deflection is can determine that from the reflection distribution of object Rate.Vacation is reflected if there is not, and another random number for distributing to the absorbing probability of photon is determined, further random depending on this Count, the photon is absorbed from medium or object or the photon continues to travel to the exit interface of object.In the outlet of object Interface, if be reflected on exit interface/optical interface occurred or if photon is passed through, it can be examined.It is directly anti- Penetrate light and generate virtual object point in this case, its position can by review (back tracing) reflection direction or consider into The original-party penetrated always determines.Need only to consider those virtual object points, photon energy travels to virtual observation from the virtual object point Person's window, that is, its apparent brightness value be not 0 or its position not outside reconstruction scope.The position of if so is located at weight Build outside scope, then generate virtual object in the region for extending through the intersection point of grenz ray of the reconstruction scope that will be rebuild Point be it is possible, itself however perspective distortion may be caused.Certainly this is also possible to be pointed to true object point outside reconstruction scope 's.These object points are so as to be projected or be imaged onto in the background of reconstruction scope.If on the way photon is refracted, this causes phase The significantly different position of the object point answered.The new position of object point is also can determine that by reviewing deflecting light beams or photon spread paths Put.Normally, it is the intrinsic brilliance of object point, only considers (transparent by launching towards the direction of imaginary observer position from object point Light source) generation photon, or direction towards imaginary observer position at medium or body surface reflects in decentralized manner Photon.If be located between such object point and imaginary observer position without other objects or medium, then close to void The numerical value for intending the photon of observer position represents the apparent brightness of such object point.

It is possible that transparent object point can generate virtual object point in the same mode of such as opaque object point.

The apparent brightness of transparent substance can be also influenceed by the scattering of light in object or diffraction.When consideration transparent substance When, preferably consider these effects with similar to the method according to the invention.

There may be certain situation, wherein the application of such random device needs a large amount of calculating time and substantial amounts of calculating Internal memory, if three-dimensional scenic especially to be reconstructed includes the high-resolution that must be calculated and a large amount of photons.Such random device Therefore can not be performed in real time.However, even if with complicated scene, simple algorithm still can be employed, and The calculating of photon can be performed by parallel processing.Therefore, if it is necessary to perform static scene or holographic video, such side Method can be applied especially.So the generation of the generation or master to hard copy is also the same.

In a preferred embodiment, in order to improve or reducing that observability put, in the form of amplifying or weaken In the apparent brightness value of single object point or apparent brightness and the calculating of color value, it is considered to positioned at object point and imaginary observer position At least one transparence value of at least one single object between putting.

It is possible that the area of scene or region are reconstructed, and the area or region of the scene include this ground with the scene Naturally the apparent different apparent brightness in area or region and/or different apparent colours.It is possible that, can be at it Apparent aspect is amplified or weakens or suppress and/or change the observability in this region.Therefore, the method according to the invention can Implement algorithm, it causes that to be changed or control the apparent of viewing area can be temporarily stored (such as in calculator memory), So only change the regional without recalculating completely to scene through changing of the scene.If, such as example in WO Disclosed in 2006/066919 A1 like that, the method according to the invention application sub-hologram is with calculating holographic coding value, then Before operation, when the summation of sub-hologram is generated, the sub-hologram of the only object point in the respective region of scene can be subtracted, in behaviour After work, the sub-hologram of the object point through changing can be added in the summation of sub-hologram.The control of operation can for example due to At least one observer interacts and is performed.For the diseased region in the visual range for emphasizing organ in medical application Domain, can be by single region for changing the transparent feature of such as scene areas or single object so as to change three-dimensional scenic to be reconstructed Apparent operation.

In another aspect of this invention, the method for the present invention can be applied to calculating will be used to be closed with sound-reproducing system The encoded radio of connection.The sound-reproducing system includes that at least two sound generating apparatus and at least two sound generating apparatus are used In the reconstruction of the three-dimensional sound field of the different sound value of the room and time to contain three-dimensional scenic.The three-dimensional scenic includes tool There is the object of sound absorption properties.Three-dimensional sound field is reconstructed to be perceived with by the ear of at least one hearer.The method includes following Step：

H) three-dimensional scenic is divided into single object point.Single object point can influence sound.Determine the coordinate of these object points.

I) position of virtual hearer is determined, it is corresponding with the position of the ear for being chosen for hearer, wherein acoustically bright It is aobvious to perceive three-dimensional scenic.

J) in the direction of the position of virtual hearer, it is determined that the institute not covered fully by other object points for fully absorbing sound There is object point.

K) all object points for being located at the locality for carrying out self-virtualizing hearer arrive the distance of the position of virtual hearer by them Classification.

L) actual loudness at the position of each object point that can influence sound, pitch and sound delivery time are determined, such as Fruit if possible, at the position of those object points, with from their angle of the location aware of virtual hearer, it is considered to scene institute There are position and the intensity of actual and virtual sound source, wherein the physical attribute of the object comprising reality and virtual sound source can be examined Consider.

M) for each object point that can influence sound, by the actual sound at the position of the object point for considering that sound can be influenceed Degree, pitch and sound delivery time, to the distance of the position of virtual hearer and between the position of object point and virtual hearer All objects or the absorption properties of object point determine apparent loudness, pitch and sound delivery time, the sound is by described apparent Loudness, pitch and sound delivery time are perceived at virtual hearer position.

N) each the sound value including loudness, pitch and sound delivery time is separately encoded, in this way, in the position of virtual hearer Place is put, the reconstruction with the sound-reproducing system can be perceived with such apparent loudness value, pitch and sound delivery time.

The sound-absorbing object of the scene between sound source and the ear of hearer reduces the loudness of sound, causes observer's Perception at the position of ear is quieter than the actual place of sound source or sound source.The face or object for reflecting sound can generate additional void Onomatopoeia source.At the ear of hearer, compared with the sound (not reflecting) of direct this sound source for reaching, so reflected Sound can later arrive at (i.e. under different phase or alternative sounds delivery time or direction), therefore cause hall or echo Effect.Phase delay or the sound delivery time of this sound are influenceed by the velocity of sound in acoustic absorption medium.Can vibrate or shake Dynamic object scene can cause vibration or vibration by subsidiary sound, therefore cause virtual sound source.

It is the left and right ear of observer for the true perception of sound according to a preferred embodiment of the invention, Calculate respectively and coded portion sound.

To three-dimensional scenic, with the help of the field of single sound generating apparatus, the reconstruction of encoded sound distribution can be performed, Wherein this constitutes at a relatively high resolution ratio, wherein according to their phase relation, single sound generating apparatus should be by synchronization Ground control.Additionally, single sound generating apparatus should include the frequency spectrum high of the sound that can be generated on it.Can respectively consider The sound generation feature and the sound of sound-reproducing system and scope (volume) of each or at least one sound generating apparatus Sound feature, wherein when the calculating of encoded radio is performed for sound generating apparatus, sound is generated.

Accordingly, with respect to optical perception (i.e. the apparent brightness or apparent brightness of three-dimensional scenic and color value), particularly preferably Be combined for the calculating of three-dimensional scenic and coding and reconstruction by ground, for example, describe these in claim 1-7.In addition, In the same time, the reconstruction of the sound of such three-dimensional scenic is calculated, encodes and/or rebuild.In other words, except according to claim Outside calculating and coding apparent brightness or apparent brightness and color value in 1-7 in any one claim, always according to right It is required that 8 calculate and encode virtual loudness, pitch and sound delivery time.

The field or arrangement of holophotal system (such as holographic projector) and sound generating apparatus are used.For example it is possible It is to generate virtual scene in the mobile scope of observer.According to the reconstruction of his moving tracing within the range or it is The whole mobile space of observer and calculate and rebuild the scene, in the range of such observer can truly see and hear and come from Each position three-dimensional scenic.

When it is determined that reality or apparent brightness value, or reality or apparent color value, and/or reality or it is apparent Loudness value, pitch and during the sound delivery time, the method according to the invention, it is not limited to ray tracing or ray is calculated.Hold The method that the analysis of this method of going can also include the fluctuation characteristic or Finite Element (FEM) that consider light.For example use three dimensional field The simulation of scene physical process, such as Temperature Distribution or mechanical tensile forces or the distribution of mechanical stress, the method for finite element method (FEM) can have It is employed sharply, wherein, perform the weight of distributed in three dimensions at false colour visualization (false colour visualization) aspect Build.

In order to perform the method according to any claim in claim 1-9, computing unit can change with optimal Mode perform single method step.Therefore, it is the reconstruction of three-dimensional scenic according to the present invention, optics light modulation is respectively provided The computing unit of the calculation code value of the sound generating apparatus and/or sound-reproducing system of device (SLM) and/or holophotal system.Three Dimension scene includes thering is transparent optical attribute and/or the object with acoustic absorption attribute, and is at least one observer's The ear of eyes and/or at least one hearer is rebuild.Any described method in computing unit perform claim requirement 1-9.

Such computing unit can include at least one programmable processor core (programmable processor ), and/or any kind of at least one programmable logic device (PLD), and/or at least one application specific integrated circuit core , and/or at least one digital signal processor (DSP) (ASIC).Device as at least two can be combined in an integrated electricity Lu Zhong.

The computing unit can further include part, device, power supply as such as store program codes and/or data (power supply), control and make mode of operation and/or result of calculation visible apparatus.

Computing unit can be a part for the system controller of the reconstruction for showing three-dimensional scenic, such as holographic display device, Holographic 3D television equipments (holographic 3D-TV-device), 3D game devices (3D-gaming-device), holographic number According to visualization/reconstruction mobile device and/or rebuild three dimensional sound distribution/sound field device.The computing unit can be with independent Unit be provided, the single unit be connected to computing electronics or another reception and/or generation and/or 3D scenes storage Device and holographic display and/or playback three-dimensional sound field system between.

Computing unit can be general-purpose computing system (the general computing of the calculating that can be used for three-dimensional scenic System a part).

Computing unit can include part different technologies, for example optically (the amount of the calculating for for example being performed based on electronics Sub- computer and computer unit) operation computing unit.Particularly advantageously, if such computing unit includes a large amount of energy Operate the unit of account of the device of parallel processing.

Brief description of the drawings

In order to be fully understood by the purpose of the present invention, technology and structure, following detailed description and drawings are should refer to, its In：

Fig. 1 (a) shows the difference between 3D stereophotographies and natural viewing/holography；

Fig. 1 (b) shows the principle of traditional holography；

The big viewing areas that Fig. 2 (a) displays are generated by using conventional method；

Fig. 2 (b) shows the viewing areas produced by using sub-hologram；

Fig. 3 is shown as the viewing window of the viewing areas reduced by using sub-hologram；

Fig. 4 shows the general overview of our holographic process flow；

Fig. 5 shows the schematic diagram of the reconstruction of camera setting and generation；

Fig. 6 (a) is such figure：It is used to explain be arranged at along a single eyes --- display --- light The reconstruction at multiple sight spots of different depth；

Fig. 6 (b) is such figure：It is used to explain in an exemplary scene, and it has the phase in holographic plan Same position, but there are multiple 3D sight spots of different depth by using multiple content-datas layer；

Fig. 7 shows the typical topology example of video frame；

Fig. 8 shows the overview of the handling process based on GPU；

Fig. 9 shows the overview of the handling process based on FPGA；

Figure 10 shows the situation of opaque article and transparent substance in actual life/nature；

Figure 11 is such figure：It is used to explain the reconstruction for calculating/generating three-dimensional scenic according to the present invention；

Figure 12 shows the principles and methods of the view of the simple three-dimensional scenic including opaque and transparent substance.

Detailed description of the invention

Accompanying drawing 12 shows the principles and methods of the view of the simple three-dimensional scenic including opaque and transparent substance.

The three-dimensional scenic that should be rebuild with holophotal system or holographic display (not shown) includes opaque article 200 and transparent substance 300.Two objects 200 and 300 can be divided into multiple single object points (not showing in Figure 12).The white of diffusion Light source (not shown) irradiates three-dimensional scenic from all directions.In fig. 12, schematically show that the light 101 of light source is arrived 110.For the eyes 400 of observer, three-dimensional scenic is rebuild.(eyes 400 of imaginary observer are from the position for position in scene Substantially perceive three-dimensional scenic), through being defined as the position of imaginary observer, also represented with reference numeral 400.Be positioned in virtually The transfer side of the object 200 of the correlation of position 400 of observer (that is, is watched, after object 200 from the position 400 of imaginary observer Object point just) is helpless to the visualization of object 200, thus without be calculated or encoded.Such as the simple scenario shown in Figure 12 Do not include the object with reflecting surface, and the scene does not also include having directional light (directional light) transmitting special The light source levied.Therefore, in the scene without dummy object or virtual light source.Because the diffusion of the three-dimensional scenic with white light is shone It is bright, so object 200,300 can be by the observer of the physical location positioned at them, according to the material characteristics with object 200,300 Related their color value and perceive.In the example of Figure 12, object 200 includes yellow.White is absorbed by object 200 The blue portion of the luminous intensity of light.In all directions, the red and green of white light is fully launched.Imaginary observer's At position 400 the visible of opaque article 200 will be perceived with its actual strength 510 (IAo=IAo_r+IAo_gr) with red Object point, if not having transparent substance 300 in three-dimensional scenic.IAo_r and IAo_gr are from the object point from opaque article 200 To the apparent intensity or actual color value of the direct light of imaginary observer position 400.The strength portion or color value of blue IAo_b Equal to 0.From the brightness of diffusion illumination, the absolute value of actual strength is also determined.What is occurred in the three-dimensional scenic of Figure 12 is transparent Object 300, respectively includes Tr=0 to red and blue light, 5 and Tb=0,5 transparency.Transparent substance 300 is fully inhaled Receive the strength portion of green light, i.e. the transparency of green light absorbs red for the Tgr=0.50% and transparent substance 300 of blue light The 50% of coloured light, and blue light and red light 50% transmitted.Opaque article at the position 400 of imaginary observer 200 object point can be perceived in apparent intensity ISo 520 times, and therefore ISo 520 is ISo=Tr*IAo_r+Tgr*IAo_gr+ Tb*IAo_b=0,5*IAo_r.In the example of Figure 12, with the half with red color intensity part at virtual location 400 The red observation object point for perceiving opaque article 200 of intensity.Calculated with this intensity level in the holographic reconstruction of three-dimensional scenic With the coding object point.

The object point of transparent substance 300 includes the purple such as seen from the direction of the position 400 of imaginary observer.With diffusion Three strength portions Ir, Igr and Ib of the red, green, blue of white light source can calculate actual strength IAt 530, IAt=0,5*Ir +0,0*Igr+0,5*Ib.There is no further transparent substance to be located between transparent substance 300 and the position 400 of imaginary observer (IAt=ISt).Therefore, the intrinsic brilliance of transparent substance 300 is also apparent brightness.When three-dimensional scenic is rebuild, these brightness are used Value calculates and encodes the object point of transparent substance 300.In this simplified example of accompanying drawing 12, the optical interface in object is not accounted for On reflection.Additionally, have ignored the change of the transparency of the light path depended in medium.

It is to be noted that opaque object point also influence be located at imaginary observer locality on transparent object point reality and Apparent brightness and/or actual and apparent color, because the light source in such opaque materials physical efficiency shield or covering scene Light.

When performing image synthesis, accurate physical effect can be more considered, can more generate the real views of three-dimensional scenic.

Especially, in order to obtain the three-dimensional perception of three-dimensional scenic to be reconstructed, for all eyes of observer Different views are generated.If watch reconstructed hologram, such as hard copy under a big visual angle simultaneously, then be directed to A large amount of such view of the position of different imaginary observers is calculated, wherein for centre position, from the view of adjacent view Place's insertion view is possible.In doing so, the interior of three-dimensional scenic can be changed with the position (multiplex holography) of imaginary observer Appearance is possible.The calculating of single view in a continuous manner and/or can be performed preferably in a parallel fashion.It is single for observer Eyes, the position of imaginary observer can be adapted with the movement of observer, if the position of observer changes.

Because respectively encoding the visible object point of opaque article 200 and transparent substance 300, it is thus possible to, due to When the object of three-dimensional scenic is seen, not there is mispairing between the convergence and regulation of eyes, so observer can use him Eyes respectively focus on each single object 200,300, and therefore almost without visual fatigue ground observation rebuild three-dimensional Scene.Therefore it is possible that, observer can observe reconstructed three dimensional field with his natural eyes movement without distortion Scape.

When the calculating of three-dimensional scenic is performed according to the present invention, known program library can be used to calculate such scene. This program library is, such as open graphic library (OpenGL), 3D Graphics parallel libraries (Direct3D) or XNA frameworks (XNA- Framework).It is possible with known mathematical method, for example, accesses in the middle of the pre-calculated values or insertion for checking table (LUT) Value.

Although the present invention has been described in conjunction with specific embodiments, it is understood that, in view of foregoing description, many is replaced The scheme in generation, modification and change will be apparent to those skilled in the art.Therefore, it is contemplated that including belong to from Belong to all these replacements, modification and the change of the scope of claim.

Annex

In order to further the open information on the invention described above provides following content.It is stressed that：Even if the letter During breath may be provided in only in the annex without superincumbent specification, but the annex constitutes the part of the application.

The generation of the content that holographic real-time shows, coding and present

What is discussed is the scheme for driving holographic display device.The holographic display device has by using regarding riel (SeeReal) The sub-hologram technology (Sub-Hologram-technology) of company and with reference to ready-made hardware real-time coding interactive mode or regard Frequency content.Introduce in order to from content aspect and holography in terms of properly generate include on hologram transparency in terms of answer The guide of miscellaneous content.The method of the hologram that traditional generation is generated through computer is carried out with our use sub-hologram scheme Compare discussion, power is calculated with quick reduction.Finally, the calculating platform of our 20 inches of intuitively holographic prototypes and explanation To introduce.

1. introduction

Due to calculating a large amount of consumption of power, the method for the hologram (CGHs) that traditional generation computer is generated is not very It is applied to interactive application well.So, using them, only the video of still image or precomputation is through performing.In order to realize The advantage of the key of 3D holographys, compared to 3D stereophotographies, interactive content is necessary, and it is provided will be typical As the road sign that the 3D application programs (3D plays or 3D TVs) of the design of specialty are combined with the viewing comfort level of 3D holographys. It is desirable for therefore, there is no need to the holographic real-time numerical procedure of high-performance calculation software.

The background technology of some attracting interactive holographic applications programs of generation will be introduced herein.Additionally, we are new having Effect can also discuss that it causes hologram meter using the change of graphics processing unit or the sub- holographic technique of field programmable gate array Calculation is carried out in real time.

2. real time holography

This chapter reviews holography and compares traditional method and regard the new sub-hologram technology of riel company.Should Sub-hologram technology is the basis of the hologram for calculating big in real time.

Why 2.1 want holography

Compared to 3D stereophotographies, holography overcomes the depth of field (depth-cue) between deep focus and convergence The problem of mispairing.This thus be referred to as regulation assemble mispairing (accommodation-convergence mismatch) cause Fatigue or headache, in addition it is of short duration get lost, therefore use 3D stereophotographies, only small depth bounds must be realized and not The use 3D stereophotographies of pause will also be very limited²。

Conversely, holography picture natural 3D viewings, it allow for very big depth bounds, in the absence of negative effect, because For eyes can simultaneously focus on and converge at the object seen.When hologram is watched, the object of the focusing seems clear, and Will appear to obscure in other objects of different distance, just as its be in actual life.Use 3D stereophotographies, eyes Mispairing occurs in itself to be focused on object but focus on display, and it causes the result being described above (see Fig. 1 a).

This be why holography will be 3D stereophotographies current fast-developing market next major step, because It is the preferable selection of its application program (that is, professional 3D designs, 3D game and 3D TVs) for many fields.

(referred to as son is holographic with the new scheme regarding riel company by the method for traditional generation hologram for next chapters and sections Figure) compare.The use of sub-hologram makes it possible to calculate big and deep hologram in real time, and it causes holographic display device On interaction content can be realized by using ready-made hardware components.

Referring to Fig. 1, it shows the difference between 3D stereophotographies and natural viewing/holography in Fig. 1 (a)：Pin To 3D stereophotographies, two eyes are focused on the object of certain depth, but are focused in display plane, for the opinion of nature See and holography, focus on and convergence is identical.Fig. 1 (b) shows the principle of traditional holography：Multiple big overlap diffraction Figure has rebuild multiple sight spot (scene-points), and the reconstruction is can be seen that in the viewing area defined when coherent light source illumination.

2.2 conventional method vs sub-holograms

Hologram is generally complicated diffraction pattern.When coherent light source illumination, the 3D scenes that sight spot is constituted are reconstructed, its Delimited area that can be in space is visible (see Fig. 1 b).

The method of the hologram (CGHs) that traditional calculating is generated by computer is generally based on following scheme：Each is complete Pixel in breath figure contributes to each reconstructed sight spot.It means that for each sight spot of scene, it is necessary to which calculating has entirely The diffraction pattern of portion's hologram size.These single holograms are all added together --- by complicated superposition --- to generate The hologram of full scene is presented.

Referring to Fig. 2, it shows in Fig. 2 (a)：When using conventional method, big viewing Area generation, but in observation Only fraction is to be actually needed at the position of person's eyes, therefore most of information waste for calculating.Fig. 2 (b) shows：When making When using sub-hologram, necessary information is only calculated.Additionally, the resolution ratio of holographic display device is relatively low and just in today manufactures energy In the range of power.

On the one hand these traditional holograms provide very big viewing areas, but on the other hand need very small Pel spacing (i.e. about 1 μm) (see Fig. 2 a) to be reconstructed.Due to the general principle of holography, the interference of diffraction light, the viewing area The size in domain is directly to be defined by pel spacing.When the viewing areas are sufficiently large, two eyes sense different distant views automatically, because They can focus on and be focused at identical point for this, and the reconstruction of the 3D scenes can be independently watched even more than user.

It is that the quantity of the pixel that each sight spot calculates is very huge in traditional hologram.Except providing useful showing Show the missing of the Display Technique of the small spacing of device size, it will need surprising calculating power.Additionally, such substantial amounts of data Treatment cause it is even more on data transfer rate, memory the problems such as.This is why to use the real-time complete of conventional method Breath art will not see the key reason of commercial interest in visible future.Due to the limitation of these technologies, up to now, only as hard copy Or stage photo as chemical film can be suitable for the size of the application program of desktop computer or similar TV and realize and with expansible Technology is realized.

When Fig. 2 b are watched, it can be seen that the most of information being computed in traditional hologram are to waste, because The information that only eyes can actually see is real needs.Therefore, instead of calculating whole viewing areas, only part of this is complete Breath figure needs to be calculated, and it is responsible for rebuilding this reduction of specific 3D sight spots sub-hologram (SH) at the eye position of observer Viewing areas be alleged viewing window (VW) (see Fig. 3).

The reduction of the size of the viewing areas is by increasing pel spacing completion-pel spacing together with other parameters Define the size of the viewing areas.By overlapping the SH (addition or overdetermination position) at different sight spots, the holography with dense sight spot 3D scenes are reconstructed and position in VW at be visible (see Fig. 3).

On the other hand the increased pel spacing result in the pixel count of violent reduction so that have another motivation to use current Display Technique.But the use of small VW implys that quick, responsible and point-device eye tracking system according to the observation The need for eyes movement carries out transfer VW.Such eye tracking system has been developed, but is used depending on riel company at present Be the one's own eye tracks scheme being integrated in holographic prototype, its via regarding riel company such as SID, FPD, The public affair such as Yokohama (Yokohama) and Finetec is proven.

Referring to Fig. 3, which show only a fraction of hologram-sub-hologram-needs in the viewing areas for reducing (viewing window) rebuilds a single sight spot.By the multiple sub-holograms in overdetermination position, generated at the viewing the window's position in space and weighed Build the hologram that whole scene is presented.

How huge have to illustrate to calculate power and save, two methods are generally carried out using a kind of example scenario Compare.

Assuming that with 40 inches of SLM (800mm × 600mm), an observer just watches the display from 2 meters of distant places, see See that region will be both horizontally and vertically ± 10 °, the content be placed on behind above 1m and hologram it is unconfined away from In the range of, hologram HDTV resolution ratio (1920 × 1080 sight spot) has rebuild scene and wavelength is 500nm, then Designated situation must be managed.

Conventional method is directed to here, and carry out the calculating based on Fourier's conversion most has with for so big hologram application The method of efficiency, 256 depth quantizations at the sight spot of step are suppose there is for this.For SH methods, it is necessary to calculate two kinds of independences Hologram, one kind correspond to each eye.

At bottom line, two methods provide identical result for single observer position, but it will be apparent that see See the resolution ratio on light regulator, frame size and calculate the significant difference of power.

In order to further reduce calculating power, it is possible to use alleged single parallax hologram, wherein SH and holographic parallax Size reduction to one-dimensional.Horizontal or vertical direction be it is possible-be referred to as only horizontal parallax (HPO) or only vertical parallax (VPO) Hologram³.Mix by by the different views of half parallax SH and each eye, such as vertical holography parallax and horizontal stereopsis Difference, will generate the real-time video hologram with low calculating demand⁸.It is single if it is well understood and is incorporated to holographic content It is sightless that the perception of the reconstruction of one parallax is limited for observer.

Table 1：The comparing of tradition and sub-hologram method

But even provide full parallax SH hologram can (such as scene can compile by using the technology for having developed of today Journey gate array (FPGAs) and GPU (GPUs)) using the algorithm process regarding riel company, it provides sufficiently meter Calculate power.This is discussed in following chapters and sections.

3. regarding the holographic process flow of riel company

Following four chapters and sections are outlined using sub-hologram as explained above, are shown in real-time 3D on holographic 3D displays The important step of appearance.

Referring to Fig. 4, which show the general overview of our holographic process flows.Step shown in Fig. 4 defines that we are complete Breath software flow, it is divided into following modules：Started with content generation, the data generated by content generators will be shifted To hologram synthesis, wherein calculating the hologram of complex value.Then, holographic encoding the hologram of complex value is converted to be compatible with through The presentation of the spatial light modulator (SLM) for using, holographic display device.Finally, post processor is by for three kinds of color components Different holograms depend on the view of Display Types to mix with two or more, and the picture for so ultimately producing can be presented on On SLM.

4. step I：Content is generated

For holographic display device, two kinds of major type of contents can be distinguished.Firstly, there are generating (CG) by computer Real-time 3D contents (such as 3D plays and 3D application programs).Secondly there are actual life or the video content of life action, it can be with It is the live video from 3D cameras, 3DTV broadcasting channels, 3D video files, Blu-ray Disc or other media.

For most of real-time CG contents (such as 3D plays or 3D application programs), current use graphics processing unit (GPUs) Application Program Interface (API) that 3D is rendered is convenient.The above all Direct3D and OpenGL- of Microsoft API。

When generating and rendering a scene, for each view, 2D figure of the generation with pixel is (based on answering that 3D is rendered With the texture of program interface), wherein each pixel provides color and its 2D position.Each pixel can also regard corresponding 3D scenes as Sight spot.This is why two API are generally especially suitable for generating the holographic process flow processing treated by regarding riel company Content.

4.1 views, color and depth information

In the method regarding riel company, for each observer, two kinds of views are generated, one kind is directed to an eyes.It is right It is to be additionally required accurate depth information for each view-be typically provided in be tied in the difference of 3D stereophotographies Color diagram is known as in depth map or z figures.This expects to see for two kinds of views of each observer for providing each eye Suitable perspective view be very important.They provide aggregate information together.The depth provided with the depth map of each view Information is used at suitable depth reconstruction sight spot, and so each 3D sight spot will be generated in the exact position in space, therefore give The eyes of user provide the correct focus information of natural 3D scenes.These views are independently and according to user position It is generated with 3D scenes in different VW, it is placed at the eye position of each observer successively.

The depth information of the offer must be very accurate, because the sight spot be given in depth map and by both views Parallax provide its depth information must be associated.This rebuilds sight spot for the suitable place of the range of views in holographic display device It is necessary, therefore focuses on and assemble and will match.The correct diffraction pattern (sub-hologram) of generation is used after the depth information, It allows that eyes accurately focus on convergent point.

4.2 virtual cameras

Another is also applied to 3D stereo camera shootings art but often by the vital point that content creator despises is correct 3D phases Machine sets (3D-camera-setup) (for the real camera or virtual camera of real-time 3D contents), to two views of eyes Generate therefrom.

Referring to Fig. 5, the schematic diagram of the reconstruction for being set which show camera and being generated.The visual field if (FOV) and camera away from Contact between (CPD) with a distance from (CD) and convergence plane is nearly equal to eye distance/observer's distance that holographic display is provided Set (ED/OD), then 1 can be realized:1 reconstruction.

Being somebody's turn to do (virtual) camera (aggregate information is recorded from there) should ideally have with observer just with its eyes sense Feel the same setting.It means that the camera should be placed on being focused at the position of the eyes of display plane.Then, remember The original scene of record and carry out 1:1 rebuilds.In general, the relation between the parameter that camera is set should show with holography The setting of offer is almost identical-and CD/CPD should be close to ED/OD (see Fig. 5).The visual field of camera should provide almost identical angle Domain, it is from display plane span to observer's eyes.Only by these limitations are considered, scene can be reconstructed and be counted as " it will truly there ".Otherwise, the scene will appear to be similar to, but with the visual angle or size somewhat changed, rely on It is undesirable in these parameters.If these parameters are different too much from suitable geometry, big visual angle mispairing will be generated.

Can simply consider these in order in real time by computer generate 3D contents limitation because virtual camera here Can freely dispose or medelling is in Virtual Space.Such virtual camera by the view of API that is referred to as being rendered based on 3D and Projection matrix is typically presented.For content (such as computer that real world (3D cameras) or offline computer are generated The animation of making), it should keep above-mentioned limitation firmly in mind, because once generating photo, camera setting can not be changed later.One compromise Be probably from the visual angle in a center automatically generate it is all needed for the views containing color and depth information, it is every other to regard Angle generates therefrom.But the obstruction information for disappearing is due primarily to, mass loss is considered.It is available due to only having a visual angle , important Viewing-angle information, object is just kept off in central view, is disabled in other visual angles.However, embodiment have can The blocking data as a part for data flow can be included.

For the real-time 3D contents generated by computer, the reality before the holographic display device with user's Trancking Software Observer position can be used to appropriately position virtual camera.Regarding riel company with only in the complete of the intraoral information of viewing window In breath 3D displays, the knowledge of eyes coordinates can also be used for virtual camera of the positioning corresponding to observer's position and movement, so There is provided whole viewings scope (effect of " looking around ") of natural scene.It means that when static object is looked around, should Scene seems to be fixed on same position as natural.Except for cosily viewing 3D provides all natural depth of field for a long time Outward, this generates the big viewing areas of identical (hologram of traditional ultrahigh resolution will be provided substantially).Additionally, the feature The distance of virtual camera is appropriately set for eye distance according to the observation, to cause that it is perfect that virtual camera is set.However, this The effect looked around is the feature of voluntary (opening or closing), and it is all satisfactory that it is not directed to all 3D contents.

4.3 transparencies

One interesting effect, is comprising transparent (translucent) depending on the exclusive feature of the holographic process flow of riel company The reconstruction of the scene of object.The transparent substance (such as glass or cigarette) of nature is influenceed from the light on intensity, direction or wavelength The light in source.In nature, two eyes can be focused on behind transparent substance or object, and it may also be transparent substance, and (part is thoroughly It is bright).

Such reconstruction can utilize the scheme realization on holography regarding riel company, and realize in the following manner In display：The multiple sight spots arranged along a different depth for eyes --- display --- light can rebuild simultaneously.This It is meant that in order to the 3D sight spots with different depth and color are in the same position overdetermination position multiple SH of hologram, and allow Eye focus are in the different sight spots in respective depth (see Fig. 6 a).Sight spot in the focal length to the eyes of observer will have been seen It is clear to come, however other back or fronts will be smudgy.Unfortunately, reconstructed 3D sight spots can not realize that physics is special Property, the transparent substance in nature will be carried out using by light wave.Therefore this characteristic must accordingly by control sight spot color and Imitate, so as to realize color filtering or damping and other effects.

In terms of content product, the transparency interaction can be added except existing color and depth by toward each sight spot α values outside value are controlled.If sight spot has 0.0 α values (fully transparent), just without rebuilding it, SH will not be also generated.1.0 α values mean that sight spot is fully opaque, thus to generate the sight spot behind single SH-all will be invisible and do not have SH is generated for it.Mean that sight spot is partially transparent between the α values of 0.0-1.0, the α value values represent the grade of transparency, Therefore for other points of transparent sight spot and back or front, sub-hologram will be generated.

The API of current real-time 3 D romance, such as Direct3D and OpenGL, only provide a depth value per pixel, because Only one color diagram and a depth map are typically used simultaneously, to store rendered scene.When render transparency effect, Typically, multiple paths are by all transparent substances of the blending in their depth orders, relative to it is rendered enter color Remaining object of figure and complete.For the path that these are admixed, the generation of depth value has typically been abandoned.It is every in depth map The ultimate depth value of individual pixel generally corresponds to the pixel behind all transparent substances.Therefore, one kind is developed depending on riel company Scheme, its API technology on the one hand being rendered using the 3D of these high developments, on the other hand to there is transparent substance to generate more Depth value.

Company depending on riel generates multiple 3D sight spots in holographic plan same position, but the principle with different depth is Use based on multiple content-datas layer (see Fig. 6 b).Each layer includes the sight spot with seperate color, depth and α information.This A little layers can be counted as ranked depth layer, wherein every layer includes one or more (with or without transparency) object. The sum of required layer corresponds to the maximum number at the overlapping transparent 3D sight spots in 3D scenes.The program with show for 2D and solid 3D The method of device generation transparency interaction is compatible.On the one hand, the difference be by admix passage result be directed to it is appropriate The cromogram of layer, rather than the existing color of covering.On the other hand, generation depth value store in the depth map of layer rather than Abandon them.

Referring to Fig. 6, it is shown along multiple sight spots of single eyes --- display --- light in Fig. 6 (a) It is continuous reconstructed, and can be used to realize transparency interaction.Fig. 6 (b) shows an exemplary scene, and it has treatment transparent One additional layer (can be more layers) at sight spot.Transparent sight spot is stored in ground floor, and background object is present in background layer In.

Finally, this layer of necessary preprocessed given α value with according to all sight spots and the shadow from sight spot behind other Ring so as to convert the color at all sight spots.Used as the two of example objects, one 100% red and background is opaque, separately One 100% white and translucent (it only damps 50% light to prospect for 50% transparency, α=0.5).After treatment, background is damped Object-its new color is 50% red, and foreground object only 50% is opaque, and so its final color will be 50% white.Work as sight When seeing such reconstruction, when the translucent white object in prospect blocks background object, background object will be more black, but both Can be seen and focus on.

Therefore, last after treatment α values, the data for being moved to hologram synthesis contain multiple views with multilayer, Each includes the sight spot only with color and depth value.Then, SH will only be effective sight spot and generate the-product of only part The transparency layer that pole uses will be processed.

In other words, the reconstruction of transparent substance will be realized by the following method：

Can not the direct computed hologram from the scene comprising transparent substance generated through computer.Transparent substance can be Such as glass, water, mist, grit or its analog.Usually not for transparent substance generates a depth value, but it is only positioned at one Opaque object behind individual or multiple transparent substances.If it is necessary that being built, it is necessary to possess the depth of object with hologram Value.

If hologram is generated for two or more objects, wherein the object is at least partially disposed at such as from a sight The person of examining see it is mutual behind, these objects will be reconstructed and all be at each of which distance (or depth coordinate) place It is visible.Object before other objects does not interfere with the apparent of other objects after the object.However, this for Transparent substance before other objects is different.For example, sent from the object behind cranberry glass disk Only RED sector light may pass through the glass plate.

In one embodiment of the invention, the method includes multiple stages.In the first stage, generate to be reconstructed complete The depth value of three-dimensional scenic, its transparent substance for not including the three-dimensional scenic.In second stage, all transparencies of three-dimensional scenic The depth value of body is generated respectively.The hologram (or it is to be encoded enter holographic display device/spatial light modulator (SLM) hologram number According to) also generated in multiple stages.First, opaque article or opaque object point are by generating the sub-hologram and plus them And process.Secondly, the transparent substance or transparent object point are by generating the sub-hologram and being processed plus them.

For the presentation or reconstruction of reality, controlled by transparent substance or the color value of opaque article of influence passes through Following event handling：The wavelength from opaque article to the light of propagation in the road of observer's eyes is not controlled at, according to position The feature of the transparent substance before the opaque article, in the position of the opaque article, controls or changes the wavelength of the light Or the color from opaque article.These color values through controlling can be generated by computer drawing method.

Together, it contains the opaque article with the color/wavelength data through controlling (if position to the hologram combination Behind transparent substance) object point information/data and the transparent substance in multistage step object point information/number According to.All objects --- transparent and/or opaque article --- are reconstructed in correct depth and with correct color.

In comparison with the visualization for being the three-dimensional scenic for being shown with conventional two dimension or using stereoscopic display, for holographic reconstruction, Extra sight spot can generate in space, wherein extra sight spot can have identical x-y (lateral attitude), but it is different Z coordinate (depth).Or, extra sight spot can generate for example to realize dense scope, such as mist or water by depth.These Extra sight spot is located on the single line of view direction (along the different depth value of the single line) or with lateral attitude, for This does not have object point to be calculated.Because the extra sub-hologram of only one needs to be generated for the extra object point, and should Sub-hologram needs to be added to every other sub-hologram, with generating extra object point to reconstructed three-dimensional scenic.

The method is very effective, and only can be realized with less calculating power.It can be real-time in Modern Graphic plate Or quasi real time carry out, including hologram calculating.

Figure 10 shows the situation of opaque article and transparent substance in actual life/nature.Figure 11 show for The situation of the reconstruction of three-dimensional scenic is calculated/generated according to the present invention.

4.4 holographic video forms

There is two ways and play holographic video：By directly loading and present be previously calculated holography or by plus Carry original sight spot and in real time computed hologram.

The first selection have the advantages that one it is very big：The data of holoframe should not be by the compression method as encoding and decoding Operated, only lossless method is suitable.By very random naturally holographic data, lossless compressiong is significantly dropping Low data area is invalid to obtain the stream (streaming) from hard disk or optical medium, and IP network (IP- is not related to Networks the stream on).

To overcome these, original sight spot of the storage in different views/figure layer described above is proposed with depending on riel.This with Depending on riel holographic real-time calculate be combined can using as H.264 or MPEG-4 state-of-the-art video compression technology (state-of-the-art video-compression technologies), it depends on the bit rate meeting or many for using Or be lost less, but there is provided outstanding compression ratio.For depth information, loss must be controlled by particularly severe, and its is straight The quality that ground connection influence is rebuild.But when high bit rate is selected, even if the very acceptable damage like that with minimal Consumption, about 1:10 compression ratio is possible.

Develop depending on riel and using simply storing the video frame form of all important data in holographic display device The upper video frame rebuild comprising color and transparency.Figure of this flexible form comprising all necessary view and each view Layer, the color of the subframe to store as being positioned in video frame, α values and depth value (referring to Fig. 7).Storage is in xml document (xml-document) or the extra metamessage that is embedded into video container includes the layout and parameter of video frame, holography is regarded Frequency player needs it to generate appropriate hologram.These information for example describe which type subframe be embedded into, they Position and original camera are set, and how to be construed to map them in particular into the warp of the 3D coordinate systems of holographic display device The depth value of storage.

Reference picture 7, which show the typical topology example of video frame.Subframe can freely be put in video frame Put --- single metamessage (meta-information) record provides described information, and which subframe is embedded into and in video Their position in framework.

This method enables to rebuild the 3D color videos with transparency interaction in real time in holography display depending on riel company On device.Metamessage provides all parameters, and player needs these parameters to generate hologram.It also ensures that video is set with camera It is engaged, and confirms the integrality (i.e. depth must be available) of 3D scene informations.

5. step 2：Hologram synthesizes

The hologram that is transformed at multiple sight spot, wherein each sight spot are characterised by color, horizontal position by hologram synthesis Put and depth.The process is respectively done for each view and color elements, when the whole available figure layers of repetition --- to every Individual view and each color elements calculate single hologram.

For each sight spot in available figure layer, sub-hologram SH is calculated and accumulated to hologram H, and it is holographic by each The complex value composition of image element, therefore referred to as hologram cell or unit.Only there is intensity/brightness b (b>0) visible sight spot quilt Conversion, which saves the calculating time, particular for the transparent figure layer being generally only partially filled with.

Assuming that SLM provides the spacing p of horizontal/vertical_x/p_y, the observer's distance between holographic plan and observer is Od, and each sight spot provides its position (ox, oy) (it is also the sub-hologram position in hologram), its depth in figure layer Spend d and af at wavelength lambda brightness b (processing color elements according to current position).

The size of the sub-hologram in display surface (hologram page) is calculated firstThen for inside SH Each cell S H (x, y) at discrete cell position (x, y) place, calculates complex valueAnd by as follows Formula calculates phaseAnd amplitude A：With

Such SH is described with focal lengthHolographic camera lens, rebuild given at d with encoded Sight spot.TermIt it is unique starting stage at each sight spot, it is that typical case is random.In hologram H In the SH that will calculate of SHs positions (ox, oy) place finally amount to hologram H.Position (ox, oy) is the son in hologram H The center of hologram SH, it crosses hologram/display surface to define by light, starts from VW and by space indoor setting The position and the 2D positions corresponding to the sight spot in figure layer of point.Now H represents the wavefront that will rebuild our 3D scenes (wave-front)。

In the next step, from complex value to the conversion of displayable actual value, holographic graph code, it is necessary to which being performed to can Hologram is write into SLM.

6. step 3：Holographic encoding

Coding is to prepare that SLM, the process of the hologram of holographic display device will be written into.SLM generally can not directly show Give instructions in reply value, it means that they can not simultaneously with a single pixel regulation and phase shift light wave.But by combine amplitude adjust and Phase adjusted display, can recognize coherent light wave modulation.Each slm pixel is controlled by the complex value (unit) in hologram Modulation.By SLM of the illumination with coherent light, the wavefront of synthesis scene is generated on hologram page, then it spreads into VW To rebuild scene.

Different types of SLM can be used generation hologram, and some examples are：A complex value is generated using 3 amplitudes⁴ The SLM with only Modulation and Amplitude Modulation (around phase-modulation (detour-phase modulation))；By two phases⁵Xiang Jie The SLM with only Modulation and Amplitude Modulation for closing；Or the combination amplitude that is combined with a phase pixel by an amplitude pixel and phase The SLM of regulation.The latter passes through phase and amplitude plate⁶Interlayer can be identified.

Accordingly, it would be desirable to the subordinate of the SLM types of our holograms, only phase amplitude, phase or the only presentation of amplitude.It is holographic Each unit in figure must be turned into appropriate presentation.After the hologram write-in SLM that will change, each slm pixel leads to Cross it phase and amplitude regulation through light wave.

7. step 4：Post processing

The final step of process chain, for the mixing of color elements and the view different holograms of execution, and by holoframe It is shown to observer.There are different methods on holographic display device and come Show Color and view.

A kind of method will be presentation (total time frequency multiplexing technique (total time- of complete time sequencing multiplexing)).Here whole colors and view even show in succession in multi-user system to different observers.Pass through The layout and currently displaying view of the VW of proper position are synchronously controlled, and is the currently displaying of λ codings by basis Hologram be that λ changes appropriate light source in the correct time, whole observers are it can be seen that the reconstruction of 3D scenes.Other method It is that there are two independent presentations of the view and time frequency multiplexing technique of different SLM, such system is via regarding riel company Showed in SID Long Beach in 2007⁷。

Two kinds of further methods, it is performed in our current prototypes, is to be based on the blend color in a single frame And view, or based on the mixing view in a single framework and continuous Show Color⁸.For both, in vertical direction (VPO) using haplopia difference hologram in, however, multiple with vertical interlacing scan (interlacing) multichannel in a horizontal direction With different views (color).

There is also many other ways and means that color and view are presented for whole observers.In a word, post processor It is responsible for being presented to the holographic frame formatting of observer.

8. implement

Whole process steps described above have been carried out, such as in the hologram reference software systems of riel company Regular software module.It is in ready-made PC hardware and the fortune in the special FPGA platform of riel company for different prototypes Capable GPU and FPGA computers, it is necessary to coding path be optimized.Two platforms are presented following, but first will Provide our 20 feet of explanations of direct view holography prototype.

The 8.1 direct view holography prototypes for regarding riel company

20 inches of direct view prototypes depending on riel company are shown with the full color holography 3D of 1D vertical codes (VPO) Device.It provides view window tracking by with integrated accurate and Rapid Eye tracing system, the accuracy of its use ± 2.5mm with The position of 60 deliverings 3D eyes per second is at most four observers.But this holographic prototype is set only for an observer Meter.Eye tracks software runs in standard PC, or is entirely integrated into the display using embedded DSP/FPGA solutions In.

Holographic panel is had the resolution ratio of 2048 × 7680 Modulation and Amplitude Modulation pixels and is run with 60Hz.Due to (to holography) The relative coarseness pel spacing of ready-made LCD, observer is apart from current setting at about 2 meters.By with around phase code⁴, to one The spacing of individual complicated pixel (unit) is 156 × 156 μm, which results in the corresponding VW of 8mm in vertical direction.In level side Upwards, two holograms of eyes are spatially separated.The optical system of determination is that each eye creates 32.5mm water wide Flat available point, its movement according to the observation is moved in the horizontal direction.Therefore observer can be both horizontally and vertically freely Slip-on head, without missing holographic view.The hologram of images of left and right eyes is entered into the framework of display by spatially multiplexing, when Color is presented serially.For each color elements (red, green, blue), the different frame of appropriate wavelength is shown.

It is possible to rebuild per the sight spot of figure layer 512 × 2560, but at the distance of the observer of 2m, the eyes of people can not Offer an explanation on 20 inch displays 512 × 2560 3D scene resolution ratio.Therefore, 3D scenes resolution ratio is arbitrarily limited to every figure layer 512 × 640 sight spots, this also provides more common forms (3:4).For bigger LCD, 3D scenes resolution ratio can easily expand Greatly to whole HD or higher.The depth bounds of 3D scenes is normally started from before holographic display device at about 1 meter and aobvious Showing can go to infinity behind device.

8.2 real time holography on PC

It is many that the power of holographic display device is driven with PC：The SLM for using DVI can be driven with test pattern plate, its Support the big resolution ratio for needing.Additionally, various existing parts are available, it is continuously improved rapidly.With wide in range API open graphic libraries (3D-rendering APIs OpenGL) and 3D Graphics parallel libraries that set 3D is rendered (Direct3D), the establishment in the real-time 3D contents of Microsft Windows platform is easily carried out.Additionally, being that 3D models and video provide lattice The form of formula and decoder, there is provided useful SDK and software library and it is simple intelligible.

When the holographic calculating of high intensity is carried out with PC, primary processor major part is not powerful enough.Even newest CPU The calculating of the real-time 3D holograms of high-resolution, such as Intel's Duo i7 (Intel Core can not sufficiently rapidly be performed I7 about 50GFlop/s) is realized⁹.Therefore it is obvious with more powerful part, what is most attracted people's attention is graphics processing unit (GPU), due to their huge memory bandwidths and powerful processing power, in spite of some expenses and flexibility is lacked.As Advantage, its programmability, flexibility and disposal ability have been significantly improved in the past few years.

8.3 real time holograms for using GPU

Direct3D 9 based on Microsoft, whole holographic process flows as the aforementioned are with attracting interactive application Implement.All are run with a NVIDIA GeForce 285GTX on PC, drive 20 " the direct view prototypes of holography 3D.Almost All calculate and completed all on graphics processing unit (GPU), CPU is only used for control program flow and offer parameter for calculating. To most step, special pixel and vertex shader program (special pixel-and vertex- is had been carried out shader-programs)¹¹, it is from switching to perform appropriate algorithm length by length.Coloration program (Shader- Program) the small coding fragment write with C speech like sounds (that is, when be Level Shading Language (HLSL) with Direct3D).When to hold When going and being edited and upload to GPU, they running parallel to process in the shader kernel (shader-core) of GPU Summit and fragment/pixel in figure flow.Modern GPU typically has more than 200 shader kernels, and each can be with about The high-frequency of 1GHz performs typical 4 dimensional vector operation.

Relative to other such as CUDA¹²GPGPU technologies (in GPU¹³(graphics processing Unit the general-purpose computations on)), selection directly using Direct3D the reason for be to be provided using GPU with greater flexibility Any feature and to the application program module in real time generation 3D contents simple interface.

Our scheme, using GPU, be very flexibly and parameterisable.There is provided encapsulation holographic calculating depending on riel company All aspects simple API, and Windows-SDK has been developed, so any application designer only needs to concentrate In the content of itself.Be related to the virtual camera for special holographic display device, hologram synthesis, holographic graph code and after The institute of processing routine is busy, is hidden by the software systems or automatically processed.Fig. 8 provides the handling process based on GPU Overview.

For regarding the 20 of riel company " directly the GPU scheme works of view prototype it is as follows：In the first module, content life Into the holographic applications program-view of generation two of-part, each is made up of at most 4 figure layers, each figure layer storage 512 × 640 Sight spot and each sight spot provide color, depth and α information.It is all that the holographic applications program can be provided with Direct3D-API Function and feature, such as coloration program of itself.The data of generation (at most generate 16 textures in GPU memory Data, two of each figure layer provide depth and color and α information) hand over to next example.This is that an application program sets The unique module that meter person must create and implement, every other part is provided by SDK.

Referring to Fig. 8, the overview of data flow in its GPU scheme for showing us.

Next module, i.e. hologram synthesize, the transparency effect for processing each figure layer to be given using α values, Yi Ji Depth between different figure layers performs some depth sortings in the case of classifying not over the application program.Then it is directed to Each 3D sight spot in each figure layer, generates and positions sub-hologram with vertex shader program.Each sub-hologram is by performing The pixel shader treatment of the calculating of the phase and amplitude of foregoing each hologram cell (complex value).Each is through meter The unit of calculation is then accumulated on the hologram of the complex value of generation.With two floating-point textures in GPU memory, (one right Substantial portion, one to virtual part) implement the hologram of this complex value.

In our first version of prototype in being presented on SID in 2007, using consult table approach, wherein to each The sub-hologram of 256 discrete depth steps pre-computed standard.It is done at this moment to overcome the limited treatment energy of GPU Power, i.e. NVIDIA 7900GTX.So far, GPU is promptly improved, but at the aspect of computing capability It is more more than being improved on stored memory broadband.The combination of now improved GPU and the optimized algorithm depending on riel company is caused Directly calculating, there is higher quality, more flexible and more efficiently sub-hologram to be possibly realized --- dynamic sub-hologram chi Very little and direct calculating causes more effectively calculating, currently uses the depth resolution (allusion quotation almost without limitation provided by GPU The 24bit of type) preferably depth resolution is provided with middle (especially to big depth bounds) in the reconstruction, and stored memory is wide Band no longer limits calculating.

In the third step of holographic encoding, complex value is encoded creating final result (with around phase-modulation⁴Compatible SLM Present), it is also completed with another set of pixel shader.

Final six holograms (two views, wherein each view have three holograms (being directed to color elements)) be in It is multiplexed on present SLM.Three SLM frames (frame corresponds to each color elements) are by using vertical interlacing scan Two views are multiplexed into a frame for each color elements and are generated.Then these three frames are continuously displayed.

8.4 real time holographies for using FPGA

In the development regarding the holographic 3D schemes of riel company, another step arrives field-programmable gate array for transplanting (port) Arrange (FPGA) and optimize the software.The motivation is to realize being integrated to the autonomous system of any display.For this point, It has been designed using the self-defined FPGA plates of Stratix III FPGA from Altera.Relative to CPU or GPU, FPGA Advantage is both optimal mixing, than concurrency more preferable in CPU and with than the more preferably flexible in programming in GPU.To the greatest extent Pipe causes the longer lead-time in this way, the complicated program model of FPGA schemes compares with the development based on PC.

In the application aspect, two schemes use PC, with foregoing content generating module (content- Creation-module) content is generated.But to this scheme, by a single DVI frame (DVI-frame) To FPGA plates, the FPGA plates are stored the data that two views pack color and depth information to be converted into using DVI frames instead of GPU Texture in device.Therefore Set Top Box or game machine will also be adapted to and be used as content source.Realized by extending DVI frame sizes The operation of more than one transparency figure layer, with most 4 figures including the video frame form similar to suggestion described above The information of layer.

Referring to Fig. 9, its overview for being displayed in data flow in our FPGA schemes.

Sub-hologram, accumulative and coding and post processor module is calculated to be implemented with VHDL.Today, this side Case provides the same function of GPU editions (GPU-version) as regarded riel company, except to the to be supported of figure layer.FPGA Plate LVDS interfaces (LVDS-interface) directly drive SLM frame is presented (referring to Fig. 9).

8.5 applications

As these technical schemes can be used on GPU and FPGA, the full display frame ratio of 60Hz can be used depending on riel company The high-resolution vital movement 3D videos of the lively coding of display, for video decoding, from the PC with standard MPEG-4 decoders Hardware frame by frame flowed.Each frame includes the original face to two views and 4 figure layers (background adds 3 transparency figure layers) Color, depth and α information, it is fully utilized.In order to significant duty factor is compared with more than all available in transparency figure layer Sight spot 80% used.

Additionally, in detailed 3D environment, real-time application has been based on reference format and is made up of multiple models Complex interaction formula 3D scenes are developed and demonstrate the 3D models that display calculates generation.Handed over to strengthen user's experience and simplifying Interaction, has been integrated with mankind's machine interface (as 3d space mouse) and Nintendo the Wii control for being chosen for the modern times Device.

9. result

Shown based on holographic 3D and corresponding owner algorithm the principle regarding riel company, two schemes (GPU and FPGA) being multiplexed with space view and continuous color can utilize the full SLM frequencies driving of 60Hz to regard the 20 of riel company Inch holography 3D prototypes directly perceived.Additionally, already today, the program is scaling up with correspondingly increasing pixel quantity and can drive Display hardware (SLM resolution ratio higher, bigger size).

The high-resolution, full frame ratio GPU schemes are run on the PC with single NVIDIA GTX 285, public depending on riel The FPGA schemes of department use an Altera Stratix III to perform all holographic calculating.Support that complexity 3D scenes and 3D are regarded Transparency effect in frequency.Even to using all 4 complicated high-resolution 3D contents of figure layer, frame ratio is for continuously More than 60Hz.Content can be provided by PC, and especially to FPGA schemes by Set Top Box, game machine etc., it is as driving Move common 2D or 3D displays.

For more or bigger sub-holograms, especially to the sub-hologram for extending in bidimensional (expire parallax hologram), two Person's technical scheme can extend with previously pooling function well.Even if using Current protocols, the reality of full parallax color hologram When to calculate be achievable, and internally test.The complexity of 3D scenes is relied on, according to effective computing capability, There may be some limitations to smaller depth bounds.Already today, by with multiple GPU (NVIDIA SLI or AMD CrossFire) or link multiple FPGA, calculating performance be fully able to simply strengthen.As an example, NVIDIAs SLI Be applied to connection two increases the GeForce 285GTX GPU that hologram calculates the speed of frame, card by 1.9 key element Understand the excellent autgmentability of the program.

These schemes can enable to show the interactive 3D scenes of the 3D videos of complexity and complexity, its whole depending on riel company Holographic ground dynamic coding.Compared to its technology sheet, it allows more to focus on holographic content and application program at present And for game or the exploitation of the appropriate form of holography TV.

10. conclusion and further prospect

Our scheme with new sub-hologram method presented herein, when GPU or FPGA hardware is adapted to, it is permitted Perhaps the real-time calculating of hologram, to drive the holographic display device of the 3D contents of display complexity and interaction.

Further development will concentrate on the new calculating tinter (Compute-Shader) and OpenGL 3/ introduced The new technology in GPU (as Direct3D 11) field of 4 and OpenCL, with improve regarding riel company GPU schemes efficiency and Flexibility.Including supporting that the FPGA schemes depending on riel company of multiple transparency figure layers will be completed in 2010.Further, in order to The development of special holographic ASIC, VHDL designs (VHDL-designs) will be optimized.

Convection current holographic video (3DTV) and 3D contents and depending on riel company Technology Integration is to existing game or applies journey In the engine of sequence, the development and adaptation of suitable format will be as another focuses.

Bibliography

[1]Goodman,J.W.,[Introduction to Fourier Optics],2nd edn,McGraw-Hill, New York(1996).

[2]Hoffman,D.M.,Girshick,A.R.,Akeley,K.&Banks,M.S.,“Vergence– accommodation conflicts hinder visual performance and cause visual fatigue,” J.Vis.8(3),1–30(2008). journalofvision.org/8/3/33/

[3]St-Hilaire,P.,Benton,S.A.,Lucente,M.E.,Sutter,J.D.&Plesniak,W.J., “Advances in holographic video,”Proc.SPIE 1914,pp.188–196(1993).link.aip.org/link/PSI/1914/188/1

[4]Burckhardt,C.B.,“A Simplification of Lee’s Method of Generating Holograms by Computer,”Appl.Opt.9(8),1949–1949(1970). ao.osa.org/ abstract.cfmURI=ao-9-8-1949

[5]Hsueh,C.K.&Sawchuk,A.A.,“Computer-generated double-phase holograms,”Appl.Opt.17(24),3874–3883(1978). ao.osa.org/abstract.cfmURI=ao- 17-24-3874

[6]Gregory,D.A.,Kirsch,J.C.&Tam,E.C.,“Full complex modulation using liquidcrystal televisions,”Appl.Opt.31(2),163–165(1992). ao.osa.org/ abstract.cfmURI=ao-31-2-163

[7]N.Leister,A.Schwerdtner,G.Fütterer,S.Buschbeck,J.-C.Olaya and S.Flon,″Full-color interactive holographic projection system for large 3D scene reconstruction,″Proc.SPIE 6911,69110V(2008). dx.doi.org/10.1117/ 12.761713

[8]R.,Reichelt,S.,Leister,N.,Zschau,E.,Missbach,R.& Schwerdtner,A.,“Large real-time holographic displays:from prototypes to a consumer product,”Proc.SPIE 7237,p.72370S(2009). link.aip.org/link/PSI/7237/ 72370S/1

[9]www.intel.com/support/processors/sb/cs-023143.htm

[10]www.business-sites.philips.com/3dsolutions/home/index.page

[11]en.wikipedia.org/wiki/Shader

[12]www.nvidia.com/object/cuda_home_new.html

[13]gpgpu.org/about

Claims (13)

1. a kind of method of the spatial light modulator calculating holographic coding value for being the holophotal system for 3 D scene rebuilding, three-dimensional Scene includes the object with transparent performance and at least one observer's eyes are rebuild, and methods described is included：

--- three-dimensional scenic is divided into single object point and the coordinate of these object points is determined；

--- determine imaginary observer position, the position corresponds to the selected obvious observer's eyes for perceiving three-dimensional scenic Position；

--- it is determined that the completely covered property that will be encoded of other object points do not seen from imaginary observer position Point；

--- by the distance of all visible object point to be encoded to imaginary observer position, will be in same angle from virtual observation The visible point all to be encoded that person sees position is classified；

--- the intrinsic brilliance of each visible object point is determined, in the angle that they are seen from imaginary observer position, it is considered to that All true and virtual light source position and the intensity of the scene at the position of a little object points, true and virtual optical is included wherein being contemplated that The physical attribute of the object in source；

It is characterized in that：

--- for the visible object point of each to be encoded, it is contemplated that it physical location intrinsic brilliance, to imaginary observer position The transparency property of the distance put and all objects between visible object point and imaginary observer position or object point, it is determined that By its object point of imaginary observer's location aware apparent brightness value；

--- for the visible object point of each to be encoded, determine at least one apparent brightness value, the apparent brightness value is depended on extremely A few wavelength, is perceived by object point described in the wavelength at imaginary observer position；

--- each object point is separately encoded with its respective apparent brightness value, to be passed to the apparent brightness value in holophotal system In rebuild as much as possible at the position corresponding to visible point so that opaque materials can be independently perceived at each of which position Point and transparent object point, and

--- change at least one object point determined by the three-dimensional scenic so that at least one observer generation described three The different views of scene are tieed up, wherein, the three-dimensional scenic depends on imaginary observer position.

2. method according to claim 1, it is characterised in that once change at least one thing of the three-dimensional scenic Point, just recalculates the intrinsic brilliance value and apparent brightness value of at least one object point immediately.

3. method according to claim 1, it is characterised in that at least one table of at least one wavelength will be depended on See brightness value and be defined as apparent colour value, it is contemplated that actual color value of the object point at its position and positioned at the visible object point With all objects or the transparency property of object point between imaginary observer position, by the apparent colour value, the object point exists It is perceived at the imaginary observer position.

4. method according to claim 1, it is characterised in that due to reflection, alleviate or scattering can be generated virtually Additional virtual object point is calculated, and it is at least one virtual or real light source light and the three-dimensional scenic that will be rebuild The result of the interaction between the object point of object, wherein the optical properties and virtual or real light sources of object can be considered, and For those as the visible virtual object point of visible true object point calculates the apparent brightness value at the position of imaginary observer, Yi Jiyong Their encoded radio is calculated as sub-hologram by their own apparent value respectively, by the apparent brightness value, the object point It is perceived at the imaginary observer position.

5. method according to claim 1, it is characterised in that due to being positioned at imaginary observer position and thing to be modified Object between point or the optical properties at object point for example reflect, alleviate or scatter, and the position of object point is relative to imaginary observer Position is corrected, and the position corrected for those and calculate apparent brightness value, and for such visible object point, with respective Apparent value difference calculation code value.

6. method according to claim 1, it is characterised in that by considering to be located between object point and imaginary observer position Object or object point the distribution of space transparency, calculate the apparent brightness value of visible object point, wherein their spectral correlation performance It is considered, by the apparent brightness value, the object point is perceived at the imaginary observer position.

7. method according to claim 1, it is characterised in that in random device and positioned at object point and imaginary observer position Between transparent and reflection and diffusion and the absorption of at least one of diffraction and the object or object point of refraction, scattering, reflect and With the help of the probability of diffraction, the apparent brightness value of visible object point is calculated, wherein their spectral correlations are considered, by institute Apparent brightness value is stated, the object is perceived at the imaginary observer position.

8. method according to claim 1, it is characterised in that in order to improve or reducing the observability of the object point, with The form amplified or weaken is in the calculating of the apparent brightness value of single visible object point, it is considered to positioned at object point and imaginary observer position At least one transparency property of at least one single object between putting.

9. method according to claim 1, it is characterised in that also comprising the encoded radio calculated for sound-reproducing system, The sound-reproducing system includes at least two sound generating apparatus, and at least two sound generating apparatus are to contain three-dimensional scenic Room and time different sound values three-dimensional sound field reconstruction, the three-dimensional scenic include with acoustic absorption attribute thing Body and be at least one hearer's ear rebuild, wherein, the calculating of the encoded radio of the second playback system is included：

--- three-dimensional scenic is divided into single object point, single object point can influence sound, and the coordinate for determining these object points；

--- it is determined that the position of virtual hearer, it is corresponding with the position for being chosen for hearer's ear, at the position acoustically Substantially perceive three-dimensional scenic；

--- in the direction of the position of virtual hearer, it is determined that not by owning that other object points for fully absorbing sound are fully covered Object point；

--- all object points of the locality for carrying out self-virtualizing hearer will be located at, the distance of the position of virtual hearer is arrived by them Classification；

--- in the angle from those object points of the location aware of virtual hearer, it is considered to all realities of scene at the position of the object point The position and intensity of border and virtual sound source, wherein consider the physical attribute of the object comprising actual and virtual sound source, it is determined that in energy Actual loudness, pitch and sound delivery time at the position of each object point of influence sound；

--- for each object point that can influence sound, by the actual loudness at the position of the object point for considering that sound can be influenceed, Pitch and sound delivery time, the institute to the distance of the position of virtual hearer and between the position of object point and virtual hearer The absorption properties for having object or object point determine apparent loudness, pitch and sound delivery time, the sound by the apparent loudness, Pitch and sound delivery time are perceived at the position of virtual hearer；And

--- calculate includes each sound value of this loudness, pitch and sound delivery time respectively, in this way, in the position of virtual hearer Place is put, the reconstruction with the sound-reproducing system can be perceived with this apparent loudness value, pitch and sound delivery time.

10. the space optics optical modulator calculation code value for 3 D scene rebuilding is used for, or for the sound of holophotal system is produced The computing device of device calculation code value, at least one observer's eyes or at least one hearer's ear, three-dimensional scenic bag The object with transparent attribute is included, wherein computing device includes at least the one of the method being configured and adapted in perform claim requirement 1 Individual programmable processor core.

11. computing devices according to claim 10, it is characterised in that further comprising at least one of the following：Appoint At least one programmable logic device and at least one application specific integrated circuit of what type, and at least one Digital Signal Processing Device；Device as wherein at least two can be combined in an integrated circuit.

12. computing devices according to claim 10, it is characterised in that this computing device is for three-dimensional scenic to be presented A part for the system controller of display.

13. computing devices according to claim 10, it is characterised in that this computing device is can be based on three-dimensional scenic A part for the general-purpose computing system of calculation.