WO2012140397A2 - Système d'affichage tridimensionnel - Google Patents

Système d'affichage tridimensionnel Download PDF

Info

Publication number
WO2012140397A2
WO2012140397A2 PCT/GB2012/000334 GB2012000334W WO2012140397A2 WO 2012140397 A2 WO2012140397 A2 WO 2012140397A2 GB 2012000334 W GB2012000334 W GB 2012000334W WO 2012140397 A2 WO2012140397 A2 WO 2012140397A2
Authority
WO
WIPO (PCT)
Prior art keywords
viewpoint
pixel
display
image
data
Prior art date
Application number
PCT/GB2012/000334
Other languages
English (en)
Other versions
WO2012140397A3 (fr
Inventor
Malcolm Mccormick
Matthew Charles Forman
Original Assignee
News Plus Media Technologies Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by News Plus Media Technologies Ltd filed Critical News Plus Media Technologies Ltd
Publication of WO2012140397A2 publication Critical patent/WO2012140397A2/fr
Publication of WO2012140397A3 publication Critical patent/WO2012140397A3/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/376—Image reproducers using viewer tracking for tracking left-right translational head movements, i.e. lateral movements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
    • H04N13/305—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using lenticular lenses, e.g. arrangements of cylindrical lenses
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/373—Image reproducers using viewer tracking for tracking forward-backward translational head movements, i.e. longitudinal movements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/366—Image reproducers using viewer tracking
    • H04N13/38—Image reproducers using viewer tracking for tracking vertical translational head movements

Definitions

  • the invention relates to methods, and associated apparatus, for rendering and displaying images in three-dimensional display systems, including autostereoscopic displays.
  • the invention relates to rendering three-dimensional images for printing, and/or displaying on an electronic display.
  • the invention further relates to a method of adjusting the centre of parallax of an autostereoscopic display.
  • Another aspect of the invention relates to a method, and associated apparatus and system, for providing a display of a three-dimensional scene.
  • the invention further relates to a method, and associated system, of generating a multi-viewpoint representation of a three-dimensional scene, as well as to an associated non-transitory computer-readable medium encoding a data structure.
  • 3D display techniques For visualising three-dimensional (3D) information a variety of display techniques exist. Different three-dimensional (3D) display techniques rely on stimulating different groups of depth cues within the human visual system. One way to create a 3D effect is by presenting different views of a 3D scene to the viewer's left and right eyes, stimulating the binocular disparity depth cue to produce a stereoscopic display. Stereoscopic display provides the perception of depth, while the display remains two-dimensional. Perception of depth is achieved by combining visual information for two or more viewpoints. This requires a means of ensuring that visual information associated with a particular viewpoint is only seen from that viewpoint. Autostereoscopic displays function without the requirement of glasses or head gear for the viewer. Instead, the display includes a means of decoding the combined visual information.
  • a decoding element such as a lenticular array ensures that different images with the correct spatial relationships are seen by each eye.
  • Different views of the 3D scene may be provided by multiple separate views encoded into an interlaced (composite) image, with the decoding element ensuring that one particular image is seen by the viewer depending on the viewer's viewpoint.
  • a more efficient viewpoint encoding method is necessary.
  • An alternative display technique uses a conventional (monoscopic) two-dimensional display but creates a 3D impression on the viewer by updating a 2D rendering of a 3D scene based on the user's changing viewpoint, employing motion disparity as a depth cue.
  • a device input where the input is representative of the viewpoint from which the viewer is observing the display.
  • the viewer may tilt the device and in response the display shows a different view of the 3D entity or scene.
  • the update is sufficiently smooth and sensitive to the viewer's changing viewpoint, then an impression of three-dimensionality can be created.
  • the user can interact with the 3D scene in an intuitive manner to inspect different views of the scene simply by tilting the device.
  • a source of 3D data is re-rendered in real time for each change in viewpoint to obtain the respective image to display.
  • Real-time rendering is subject to a trade-off between complexity of the 3D scene that can be rendered, resolution of the rendered image, and responsiveness of the display system.
  • the demand for real-time rendering of complex scenes with high resolution and good responsiveness may pose a computational challenge, particularly for mobile devices (with typically relatively low computational power).
  • the display may suffer a lag between user interaction and response. This may impede intuitive recognition of the 3D nature of the scene and detract from the overall usability of the display unit.
  • the present invention seeks to alleviate some of the above problems.
  • a method of rendering images for autostereoscopic display comprising: generating a pixel value for each pixel of an image, each pixel being mapped to a single viewpoint, such that an integer number of pixel values are generated for at least one element of a decoder array; storing each generated pixel value in a buffer matrix; resampling the pixel values stored in the buffer matrix in dependence on the number of display pixels for each element of the decoder array; and displaying the resampled pixel values.
  • the integer number of pixel values is chosen to be greater than the number of display pixels for the at least one element of the decoder array. More preferably, the number of display pixels is rounded up to the nearest integer.
  • the number of display pixels is rounded up to an integer number 1 , 2, 3, 4, or 5 more than the number of display pixels.
  • the error associated with resampling may be reduced.
  • the view point is mapped utilising ray tracing. More preferably, the pixel value is generated by integrating along the traced ray.
  • the integration scheme utilised may be one of: emission absorption model; maximum intensity projection; and isosurface detection.
  • a ray direction vector is computed directly, and in one arithmetic operation, from the view point for the each display pixel.
  • the ray is traced from a point a distance in front of the display to a point a distance behind the display, such that 3D object space extends in front of and behind the display surface.
  • a masking operation is utilised to determine unused pixels. More preferably, the masking operation uses data volume convex hull projection. More preferably, the pixel values are not generated for the unused pixels.
  • the method further comprises determining an intensity distribution from the generated pixel values, wherein the resampling utilises said intensity distribution to determine said resampled pixel values.
  • the intensity distribution may be determined utilising linear interpolation, or a fitted polynomial curve.
  • the method further comprises performing a transformation on the generated pixels before resampling.
  • the transformation may be at least one of: linear scaling; finear shift; translation; and other affine.
  • the resampling is performed in dependence on at least one physical parameter of the decoder array.
  • the at least one physical parameter may be at least one of: temperature; stress; and deformation.
  • the transformation is performed in dependence on at least one physical parameter of the decoder array.
  • the at least one physical parameter may be at least one of: temperature; stress; and deformation.
  • the resampling is performed in dependence on at least one environmental parameter.
  • the at least one environmental parameter may be at least one of: humidity; and temperature.
  • the transformation is performed in dependence on at least one environmental parameter.
  • the at least one environmental parameter may be at least one of: humidity; and temperature.
  • the decoder is a micro-optical array comprising a plurality of lenslet elements or a parallax barrier.
  • the plurality of lenslet elements may be substantially cylindrical or rotationally symmetrical.
  • the resampling utilises a graphical processing unit.
  • the resampled pixels are printed and/or displayed electronically.
  • images are rendered for autostereoscopic display by:
  • the multiple is 4* ⁇ 3.
  • display data is subjected to affine transformation by transforming data lookup coordinates.
  • a method of image rendering wherein a virtual composite image is constructed from a plurality of views of a three-dimensional entity, and the virtual composite image is resampled to a displayed composite image.
  • the composite image is encoded according to a decoding element of an autostereoscopic display.
  • the virtual composite image may be constructed in dependence of the display decoding element.
  • the decoding element is monitored by a decoding element monitoring unit, and the virtual composite image is adapted in dependence of changes of the decoding element.
  • the virtual composite image is scaled prior to resampling to display a composite image that is adapted to the scaling of the decoding element.
  • the virtual composite image is scaled prior to resampling to display a composite image that is adapted for viewing at a position different than optical infinity.
  • the viewing distance of a viewer is detected by a viewer detection unit, and the displayed composite image is adapted in dependence of the viewer's viewing distance.
  • the virtual composite image is shifted prior to resampling to display a composite image that is adapted to a particular centre-of-parallax.
  • the autostereoscopic display encodes a one-dimensional, or two- dimensional, distribution of viewpoints.
  • the view point is mapped utilising ray tracing.
  • the pixel value is generated by integrating along the traced ray.
  • Examples of an integration scheme that may be utilised include: emission absorption model; maximum intensity projection; and isosurface detection.
  • a ray direction vector is computed directly, and in one arithmetic operation, from the view point for the each display pixel.
  • the ray is traced from a point a distance in front of the display to a point a distance behind the display, such that 3D object space extends in front of and behind the display surface.
  • the method further comprises determining unused pixels utilising a masking operation.
  • the masking operation uses data volume convex hull projection.
  • the pixel values are not generated for the unused pixels.
  • the method further comprises determining an intensity distribution from the generated pixel values, wherein the resampling utilises said intensity distribution to determine said resampled pixel values.
  • said intensity distribution is determined utilising at least one of: linear interpolation; and polynomial curve fitting.
  • the method further comprises performing a transformation on the generated pixels before resampling.
  • Said transformation may for example be at least one of: linear scaling; linear shift; translation; and other affine.
  • said transformation is performed in dependence on at least one physical parameter of the decoder array.
  • resampling is performed in dependence on at least one physical parameter of the decoder array.
  • the at least one physical parameter is at least one of; temperature; stress; and deformation.
  • transformation is performed in dependence on at least one environmental parameter.
  • Preferably resampling is performed in dependence on at least one environmental parameter.
  • the at least one environmental parameter may be at least one of: humidity; and temperature.
  • the decoder is a micro-optical array comprising a plurality of lenslet elements or a parallax barrier.
  • the plurality of lenslet elements are substantially cylindrical or rotationally symmetrical.
  • the resampling utilises a graphical processing unit.
  • the method further comprises: computing the viewpoint position from the stored pixel position for the decoder array; calculating a ray direction from the viewpoint through the stored pixel; moving back along the ray vector from the stored pixel by a single data volume side length; casting the ray through the volume for a multiple of volume side lengths, in a stepped manner, to accumulate the pixel colour value; and adjusting the image and viewing distance parameters for display.
  • the multiple is 4 * ⁇ 3.
  • a method of image rendering comprising: a virtual composite image composed of discrete pixels is constructed from a plurality of discrete views of a three-dimensional entity, the virtual composite image is interpolated to approximate a continuous composite image that represents continuous views of the three-dimensional entity, and the continuous composite image is resampled to a display composite image that is displayed by means of an autostereoscopic display.
  • the virtual composite image is constructed from a plurality of views of a three- dimensional entity, and the virtual composite image is mapped to a physical display, whereby the mapping scales and resamp!es the virtual display image to display pixels.
  • the resampling accounts for a non-integer pixel-to-decoding element ratio.
  • the scaling accounts for viewing distance for the display.
  • the scaling accounts for the horizontal/vertical centre-of-parallax in the display.
  • the scaling accounts for dynamic changes of the decoding element.
  • a method of providing a display of a three-dimensional scene comprising: receiving an input representative of a given viewpoint; and, in dependence on the input, displaying an image providing a view of the three-dimensional scene corresponding to the given viewpoint, the displaying comprising: accessing data defining a viewpoint distribution representing a plurality of views of the three-dimensional scene; sampling the viewpoint distribution in dependence on the input to generate a view of the three-dimensional scene corresponding to the given viewpoint; and displaying an image based on the generated view.
  • viewpoint distribution preferably refers to a distribution of data values representative of a plurality (or more specifically a range) of views of the three- dimensional scene, each view corresponding to a particular viewpoint.
  • the data defining the viewpoint distribution may comprise samples of the distribution (in which case sampling the distribution may comprise a resampling operation).
  • sampling the distribution may comprise a resampling operation.
  • the data defining the viewpoint distribution thus provides an encoding of multiple views of the three- dimensional scene, and such data is accordingly also referred to herein as a "multi- viewpoint representation".
  • An alternative term also used herein to describe such data is "composite viewpoint representation data" since the data can represent multiple viewpoints simultaneously in a composite form.
  • the data defining the viewpoint representation may be generated by a variety of rendering techniques, as will be discussed in more detail below.
  • a display image By sampling the viewpoint distribution (for instance by resampling data samples of a multi-viewpoint representation) a display image can be generated without rendering the three-dimensional scene from scratch. This may reduce the processing time required to generate the display image and permit a quicker, more responsive display that allows real-time viewer interaction and a consistent level of usability. In particular, the processing time is dependent on the sampling operation but independent of any rendering processes required to generate the data defining the viewpoint distribution. Consequently scene rendering complexity can be decoupled from the performance of the display device, allowing great flexibility in application of industry standard image origination, processing and compositing processes in the content pre-rendering stage to produce highly realistic results.
  • the generated view is preferably an image, and the displayed image may simply be the generated view, or alternatively some post-processing may be performed in relation to the generated view to produce the displayed image.
  • the method comprises continuously updating the displayed image in response to changes in the input representative of the given viewpoint.
  • This continuous updating of the display can generate an impression of three-dimensionality on the viewer.
  • scene complexity is decoupled from the performance of the display device, complex scenes with high resolution can be displayed with good responsiveness. This assists intuitive recognition of the three-dimensional nature of the scene and promotes the overall usability of the display unit.
  • sampling the viewpoint distribution comprises convolving the viewpoint distribution with a sampling function.
  • the sampling step comprises sampling the viewpoint distribution in dependence on one or more viewpoint parameters determined based on the input.
  • the one or more viewpoint parameters preferably define the viewpoint corresponding to which a view is to be generated.
  • the one or more viewpoint parameters comprise one or more of: viewing angle information; viewing distance information; and viewpoint coordinate information. Use of an input that is representative of a viewpoint, such as viewing angle information, viewing distance information, or viewpoint coordinate information, assists intuitive recognition of the three-dimensional nature of the scene.
  • the viewpoint distribution represents a range of continuously varying views of the three-dimensional scene.
  • the distribution is preferably defined by a finite set of samples. Based on the defined distribution, it is then preferably possible (typically by resampling e.g. by interpolation) to generate an arbitrary view within the range of the defined distribution.
  • the viewpoint distribution may represent a one-dimensional range of viewpoints (e.g. corresponding to a left-to-right shift in the viewer's viewpoint).
  • the viewpoint distribution may represent a two-dimensional range of viewpoints (e.g. corresponding to viewpoint shifts in both horizontal and vertical dimensions).
  • the viewpoints may be distributed on a substantially cylindrical surface surrounding (or part enclosing) the three-dimensional scene, or on some other predetermined viewpoint plane/surface.
  • the viewpoint distribution may represent a three-dimensional distribution of viewpoints (e.g. combining a range of viewing distances with horizontal and vertical ranges).
  • changes in viewing distance may be accommodated by scaling, as described in more detail below.
  • the viewpoint distribution preferably comprises a distribution of image data values.
  • the data comprises data samples defining the viewpoint distribution.
  • the data samples preferably comprise image data samples and the sampling step may then comprise resampling the image data samples.
  • the image data samples may include samples for each of a plurality of image pixels and/or for each of a plurality of pixel components (e.g. for individual colour components of the pixels).
  • the data may in effect define a plurality of image data distributions, each distribution defining the variation of a particular pixel or pixel component in the target image (i.e. the image to be generated).
  • the plurality of image data distributions may then together define the viewpoint distribution.
  • the sampling operation may sample each of these distributions, typically by resampling data for each pixel or pixel component to obtain the corresponding target pixel (component) value for the view to be generated.
  • the sampling of the viewpoint distribution may be understood as a sampling of individual distributions of pixel (component) values for an array of target pixels, with the resulting array of pixel samples forming the target view.
  • the image data samples preferably correspond to a plurality of pre-rendered views of the scene.
  • the data preferably comprises a plurality of pre-rendered views of the scene.
  • the views may be interlaced to form a composite, interlaced image.
  • the sampling step comprises performing an interpolation based on at least two of the pre-rendered views.
  • the method comprises: generating a source pixel array from the data; generating a destination pixel array based on the source pixel array, the generating step including performing a resampling operation in relation to the source pixel array; and generating an output image based on the destination pixel array.
  • the sampling is performed using a texture mapping engine, preferably forming part of a graphics processing unit (GPU). This may allow fast processing using standard tools.
  • a texture mapping engine preferably forming part of a graphics processing unit (GPU). This may allow fast processing using standard tools.
  • the input comprises one or more of: device orientation or motion information; viewer position or motion information; eye position information; and eye movement information.
  • the method is performed at a device having an image sensor, and receiving an input comprises: receiving image information from the image sensor; and analysing the image information to determine in relation to a viewer of the device one or more of: viewer position information; viewer motion information; eye position information; and eye movement information.
  • the analysis may, for example, include face or eye detection and/or face or eye tracking. Alternatively some other marker may be detected and/or tracked.
  • the input comprises information relating to the distance between a display and a viewer, the method further comprising scaling the sampled data in dependence upon the input. The scaling may be part of the sampling function, or a separate step performed on the output of the sampling function.
  • the method comprises generating a plurality of images corresponding to respective different viewpoints based on the viewpoint distribution, and outputting the plurality of images.
  • the plurality of images may be displayed using a three-dimensional display system, wherein the plurality of images comprises first and second images for provision respectively to left and right eyes of a viewer.
  • the three-dimensional display system may be an autostereoscopic display. Respective separate viewpoints may be determined for the left and right eyes based on the input.
  • multi-viewpoint representation preferably refers to a data entity (e.g. data structure) which comprises data representing or defining a plurality (or range) of views of a three-dimensional scene (each view associated with a particular viewpoint), and from which a plurality of such views can be reconstructed.
  • data entity e.g. data structure
  • a continuous viewpoint structure By projecting the three-dimensional scene through at least one virtual encoding element onto a sampling plane, a continuous viewpoint structure may be generated. With this method, no discrete, representative views of the scene are selected; instead, the complete 3D scene information can be included in the multi-viewpoint representation. Therefore the parallax behaviour of the replayed scene can be smoother.
  • the calculating step comprises, for each of a plurality of scene elements of the three-dimensional scene, calculating a projection of the scene element onto the sampling plane through the virtual encoding element.
  • the scene elements may for example be polygons or voxels.
  • the projection is calculated taking account of possible obscuration by other scene elements, for example, to ensure that only non- obscured parts of scene elements contribute to the projection.
  • the sampling comprises determining a contribution of the projected scene element (or non-obscured part thereof) to one or more pixel samples.
  • the sampling step comprises accumulating pixel sample values based on pixel sample contributions determined for each of a plurality of scene elements.
  • the method comprises performing the calculating step for each of a plurality of virtual encoding elements.
  • the sampling step preferably comprises accumulating pixel sample values based on pixel sample contributions determined for each of the virtual encoding elements (and preferably for each scene element projected through each encoding element).
  • Each virtual encoding element may be associated with a projection field in the sampling plane, and the projection of scene elements through a virtual encoding element may contribute only to pixel samples within its associated projection field (referred to as "clamping" in the detailed description given below).
  • the projection is calculated based on optical characteristics of the one or more virtual encoding elements.
  • virtual encoding element preferably refers to a virtual representation of an encoding element (e.g. as a mathematical construct or operation, or an encoding function), which may be analogous to a physical encoding element that could be used to encode (or decode) optical information.
  • calculating a projection through a virtual encoding element preferably involves calculating a geometric projection of a scene or scene element onto a projection plane in dependence on one or more encoding parameters or encoding characteristics (the encoding parameters or encoding characteristics being determined by the virtual encoding element).
  • the one or more virtual encoding elements may comprise one or more virtual lenslets. Alternatively, the one or more virtual encoding elements may comprise one or more virtual parallax barriers.
  • a non-transitory computer-readable medium encoding a data structure storing a multi-viewpoint representation of a three-dimensional scene, wherein the multi-viewpoint representation comprises data defining a viewpoint distribution representing a range of views of the three-dimensional scene.
  • Encoding a data structure storing a multi-viewpoint representation of a three-dimensional scene assists the processing required to generate a viewpoint-dependent display image. This may result in quicker processing and faster display speed.
  • the viewpoint distribution comprises a distribution of image data values.
  • the multi-viewpoint representation comprises image data samples defining the viewpoint distribution.
  • the image data samples preferably correspond to a plurality of pre-rendered views of the scene .
  • the multi-viewpoint representation is arranged to enable generation of arbitrary views of the three-dimensional scene within the range of views represented by the viewpoint distribution.
  • the multi-viewpoint representation is compressed.
  • the data structure further comprising metadata relating to the multi-viewpoint representation.
  • the metadata may comprise at least one of: autostereoscopic lenslet width; autostereoscopic lenslet radius of curvature; autostereoscopic lenslet thickness; number of viewing window samples; viewing window definition; and compression parameters.
  • a system for providing an image of a three-dimensional scene comprising: means for receiving an input representative of a given viewpoint; means for, in dependence on the input, outputting an image providing a view of the three-dimensional scene corresponding to the given viewpoint, the outputting means comprising: means for accessing data defining a viewpoint distribution representing a plurality of views of the three-dimensional scene; means for sampling the viewpoint distribution in dependence on the input to generate a view of the three-dimensional scene corresponding to the given viewpoint; and means for outputting an image based on the generated view.
  • the system continuously updates the output image in response to changes in the input representative of the given viewpoint.
  • the means for sampling the viewpoint distribution comprises means for convolving the viewpoint distribution with a sampling function.
  • the means for sampling comprise means for sampling the viewpoint distribution in dependence on one or more viewpoint parameters determined based on the input.
  • the one or more viewpoint parameters preferably define the viewpoint corresponding to which a view is to be generated.
  • the one or more viewpoint parameters comprise one or more of: viewing angle information; viewing distance information; and viewpoint coordinate information. Use of an input that is representative of a viewpoint, such as viewing angle information, viewing distance information, or viewpoint coordinate information, assists intuitive recognition of the three-dimensional nature of the scene.
  • the viewpoint distribution represents a range of continuously varying views of the three-dimensional scene.
  • the distribution is preferably defined by a finite set of samples. Based on the defined distribution, it is then preferably possible (typically by resampling e.g. by interpolation) to generate an arbitrary view within the range of the defined distribution.
  • the viewpoint distribution may represent a one-dimensional range of viewpoints (e.g. corresponding to a left-to-right shift in the viewer's viewpoint). Alternatively, the viewpoint distribution may represent a two-dimensional range of viewpoints (e.g. corresponding to viewpoint shifts in both horizontal and vertical dimensions).
  • the viewpoint distribution preferably comprises a distribution of image data values.
  • the data comprises data samples defining the viewpoint distribution.
  • the data samples preferably comprise image data samples and the sampling step may then comprise resampling the image data samples.
  • the image data samples may include samples for each of a plurality of image pixels and/or for each of a plurality of pixel components (e.g. for individual colour components of the pixels).
  • the input comprises one or more of: device orientation or motion information; viewer position or motion information; eye position information; and eye movement information.
  • a system for generating a multi-viewpoint representation of a three-dimensional scene comprising: means for calculating a two-dimensional projection of the three-dimensional scene through at least one virtual encoding element onto a sampling plane; and means for sampling the projection in the sampling plane to generate sampled pixel data; wherein the composite multi-viewpoint representation comprises the sampled pixel data.
  • a method of providing a display of a three-dimensional scene comprising: receiving an input representative of a given viewpoint; in dependence on the input, displaying an image providing a view of the three-dimensional scene corresponding to the given viewpoint, the displaying comprising: accessing data comprising a plurality of views of the three- dimensional scene; convolving the data with a sampling function to generate a view image from the given viewpoint; and displaying an image based on the generated view image.
  • a method of providing a display of a three-dimensional scene comprising: receiving an input representative of a given viewpoint; in dependence on the input, displaying an image providing a view of the three-dimensional scene corresponding to the given viewpoint, the displaying comprising: accessing image data comprising a plurality of views of the three-dimensional scene; interpolating between at least two of the plurality of views to derive an interpolated view; and displaying an image based on the interpolated view.
  • the invention also provides a computer program and a computer program product for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein, and a computer readable medium having stored thereon a program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein.
  • the invention also provides a signal embodying a computer program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein, a method of transmitting such a signal, and a computer product having an operating system which supports a computer program for carrying out any of the methods described herein and/or for embodying any of the apparatus features described herein.
  • Any apparatus feature as described herein may also be provided as a method feature, and vice versa.
  • means plus function features may be expressed alternatively in terms of their corresponding structure, such as a suitably programmed processor and associated memory.
  • any feature in one aspect of the invention may be applied to other aspects of the invention, in any appropriate combination.
  • method aspects may be applied to apparatus aspects, and vice versa.
  • any, some and/or all features in one aspect can be applied to any, some and/or all features in any other aspect, in any appropriate combination. It should also be appreciated that particular combinations of the various features described and defined in any aspects of the invention can be implemented and/or supplied and/or used independently.
  • Figure 1 shows the basic principles that underlie autostereoscopic displays
  • Figure 2 shows a pixel array with a lenticular lens decoder array
  • Figure 3 shows a lenticular lens decoder array with cylindrical lenses
  • Figure 4 shows a lenticular lens decoder array with rotationally symmetrical lenses
  • Figure 5 shows volume ray-casting
  • Figure 6 shows a realistic display with irregular pixel-viewpoint association
  • Figure 7 shows resampling of a virtual composite image to a displayed composite image
  • Figure 8 shows a linear scaling transformation applied to the virtual composite image prior to resampling it to a displayed composite image for accommodation of proximity viewpoints, for an ideal display
  • Figure 9 shows a linear shifting transformation applied to the virtual composite image prior to resampling it to a displayed composite image for accommodation of centre-of- parallax, for an ideal display
  • Figure 10 shows implementation of a volume ray-casting algorithm
  • Figure 11 shows a flow diagram of the ray-casting process
  • Figure 12 shows the use of graphics tools to sample composite viewpoint representation data and generate a display image
  • Figure 13 shows a different particular selection of pixels that relate to a different viewpoint being drawn onto a destination pixel array
  • Figure 14 shows an example where a new image is generated as a combination of source pixel values
  • Figure 15 shows a source quad being sampled onto a destination pixel array
  • Figure 16 shows the relationship between the viewer angle and the mapping of the source quad and destination pixel array relative to one another
  • Figure 17 shows the core scheme of forward mapping
  • Figure 18 shows the core scheme of forward mapping
  • Figure 19 shows tilt compensation in the case of a static parallax barrier
  • Figure 20 shows tilt compensation in the case of a lenslet array
  • Figure 21 shows a File format that facilitates generating a display image
  • Figure 22 shows a process diagram for generating a display image
  • Figure 23 shows a typical graphics pipeline
  • Figure 24 shows components of a device utilising the method of generating an image.
  • FIG. 1 shows the basic principles that underlie autostereoscopic displays.
  • a rectangular array of independently addressable colour square (generally rectangular) pixels 100 is overlaid by a decoding element.
  • the decoding element is a micro-optical array that enables the viewer 104 to see only certain pixels from a given viewing position 106.
  • a lenticular lens decoder array 102 an array of small lenses 108 (lenslets) is formed in a substrate, typically a polymer.
  • the pixel array 100 is coincident with the rear focal distance 10 of the lenses 108, so that, in the ideal case, a focussed spot at the pixel plane 114 projects a parallel beam 200 at a given angle 202. This is shown in more detail in Figure 2.
  • the array of lenses 102 serves to decode the information placed at the pixel plane 114. From a defined viewpoint 106-1, a defined set of pixels 112-1 is seen by the viewer 104. If the viewer 104 moves to a different viewpoint 106-2, a different image composed of a different subset of pixels 112-2 is seen by the viewer 104. This can be employed to generate a volumetric optical model that may be viewed and perceived as a 3D image with full, natural parallax (look-around). The image at the display is thus a composite image that represents a plurality of views of a scene from multiple viewpoints.
  • the lenses may be cylindrical, as in Figure 3, in which case lens action, hence image decoding and parallax, take place only in one dimension.
  • Figure 3 shows a horizontal- only parallax decoder array with vertical cylinder alignment.
  • the lenses may be rotationally symmetrical, as in Figure 4, giving omnidirectional lens action and viewable parallax in all directions.
  • alternative types of decoder arrays such as parallax barrier arrays, as are well known in the art, can be used to generate autostereoscopic displays.
  • volume ray-casting is a popular technique.
  • volumetric data sets include CGI/CAD models and scenes, multiple 2D slice-based (tomograms) or 2D+Depth ap data sets, 3D/4D ultrasound data, and 3D TV/video signals.
  • VRC is an image-orientated scheme that makes use of pixel-parallel computing capabilities of modern off-the-shelf Graphics Processor Units (GPUs).
  • GPUs Graphics Processor Units
  • Volume ray-casting is illustrated in Figure 5.
  • the system casts rays 502 from a (virtual) viewpoint 106 in front of the display 100, through each display surface pixel 504, and then through to the (virtual) location of a volumetric data set 500.
  • the ray 502 is stepped through the data set 500, integrating contributions to the colour and intensity of the pixel 504 at each step location, until it has exited the far side of the volumetric data set.
  • Many integration schemes can be used to control the final pixel colour, for example the emission/absorption model which takes into account voxel opacity to enable translucent rendering, the maximum intensity projection as commonly used in X ray applications, or isosurface detection. Other integration schemes are possible too.
  • each pixel has a viewpoint associated with it.
  • Figures 1 and 2 an idealised situation is illustrated, whereby the lenslet-pixel ratio is an integer, and each lenslet edge aligns perfectly with a pixel edge.
  • Figure 6 A more realistic situation is illustrated in Figure 6.
  • the pixel-viewpoint association is more complex than for the idealised situation 610. Pixels are not always in the same relative positions within their respective lenslet. Neither VRC nor any other viewpoint-dependent rendering techniques can be applied directly.
  • the calculated VRC intensity distribution across the pixels of a display is only a discrete sample of a full-information integral image that contains an angular continuum of viewpoints. Therefore, it is possible to calculate the intensity distribution for a virtual, idealised lenslet-pixel association, and then resample the obtained intensity distribution to obtain an intensity distribution suitable for the real, non-ideal lenslet-pixel association 720.
  • This concept is illustrated in Figure 7.
  • a virtual, idealised lenslet-pixel association 710 is assumed.
  • the real decoding array lenslet configuration is known (from a product datasheet, for instance, or from measurement) and is chosen for the virtual, idealised lenslet-pixel association 710.
  • the actual, non-integer number of pixels appearing behind each lenslet on the decoding array is rounded up to the nearest integer, producing a viewpoint count.
  • the non-integer number of pixels is rounded up to an integer number 1, 2, 3, 4, 5 or more, more than the non-integer number of display pixels behind each lenslet element of the decoder array.
  • the actual lenslet-pixel ratio is 1 :4.3
  • the viewpoint count is 5.
  • the VRC procedure is applied to generate a virtual composite image for the virtual, idealised lenslet-pixel association 710, with an intensity value 700 for each virtual pixel 702.
  • the intensity distribution 704 is taken as a sample of the continuum intensity distribution 706 for infinite viewpoints and infinite pixels.
  • the continuum intensity distribution 706 can be approximated by linear interpolation of the discrete intensity values 700 of the virtual composite image, or it can be approximated by other models such as a fitted curve. Finally, the intensity distribution 714 for the real pixels 712 is found from the continuum intensity distribution 706.
  • a smoother intensity distribution pixel value distribution
  • more virtual pixels do not improve final image quality. If more display pixels are available, quality is improved by using more render pixels (and hence more viewpoint samples).
  • the volume ray-casting is implemented using the pixel-parallel computing capabilities of a Graphics Processor Unit (GPU).
  • GPU Graphics Processor Unit
  • the real-time version executes entirely in GPU hardware. If real-time display is not required, for instance for printing applications, a CPU-only or a CPU/GPU hybrid implementation may be used.
  • the GPU's texture manipulation facilities the virtual image is mapped internally to a rectangular primitive (a 'quad') which is drawn to display memory.
  • the mapped quad is then scaled in real time using the GPU's geometric transformation facilities. This results in a filtered resampling using built-in interpolation facilities of the GPU's texture engine.
  • the display image is now correctly aligned with the autostereoscopic display decoder lenslets. Because it uses techniques for the execution of which the GPU is highly optimised, this results in a miniscule and negligible performance penalty.
  • the image rendering procedure as described is equally applicable to parallax barrier arrays as to the lenticular lens decoder array.
  • barrier slits replace the decoder lenslets, with all the other features regarding alignment and resampling remaining same.
  • Other decoding elements may perform the function of associating different pixels with different viewpoints.
  • a lenticular decoding array or other uni-directionally selective means can be used with a horizontal group of viewpoints to provide a reduced information integral image with horizontal parallax only.
  • a square- or hexagonally-based full parallax microlens array can be used as a decoder. With a two dimensional matrix of origin viewpoints this provides full omni-directional parallax in the display, so that all-around (rather than just horizontal) look-around is possible.
  • rays may be cast separately from each colour component sub-pixel. Additionally, multiple rays can be cast from each pixel and integrated, for a still greater viewpoint resolution gain, resulting in smoother look-around (effectively a form of oversampling).
  • Figure 8 illustrates how, in the description so far, the viewpoints have been assumed to be at a great distance from the display, at optical infinity 106-1 , defined in that the viewing direction from a viewpoint is considered parallel for all pixels of the display. If however the viewpoints 106-2 are closer to the display, the pixel group relating to a lenslet must be slightly wider than the lenslet itself for the viewing rays to converge correctly. This can be accommodated with ease in the procedure of resampling the virtual composite image to a displayed composite image.
  • a linear scaling transformation 802 is applied to the virtual composite image prior to resampling it to a displayed composite image. The linear scaling of the virtual intensity distribution allows the displayed composite image to be slightly wider than for infinity viewpoints.
  • Figure 8 illustrates an ideal lenslet-pixel alignment with display pixels perfectly aligned with decoder elements, analogous to the idealised situation 610 in Figure 6.
  • the ideal lenslet-pixel alignment is for illustration purposes only, and it is obvious that the concept of scaling the virtual intensity distribution for non-infinity viewpoints is equally applicable to a realistic lenslet-pixel alignment with a non-integer number of pixels appearing behind each lenslet.
  • a virtual idealised lenslet-pixel association is used to generate the virtual intensity distribution which is scaled before being resampled to the real lenslet-pixel alignment.
  • the above procedure of correcting the displayed composite image for a particular viewing distance can be adapted in response to a viewer positioning himself in front of the display. Detection of an actual viewer position can be implemented for instance by a webcam, as is frequently built into displays. In this manner the optimum viewing distance of the display is adapted in dependence of an actual viewer position, in realtime.
  • the configuration of display and decoder material may shift. For instance thermal expansion, physical stress, any external forces applied to the decoder could cause the decoder material to expand and shift.
  • environmental parameters such as humidity and temperature, can also cause the decoder material to expand and shift; for example, the decoder array may expand/contract due to changing humidity. This can be accommodated in the procedure of resampling the virtual composite image to a displayed composite image. A linear scaling transformation is applied to the virtual composite image prior to resampling it to a displayed composite image. In this manner, automatic compensation for shifts of the configuration of display and decoder material may be implemented.
  • FIG. 9 illustrates how manipulation of the horizontal/vertical centre-of-parallax (centre of angular viewing zone) in the display can be accommodated in the procedure of resampling the virtual composite image to a displayed composite image.
  • the centre-of- parallax is normally in the physical centre line of the display 854, but there are advantages in installation flexibility in certain applications by being able to re-centre the display 556. This then gives the viewer the impression that he is looking at the central view of the scene, even if he is actually not looking at the display from a central position.
  • the centre-of-parallax may be adapted if a fixed obstacle is positioned in the normal viewing zone of the display.
  • the 3D monitor may be positioned for a fixed off-centre viewing.
  • adjustment of the centre-of-parallax can be obtained by applying a linear shift transformation to the virtual composite image prior to resampling it to a displayed composite image.
  • Figure 0 illustrates that, in general, the ray start point 904 can be in front of the virtual encoder array/render pixel surface arrangement 710, resulting in content appearing in front of the 3D display unit screen plane.
  • the ray start point 904 can be in front of the virtual encoder array/render pixel surface arrangement 710, resulting in content appearing in front of the 3D display unit screen plane.
  • approximately 1/3 of the data volume (the object space) is in front of the display, and 2/3 is behind the display.
  • Standard ray casting schemes assume that the object space is completely behind the display screen plane (there is no in-front/behind concept in a 2D display).
  • the algorithm described above is executed for every pixel on the render pixel surface, although some of these may not actually have any contribution whatsoever from the data volume since the associated viewpoint rays completely miss it. In these cases the full stepped ray cast will still be performed, wasting computational resources and thus reducing rendering performance. To prevent this, the system may create a mask from the data volume content boundaries, which can be used to determine whether or not a render surface pixel should be processed (and hence result in a ray cast), thus improving efficiency by avoiding redundant ray trace processing completely.
  • the method consists of the following steps:
  • a complementary lenslet encoder array is modelled in the rendering process. Rays are traced from positions on a continuum of possible viewpoints, through the modelled encoder array and a memory-based (virtual) render pixel surface. While being traced they impinge on a data volume that contains the three dimensional information distribution to be rendered.
  • the final displayed, classified representation of the data volume may be subject to affine transformations (rotation, translation, scaling) to allow the viewer to manipulate the view. It is normal in 3D computer graphics for this to be achieved by transformation of vertex coordinates within the world space. However, in the system the virtual encoder array and viewpoints must remain static and untransformed as they exist in viewer space. Normally in 3D computer graphics, object and viewer space are equivalent but in opposite sense of one another.
  • the first step 1 02 is to compute the viewpoint origin position 106 (in fixed viewer space) according to the (virtual) render pixel number.
  • the current operation pixel 908 from the render pixel surface is stored in RenderPixellndices.
  • the number of pixels mapping to one lenslet in the modelled encoder array is stored as enderPixelsPerLensletHoriz (horizontally) by RenderPixelsPerLensletvert (vertically, only applicable if an omnidirectional parallax display is used, with for instance a square-based microlens array or two perpendicular cylindrical lens arrays).
  • the viewpoint origin position is computed that corresponds to the exact centre of the render pixel. For horizontal-only parallax (e.g. using a cylindrical lenslet decoder) this is:
  • the next step 1104 in the ray casting process is computing the ray direction vector 902 (in fixed viewer space) according to the viewpoint origin 106, and then computing the transformed stepped direction vector for tracing.
  • the screen plane ray position vector, RayScreenPos is proportional to the current render pixel index across the screen. This is the position, in fixed viewer space, at which the viewpoint ray crosses the render pixel surface.
  • the RayScreenPos vector the normalised ray direction vector (in viewer space) is computed:
  • RayDirection RayScreenPos - ViewpointPos
  • a stepped direction vector is generated to enable ray advancement through the data volume of a certain StepSize. This is also subjected to any necessary view transformations via matrix T v o to place it in data volume space:
  • the next step 1106 is to compute the ray start point 904 (in fixed viewer space) in front of data volume 900.
  • the ray start point can be in front the virtual encoder array/render pixel surface arrangement.
  • the ray start point 904 is computed so that it is guaranteed to be in front of the (possibly transformed) data volume 900, by travelling back from RayScreenPos by the correct proportion of the total ray cast length according to the depth of the data volume that is to appear in front of the plane of the final 3D display.
  • ProplnFront is the proportion of the transformed data volume displayed in front of the screen plane (commonly 0.5 for half in front and half behind, or 0.33 for 1/3 in front and 2/3 behind);
  • avCastLen is the total length of the ray cast through the data volume, and is typically 4* ⁇ 3 for a cubic, Cartesian data volume. Then, the ray start point in viewer space is:
  • RayStartPoint RayScreenPos - (ProplnFront ⁇ RayCastLen) ⁇ RayDirection .
  • a nested loop procedure commences to trace the ray to the rear of the data volume in stepped manner. The following steps are looped until the full ray cast length has been covered.
  • the first step 1110 in the ray-tracing sub-loop 1 08 is to look up data volume value(s) using transformed lookup coordinates:
  • OpacityValue Opacity_data_texture_lookup(RayLookupPos D ataspace)
  • a value classification step 1112 may be performed. This step may also be a combined classification operation to determine the contribution to current render pixel.
  • Each of the data values contained in the current value vector can contribute both colour and opacity (alpha) components to the current render pixel colour and opacity.
  • colour (or opacity) contributions are looked up for the data values in a colour (respectively opacity) table and assigned to the destination vector.
  • ClassifiedColour.rgb Datatype1_colour_map_lookup( Datatypel Value) +
  • ClassifiedColour.a Datatype1_ opacity_map_lookup( Datatypel Value) +
  • Soft thresholding may be applied to the opacity component.
  • a transfer function is used with a linear rather than step transition. It is parameterised by the data values at the lower and upper limits of the linear transition region.
  • An example implementation is the GLSL 'smoothstepO' function.
  • ClassifiedColour may be determined in many ways according to the needs of the application and type of data being rendered, using techniques known in the computer graphics field.
  • the next step 1114 in the ray-tracing sub-loop 1108 is to accumulate the current render pixel colour.
  • the emissive classified colour r,g,b components determined above are premultiplied by the associated opacity value a, and the result used to contribute to the final accumulated render pixel colour:
  • FinalColour.a FinalColour.a + (1 -FinalColour.a ).
  • ClassifiedColour .a The next step 11 8 in the ray-tracing sub-loop 1 08 is to step to next data lookup position.
  • the lookup position in transformed data space is advanced by adding the partial, transformed (stepped) direction vector to the current position:
  • RayLookupPosoataspace RayLookupPosoataspace + RayDirDeltaoataspace-
  • the ray-tracing sub-loop 08 is now repeated by returning to first step 1110 looking up data volume value(s) for the updated lookup position.
  • Embodiments of the present invention provide a display technique for 3D data (entities or scenes), in which a different image is shown to the viewer in dependence on the viewer's viewpoint, as determined by some input (e.g. a tilt sensor or camera tracking).
  • 3D data entities or scenes
  • 2D images of the 3D scene are generated and displayed on a conventional 2D-display.
  • the display is continuously updated as the viewer's viewpoint changes to present a different view of the 3D scene.
  • the continuous updating of the scene produces an impression of three-dimensionality on the user, and the user can inspect different aspects of the scene by changing his viewpoint.
  • the present approach is based on a pre-generated multi-viewpoint representation of the 3D scene.
  • the multi-viewpoint representation encodes multiple views (or viewpoint images) of the 3D scene corresponding to a defined range of viewpoints. Once generated, the multi-viewpoint representation may be "replayed" to produce arbitrary views of the scene within the encoded range of views without the need to render the views from scratch.
  • the range of viewpoint images may be encoded in area sampled composite viewpoint representation data, where the samples are representative of a continuum of viewpoints.
  • the composite viewpoint representation data thus encodes a continuum of viewpoint images. This continuum of viewpoint images is also referred to herein as a viewpoint distribution.
  • the viewpoint distribution may be made up of distributions of pixel values for a plurality of target replayed angular image segments which are then interrogated by the viewer.
  • the viewpoint distribution may be made up of distributions of pixel values for a plurality of target image pixels.
  • the composite viewpoint representation data may then comprise pixel value samples which define those pixel value distributions.
  • a particular viewpoint image may be generated from the composite viewpoint representation data in dependence upon a viewer's viewpoint and displayed to the viewer by sampling the viewpoint distribution.
  • the sampling of the viewpoint distribution involves resampling (the composite viewpoint representation data with a sampling function.
  • the sampling function can also be understood as a replay function which is used to replay the composite viewpoint representation data to generate a particular view (where the replay function defines the sampling characteristics used to sample the data).
  • the sampling function is analogous to the optical function of a decoding lenslet array in a lenticular display.
  • the sampling function may be defined in relation to the characteristics of a decoding lenslet array and the angle of the desired viewpoint.
  • the sampling of the viewpoint distribution is unlike the technique of morphing. Morphing requires warping of a first image into a second image, usually by means of a mesh with defined control points. The sampling of the viewpoint distribution requires no warping, and is consequently of lower computational complexity than morphing.
  • multi-viewpoint representation composite viewpoint representation data
  • data defining a viewpoint distribution data defining a viewpoint distribution
  • the process required to generate a display image from pre-rendered composite viewpoint representation data by resampling is far less computationally demanding than continuously re-rendering an entire scene in real-time as the viewpoint changes, and therefore allows a more responsive display to be provided without the need for advanced rendering capabilities in the device, whilst still providing high image quality.
  • This in turn allows provision of a more convincing and consistent 3D display to the viewer and hence enhanced usability of the 3D display device.
  • the approach decouples scene complexity completely from display performance, and therefore enables a wider range of 3D media origination processes which can provide richer and more realistic displays than are possible using real time computer graphic techniques.
  • image data may be represented using a wide variety of encoding schemes.
  • Monochrome image data may simply include intensity/brightness values for each pixel.
  • Colour image data typically includes multiple colour components per pixel, which may be encoded using a variety of colour encoding schemes / colour spaces.
  • colour may be encoded using HSL (hue, saturation, lightness) colour components or RGB (red, green, blue) colour components.
  • HSL hue, saturation, lightness
  • RGB red, green, blue
  • a particularly useful and efficient method of implementing the above-mentioned display uses the texture engine of standard graphics tools (e.g. hardware, for instance a Graphics Processing Unit (GPU)) to perform appropriate sampling of the stored composite viewpoint representation data to generate a display image.
  • standard graphics tools e.g. hardware, for instance a Graphics Processing Unit (GPU)
  • GPU Graphics Processing Unit
  • the composite viewpoint representation data 1201 is composed of pixels, each of which is mapped to a respective range of possible viewpoints.
  • each pixel centre maps to one of 5 views; and the composite viewpoint representation data 1201 contains 5 complete view ranges 1202, 1204, 1206, 1208, 1210 of the 3D scene.
  • the pixels of the composite viewpoint representation data 1201 are merely area samples of the continuous viewpoint distribution, there is no need for them to map to regularly spaced viewpoints or indeed to the same viewpoints.
  • the composite viewpoint representation data 1201 is drawn internally onto a rectangular primitive (a 'quad') 1200.
  • This quad is an internal image representation, and is used for internal image processing.
  • the source quad 1200 is mapped (using the rendering engine) onto a destination pixel array 1212.
  • the destination pixel array 1212 is used as the display image 1218 (though post-processing may also be performed).
  • each of the 5 complete views 1202, 1204, 1206, 1208, 1210 has 3 pixels per row.
  • the composite viewpoint representation data 1201 has 15 pixels per row 1203, the source quad 1200 has a mapped pixel array with 15 pixels per row, and the destination pixel array 1212 has 3 pixels per row.
  • the display image 1218 has 3 pixels per row 1207.
  • the pixel values of the destination pixel array are generated in dependence on the values of the pixel array mapped to the source quad.
  • the texture engine samples from the pixel array mapped to the source quad onto the destination pixel array.
  • a particular group of pixels 1206 is selected and drawn onto the destination pixel array 1212.
  • the group of pixels 206 relate to the same viewpoint.
  • a different particular selection of pixels 1204 (relating to a different viewpoint) is drawn onto the destination pixel array 1212.
  • Figures 12 and 13 show how the source quad 200 relates to the lenticular lens decoder array 102 of a lenticular autostereoscopic display.
  • the mapping from the source quad to the destination pixel array 1212 is analogous to the viewer position angle 1220 in an autostereoscopic display.
  • the destination pixel array has the same number of pixels ⁇ in the view decoding direction) as there are lenslets represented in the original composite viewpoint representation data.
  • the lenticular array determines the final image (selected from the source composite viewpoint representation data) that is seen by the viewer.
  • the mapping from the source quad to the destination pixel array 1212 determines the final image seen by the viewer. In this manner the described method can effectively simulate an autostereoscopic display.
  • Figure 14 shows an example where none of the available interlaced images 1202, 204, 1206, 1208, 1210 is drawn onto the destination pixel array 1212. Instead, a new image is generated as a combination of source pixel values.
  • the continuum intensity distribution 706 can be approximated by linear interpolation of the discrete intensity values 700 of the composite viewpoint representation data 1201, or it can be approximated by other models such as a fitted curve. This linear interpolation or fitted curve may also be generated with a sampling function that is associated with the simulation of an autostereoscopic display.
  • the texture engine is configured to sample and interpolate (resample) from the source quad 1200 onto the destination pixel array 1212 with a pre-determined sampling function 1214.
  • the source is selected only within a predetermined sampling width 1216, and according to a predetermined function.
  • the sampling width 1216 is equal to the width of the (lenslet) segments 1222 of the composite viewpoint representation data.
  • the sampling function can be a rectangular function 1214 for linear interpolation.
  • the function can take other shapes to effect non-linear interpolation, for example a triangular function 1218. This might be required for different effective replay lenslet simulations.
  • the intensity values of the two images 1206 and 1208 are combined, each (in this example) with respective proportions of approximately 70% respectively 30% for the rectangular function; or approximately 82% respectively 8% for the triangular function.
  • the continuum intensity distribution 706 (also referred to as a viewpoint distribution) is a distribution of data values representative of a plurality (or more specifically a range) of views of the three-dimensional scene, each view corresponding to a particular viewpoint.
  • the composite viewpoint representation data 1201 is used to represent the continuum intensity distribution 706.
  • the data defining the viewpoint distribution (the composite viewpoint representation data 1201) may comprise samples of the distribution, such as discrete intensity values 700. In this case sampling the distribution may comprise a resampling operation. As shown in Figure 15, the source quad 1200 is sampled and interpolated (resampled) onto the destination pixel array 1212 only within the sampling width 1216.
  • the mapping from the source quad to the destination pixel array determines which image corresponding to a particular view is generated on the destination pixel array.
  • the upper and the lower row in Figure 15 each show a respective configuration for generating different display images.
  • what in a lenticular (or parallax barrier) autostereoscopic display would be the viewing angle of the viewer can be used as an input to manipulate how the source quad 200 is mapped to the destination pixel array 1212, and in doing so determine which image corresponding to a particular view is generated.
  • the mapping can be modified in dependence upon a tilt orientation or other input to achieve a change in viewpoint.
  • the texture engine is optimised for drawing a source quad onto a destination pixel array (and performing the necessary sampling and interpolation) and can do so very efficiently.
  • the pre-rendered composite viewpoint representation data can be processed very quickly to generate a display image.
  • This allows real-time display of images as the viewer angle changes. Because new images are generated from the pre- rendered image by interpolation as a simulation of the optical action of lenslets in an auto-stereoscopic display, the display does not jump from one image to the next, but shows smooth transitions as does the equivalent autostereoscopic display. This allows very natural and convincing representation of a 3D scene.
  • the display method described here does not render the image from the 3D data scene at the time of viewing, but only generates a display image from the (pre-rendered) composite viewpoint representation data. This may reduce the processing time required to generate the display image and permits a quicker, more responsive display that allows real-time viewer interaction and a consistent level of usability.
  • the processing time can be essentially constant, being determined only by the complexity of the sampling operation, and is independent of the way the scene has been produced or the complexity of the scene. Consequently scene complexity is decoupled from the performance of the display, allowing great flexibility in application of industry standard image origination, processing and compositing processes in the content pre-rendering stage to produce highly realistic results.
  • the method of determining the position or mapping of the source quad and the destination pixel array relative to one another relates to the viewer angle as follows and as illustrated in Figure 16, where:
  • P L is the width of the lenslet 1608.
  • the composite viewpoint representation data only represents a pre-determined range of viewpoints.
  • the multiple data associated with each (virtual) lenticule relate to the range of viewpoints.
  • a discontinuity occurs, insomuch as neighbouring segments within the composite viewpoint representation data relate to viewpoints that are at either extreme of the viewpoint range, and are not necessarily closely related to one another.
  • the mapping of the sampling function approaches the end of the viewing range, there may be a provision to prevent the sampling function from moving on into the next lenticule.
  • the sampling function may be redirected to the other extreme segment in the same (virtual) lenticule. In this manner the display flips from the end of the viewing range back to the beginning of the viewing range.
  • the sampling function may be fixed at the last segment in the (virtual) lenticule. In this manner the display continues showing the end of the viewing range even when the viewing angle input exceeds the value that corresponds to the end of the viewing range.
  • a detection unit that is capable of detecting a viewer and determining the viewer's position relative to the detection unit may generate a viewer angle parameter.
  • a video device such as is commonly incorporated in electronic devices and an image recognition unit such as is available in commercial software.
  • eyeball tracking can be used to determine a viewer angle parameter.
  • Eyeball tracking systems that are commercially available can determine eye position or eye movement.
  • tilt detection units are incorporated.
  • the tilt of a device relative to the horizontal plane for example can be used as an indicator of the viewer angle parameter. For example if the user tilts the display away from himself, the images he sees of the 3D data scene are the ones that would be seen if the viewer had moved around the scene.
  • Tilt can be determined by different methods and sensors, including accelerometers, gyroscopes, or front/rear/anywhere facing cameras.
  • handheld electronic devices often incorporate an acceleration detection unit.
  • This type of unit can be used to detect motion of the device, which in turn can be used to determine a viewer angle parameter. For example, if a user moves the device from right to left, the images he sees of the 3D data scene are the ones that would be seen if the viewer had moved from left to right.
  • the viewer angle parameter can also be selected manually by the viewer, for instance with a graphical user interface element such as a slider, or with an input device such as a mouse.
  • a graphical user interface element such as a slider
  • an input device such as a mouse.
  • a whole rectangular area of viewpoints could be considered, and a composite representation of the full two dimensional range of viewpoints produced.
  • two parameters could be used to generate and display an image, for instance an x- and a y- position, or two tilt angles (e.g. one to an x-axis and the other to a y- axis).
  • the mapping in two dimensions would determine which image corresponding to a particular view is generated in the destination pixel array.
  • the generated display image can be adapted for viewpoints nearer or further away from the display.
  • viewpoints are assumed to be at a great distance from the display, at optical infinity defined in that the viewing direction from a viewpoint is considered parallel for all pixels of the display.
  • viewpoint proximity correction can be done analogously.
  • viewpoint proximity correction can be accommodated in the procedure of drawing from source quad to destination pixel array.
  • a linear scaling transformation is applied to either the source quad (increasing the relative size of the source quad) or to the destination pixel array (decreasing the relative size of the destination quad).
  • the procedure of selection and interpolation from source quad to destination pixel array is the same as described above.
  • the above-described procedure of correcting the displayed composite image for a particular viewing distance can be adapted in response to a viewer positioning himself in front of the display. Detection of an actual viewer position can be implemented for instance by a webcam, as is frequently built into displays. In this manner the optimum viewing distance of the display is adapted in dependence of an actual viewer position, in real-time.
  • the composite viewpoint representation data carries the source 3D information as a sampled continuous data distribution comprising intensities/colours. This is mapped to the source quad and decoded into an image for display.
  • viewpoints may be selected arbitrarily, although spacing at regular angular intervals helps maintain consistent display quality.
  • the composite viewpoint representation data, representing a continuum of viewpoint image data can be generated in several ways:
  • It may be rendered from volume or multi-depth data sets using view ray casting 3. It may be generated by projecting a 3D scene through a virtual encoding element
  • the composite viewpoint representation data may be produced from a set of pre- rendered viewpoint images of a 3D data scene. Pre-rendering of such viewpoint images may follow any customary rendering algorithm such as ray tracing, scanline rendering or volume ray-casting, for example.
  • pre-rendered images of a 3D data scene a multitude of images from a multitude of viewpoints may be taken of an actual scene with a camera or other image recording device. The multitude of images may be recorded, for example, with digital cameras, camcorders, or other recording devices.
  • the multitude of pre-rendered viewpoint images may be interlaced to produce composite viewpoint representation data.
  • the width of the interlaced image segments is determined by the lenslet width: the interlaced image segment width is equal to the lenslet width divided by the number of images from different views in the composite viewpoint representation data.
  • the image segments are as narrow as possible, and the mismatch at the image boundary is the smallest possible (without increasing image resolution and pixel numbers).
  • the interpolation required to generate an image for an arbitrary viewer angle input parameter is a better approximation to the ideal, continuous-viewpoint image.
  • the virtual lenslet is then optimally of a width that is equal to the number of viewpoints multiplied by the width of an image segment (optimally: one pixel).
  • the number of viewpoints can be chosen depending on the size limitation preferences of the composite viewpoint representation data (size increases with number of viewpoints), and the level of desired accuracy of the composite viewpoint representation data in its representation of the continuum intensity distribution.
  • the composite viewpoint representation data may be produced by rendering from volume or multi-depth data sets using view ray casting.
  • the composite viewpoint representation data are generated pixel by pixel. For each pixel the following procedure is repeated:
  • determining the viewpoint position from the current pixel (item 1 above) in view ray casting is dependent on the specific applied decoding element.
  • the viewpoints could be defined prior to rendering.
  • a pre-determined virtual decoding element and a predetermined pixel array may be used.
  • An alternative method of generating composite viewpoint representation data is the 'forward mapping' method, wherein the 3D scene is projected through virtual lenslets of a virtual encoding element to generate the composite viewpoint representation data. This will in general generate a sampled data distribution without any inherent regularly spaced viewpoint structure. This is described in more detail as follows.
  • Figures 17 and 18 illustrate the core scheme of forward mapping.
  • a source three dimensional image data structure (with scene data elements 1700) is defined, as it would be in the ray casting method described above. This is typically in the form of a data volume, or more generally any sparse set of image elements 1700 at defined Z depth positions.
  • the method described here makes a direct geometric mapping, through the centre of curvature of every lenslet 102, from the boundaries of the scene data element 700 to corresponding locations along the destination pixel plane 1702. This results in a Tilled line' 1706 existing in continuous space at the destination pixel plane 1702, which is then drawn into the corresponding pixels 1704.
  • the projection of geometry 1800 and 1802 that contributes to the filled line 1706 is shown in solid lines. Note that the mapped edges may in general not correspond with destination pixel 704 centres (which are themselves at arbitrary locations); a sampling process takes place to determine the correct contribution to each pixel 1704 according to coverage and some predefined sampling filter function.
  • the mapping of scene data elements 1700 imposes its own sampling function in scene space.
  • the sampling function is zeroth-order (impulse), but better (smoother) sampling could be done using 1st order (box), or more elaborate functions.
  • a sampling filter function may be imposed on scene data elements 1700 before they are mapped through the lenslets 102.
  • the method of generating composite viewpoint representation data can be considered to be a 'scene data-order' version of the existing 'distribution data-order' process, with sampling then taking place at the distribution surface rather than within the scene data.
  • Figure 17 illustrates that the effect of mapped scene data elements is limited to the field of the lenslet through which they have been mapped. That is, the width of a hit distribution for a lenslet is clamped to the width of the lenslet's field p f 1608.
  • the projection of geometry 1800 and 1802 that is non-contributing to the filled line 1706 is shown in dashed lines.
  • the width of the field 708 is determined by the distance of the source aperture for the whole system, which sets the optimal viewing distance if the composite viewpoint representation data is decoded using an identical lenslet array with no scaling of the distribution.
  • This forward mapping method may inherently store a completely continuous viewpoint structure. This means that the parallax behaviour of the replayed scene can be optimally and perfectly smooth, with no regular discretisation into viewing zones. In this sense the method is comparable to the use of infinite source viewpoints in the reverse ray casting algorithm.
  • the forward mapping method proceeds as follows. First, the receiving pixel array which is to accumulate the mapped data distribution is cleared. Then, for each scene data element, the steps of the method repeated for each simulated encoding lenslet across the array are: 1. Compute the points at the hit surface (behind the lenslet) corresponding to the edges of the scene data element.
  • each lenslet field has width p f :
  • XfLC 0 max ( Xf L (i) , i p f - p f l2 )
  • Hit point locations should also be tested for obscuration by closer scene data elements, and clamped to the appropriate lateral positions computed for the closer scene data elements according to the method of (1) above, if necessary.
  • the accumulated data distribution is scaled and stored ready for subsequent display and 3D replay.
  • the above description includes only one lateral dimension (generally but not necessarily the horizontal), but the scheme is not limited to this and may for example be extended to operate over a two dimensional array of lenslets to generate a distribution encoding omnidirectional parallax.
  • corresponding variables are computed for a second lateral direction y and lenslet number j (usually but not necessarily orthogonal to the first direction x and lenslet number ).
  • the scene data element will have vertical extent h p .
  • All the above described methods for generating composite viewpoint representation data may use source 3D scene data from a variety of sources, including non-optical sources.
  • sources including non-optical sources.
  • 3D medical visualisation data from X-ray, magnetic resonance, computer tomography, or ultrasound; confocal microscopy, infrared imaging, multispectral data, or other sources may be used.
  • the viewpoint-dependent display described above only presents a single image that corresponds to a single viewpoint.
  • the lenticular array determines the final image (generated from the composite viewpoint representation data) that is seen by each eye of the viewer.
  • the sampling function determines the final image seen by both eyes of the viewer. Changing the presented image (corresponding to a different viewpoint) provides a sense of parallax to the viewer, and gives an indication of the three-dimensional nature of the scene.
  • a stereoscopic display panel presents an image to the viewer's right eye, and a different image to the viewer's left eye.
  • the described viewpoint-dependent display method can be extended to accommodate a stereoscopic display, using for instance a display with a built-in static parallax barrier or a lenslet array; or a display that requires stereo- glasses; or any other autostereo- or stereo-display.
  • the method is applied twice, to extract two viewpoint images from the composite viewpoint representation data, and then present them to the stereoscopic display system for onward stereoscopic display to the user using whatever left/right multiplexing means it employs. This is described in more detail in the following section.
  • the basic tilt-dependent display unit described hitherto uses for example a conventional flat panel display component, such that an identical image is provided to each eye of the viewer.
  • the interactive extraction of geometrically correct viewpoints from a stored continuum of such viewpoints according to viewer/display angle leads to effective stimulation of the motion disparity depth cue in the human visual system, and this effect is largely responsible for the highly effective three dimensional capability of the unit.
  • a yet more powerful three dimensional experience is gained if the above basic scheme is combined with a unit which has a physical stereoscopic (or preferably autostereoscopic) display component; this can augment the display system with the binocular disparity depth cue, providing a different image to each eye in the correct spatial relationship.
  • Stereoscopic or non-autostereoscopic: the viewer must wear special apparatus (e.g. glasses) to demultiplex the stereo image pair;
  • special apparatus e.g. glasses
  • the first requirement is that now two viewpoints rather than one are generated from the composite viewpoint representation data; each of the two viewpoints corresponds to a view of the original scene which would be seen by the respective eye of the viewer. Meta-information about the viewing geometry for the stored scene should be stored with the composite viewpoint representation data to enable the display unit to compute the viewpoint angles correctly.
  • Non-autostereoscopic displays have no inherent viewer angle dependence - the same stereo image pair is available wherever the viewer moves within the overall field of view of the display.
  • To support non-autostereoscopic display units in a full parallax interactive display system no special processing or treatment is required beyond that which is needed to provide the left right image multiplex at the display panel component, and would be needed to support a standard, fixed viewpoint stereoscopic image using the same equipment (typically time-multiplexed using active shutter glasses, or spatially multiplexed using polarising filters, using passive (depolarising) glasses).
  • the effect identified above may be compensated by incorporating an extra processing step for use with fixed native autostereoscopic display components.
  • this tilt compensation step the displayed image information is transformed. This may be achieved, for example, with a lateral shift transformation in dependence upon the tilt angle of the unit. With this transformation left and right image pixels remain projected to the correct zones relative to the viewer.
  • Figure 19 illustrates the tilt compensation in the case of a static parallax barrier 1900
  • Figure 20 illustrates tilt compensation in the case of a lenslet array 2000.
  • the display pixel surface 904 displays image information for the viewer's left eye 1906 and the viewer's right eye 1908.
  • the parallax barrier 1900 with the barrier slits 1902 ensures that the correct image is seen by each eye.
  • a pixel data shift 9 2 is performed in the tilt compensation step.
  • the tilt compensation step incurs a negligible performance penalty, as the required image shift can be integrated into the transfer of samples from the source distribution to the final rendered display.
  • the tilt compensation step may occur after the display image is generated.
  • the tilt compensation step may also be integrated into the algorithm that generates the display image. Expressed as a proportion of the total field width Pf associated with a lenslet or barrier slit, the translation s c 1912 required is:
  • the display described above may provide a three dimensional display unit using interactive motion disparity and binocular disparity to produce a highly engaging and realistic display with full parallax features, using commercially available mobile display devices and components.
  • the described concept is based on the recognition of there being a relationship between the discrete images; discrete images provide 'point angular samples' of the space (angular impulse sampling). On the basis of this recognition information on pixels (and hence views) that lie between the discrete images can be deduced. This is distinct from an approach where there is no known (a priori) relationship, and no information on pixels that lie between their source images; in this approach an estimate of the underlying 3D structure must be made in order to generate new viewpoints. To estimate the 3D structure from 2D image pixels it is then necessary to find correspondence, that is, identify points in the 3D scene that might correspond in view images. Finding correspondence is a difficult (and underdetermined) problem, due to occlusion problems, changes in lighting with angle, etc.
  • difficult mapping or estimating of object structure may be required.
  • mesh-based models as an estimate of the 3D structure of the scene may be generated, and then a new view of the model generated.
  • morphing/warping mesh points are manipulated to create new view. While these approaches fundamentally require information regarding the relationship between 3D object space and 2D images from cameras capturing views of it, the concept described here stays entirely within image (pixel) space, and requires no further information regarding the original 3D scene.
  • the concept described here derives an in-between view image pixel using filtered resampling of the angular distribution recorded behind a [virtual] lenslet.
  • angular 3D scene image information is within the Nyquist limit - sampling rate defined by the no. of views).
  • the band-limited continuous intensity function is imposed by the image creation process, and hence fully known, linear resampling of the continuous intensity function produced the required new image.
  • Figure 21 illustrates a file format that facilitates use of the method of generating an image as described above.
  • the composite viewpoint representation data 1201 comprises view data from a continuous viewpoint range, and as such offers a representative sample of a continuous viewpoint representation.
  • the composite viewpoint representation data 1201 is preferably stored in a dedicated file format 2100 that usefully includes further information that is pertinent to the composite viewpoint representation data 1201.
  • compression of the data may be desirable to reduce file sizes.
  • Compression parameters may be included as metadata 2104 to the file format.
  • the dedicated file format described above does not necessarily have to be displayed using a method of generating and displaying an image from the composite viewpoint representation data as described in the forgoing description.
  • the same file format can be used for autostereoscopic displays, for example.
  • the file format can provide data for - for example - printed 3D large-format or high-volume, small-format display panels, specialised visualisation monitors, 3DTV, or other stereo/autostereo 3D display units.
  • Figure 22 shows a process diagram with the main steps undertaken to generate a display image for a viewpoint-dependent display.
  • a display image is generated 2204 in dependence on an input 2202.
  • the generated display image is outputted 2206 to a display 2208.
  • the composite viewpoint representation data 1201 is sampled with a sampling function 2216 in dependence on an input 2202.
  • One way of achieving this is by drawing the composite viewpoint representation data 1201 onto a source quad 22 0.
  • the position (or mapping) of a destination pixel array is determined 2212.
  • the source quad is drawn onto the destination pixel array 2214 according to the sampling function 2216, therewith producing the display image.
  • Figure 23 illustrates a typical graphics pipeline, such as may be used for generating a display image as described above.
  • Quads are primitives in the geometry path 2322
  • the (source) composite viewpoint representation is a pixel array that is mapped to the quad and is stored in the texture memory 2318
  • the destination pixel array is produced as the result of rasterisation 2310.
  • the sampling, or filtering, or interpolation that generates the display image occurs in the process step of rasterisation 2310.
  • Figure 24 shows major components of a device utilising the process described above.
  • the input 2202 is generated from a tilt sensor 2400, a camera 2402, or another source as discussed above.
  • the display image is generated by a processing unit 2404 that has a storage unit 2408 and a unit that is capable of processing graphical data, such as a GPU 2404.
  • the GPU (which may be integrated into a main CPU or may be discrete) comprises a texture engine 2406 that is especially useful for generating and manipulating the source quad and destination pixel array.
  • the invention may alternatively be implemented on a general purpose processor (e.g. in a single processor device) or using any other suitable processing resources of the device.
  • the processing unit may provide further functionality, for instance converting an input into a viewpoint parameter for use in generating the display image.

Abstract

La présente invention se rapporte à des procédés et à un appareil associé, destinés à rendre et à afficher des images dans des systèmes d'affichage tridimensionnel, y compris des affichages autostéréoscopiques. Ledit procédé comprend les étapes consistant à : générer une valeur de pixel pour chaque pixel d'une image, chaque pixel étant mappé vers un seul point de vue, de telle sorte qu'un nombre entier de valeurs de pixels soient générées pour un élément au moins d'un réseau décodeur ; stocker chaque valeur de pixel générée ; rééchantillonner les valeurs de pixels stockées selon le nombre de pixels d'affichage pour chaque élément du réseau décodeur ; et afficher les valeurs de pixels rééchantillonnées. L'invention se rapporte en outre au rendu d'images tridimensionnelles pour une impression et/ou un affichage sur un affichage électronique. L'invention se rapporte en outre à un procédé destiné à régler le centre de parallaxe d'un affichage autostéréoscopique. Un autre aspect de l'invention se rapporte à un procédé et à un appareil et à un système associés, destinés à fournir un affichage d'une scène tridimensionnelle. Ledit procédé comprend les étapes consistant à : recevoir une entrée représentative d'un point de vue donné ; et selon l'entrée, afficher une image qui fournit une vue de la scène tridimensionnelle qui correspond au point de vue donné ; l'étape d'affichage comprend les étapes consistant à accéder à des données qui définissent une distribution de point de vue qui représente une pluralité de vues de la scène tridimensionnelle ; échantillonner la distribution de point de vue selon l'entrée de façon à générer une vue de la scène tridimensionnelle qui correspond au point de vue donné ; et afficher une image sur la base de la vue générée. L'invention se rapporte en outre à un procédé et à un système associé destinés à générer une représentation à plusieurs points de vue d'une scène tridimensionnelle ainsi qu'à un support non temporaire associé pouvant être lu par un ordinateur qui code une structure de données.
PCT/GB2012/000334 2011-04-11 2012-04-11 Système d'affichage tridimensionnel WO2012140397A2 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
GB1106111.6 2011-04-11
GBGB1106111.6A GB201106111D0 (en) 2011-04-11 2011-04-11 Rendering images for autostereoscopic display
GB1114032.4 2011-08-15
GBGB1114032.4A GB201114032D0 (en) 2011-04-11 2011-08-15 Three-dimensional display system

Publications (2)

Publication Number Publication Date
WO2012140397A2 true WO2012140397A2 (fr) 2012-10-18
WO2012140397A3 WO2012140397A3 (fr) 2012-12-27

Family

ID=44122924

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2012/000334 WO2012140397A2 (fr) 2011-04-11 2012-04-11 Système d'affichage tridimensionnel

Country Status (2)

Country Link
GB (2) GB201106111D0 (fr)
WO (1) WO2012140397A2 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014109851A1 (fr) * 2013-01-10 2014-07-17 Qualcomm Incorporated Conversion stéréoscopique avec orientation de vision pour un contenu graphique basé sur un nuanceur
WO2015032636A1 (fr) * 2013-09-03 2015-03-12 Koninklijke Philips N.V. Dispositif d'affichage multivues
KR20160128483A (ko) * 2015-04-28 2016-11-08 광운대학교 산학협력단 Table-top 3D 디스플레이 시스템 및 그 방법
EP3328058A1 (fr) * 2016-11-25 2018-05-30 Samsung Electronics Co., Ltd. Appareil de capture à base multi-lentille et procédé
GB2564386A (en) * 2017-06-30 2019-01-16 Sony Interactive Entertainment Inc Content modification device and method
RU2692432C2 (ru) * 2014-06-19 2019-06-24 Конинклейке Филипс Н.В. Способ и устройство для формирования трехмерного изображения
CN111527749A (zh) * 2017-12-20 2020-08-11 镭亚股份有限公司 交叉渲染多视图摄影机、系统、及方法
WO2020224719A1 (fr) * 2019-05-03 2020-11-12 Carl Zeiss Jena Gmbh Dispositif d'affichage et procédé de représentation simultanée d'images différentes dans au moins deux yeux

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB1106110A (en) 1964-04-20 1968-03-13 Reiners Walter Hydrostatic torque-measuring apparatus

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB191021828A (en) * 1910-09-20 1910-12-15 Egbert Reynolds Dull Moving Picture Toy.
JPH07287761A (ja) * 1994-04-19 1995-10-31 Canon Inc 画像処理装置及び画像処理方法
JP2010122879A (ja) * 2008-11-19 2010-06-03 Sony Ericsson Mobile Communications Ab 端末装置、表示制御方法および表示制御プログラム

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB1106110A (en) 1964-04-20 1968-03-13 Reiners Walter Hydrostatic torque-measuring apparatus

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9538164B2 (en) 2013-01-10 2017-01-03 Qualcomm Incorporated Stereoscopic conversion with viewing orientation for shader based graphics content
WO2014109851A1 (fr) * 2013-01-10 2014-07-17 Qualcomm Incorporated Conversion stéréoscopique avec orientation de vision pour un contenu graphique basé sur un nuanceur
US10061134B2 (en) 2013-09-03 2018-08-28 Koninklijke Philips N.V. Multi-view display device
WO2015032636A1 (fr) * 2013-09-03 2015-03-12 Koninklijke Philips N.V. Dispositif d'affichage multivues
RU2692432C2 (ru) * 2014-06-19 2019-06-24 Конинклейке Филипс Н.В. Способ и устройство для формирования трехмерного изображения
KR20160128483A (ko) * 2015-04-28 2016-11-08 광운대학교 산학협력단 Table-top 3D 디스플레이 시스템 및 그 방법
KR101691292B1 (ko) * 2015-04-28 2016-12-30 광운대학교 산학협력단 Table-top 3D 디스플레이 시스템 및 그 방법
US10187627B2 (en) 2016-11-25 2019-01-22 Samsung Electronics Co., Ltd. Multi-lens based capturing apparatus and method
KR20180059233A (ko) * 2016-11-25 2018-06-04 삼성전자주식회사 멀티 렌즈 기반의 촬영 장치 및 방법
CN108111714A (zh) * 2016-11-25 2018-06-01 三星电子株式会社 基于多透镜的捕获设备和方法
EP3328058A1 (fr) * 2016-11-25 2018-05-30 Samsung Electronics Co., Ltd. Appareil de capture à base multi-lentille et procédé
US10652518B2 (en) 2016-11-25 2020-05-12 Samsung Electronics Co., Ltd. Multi-lens based capturing apparatus and method
CN108111714B (zh) * 2016-11-25 2021-05-28 三星电子株式会社 基于多透镜的捕获设备和方法
US11064183B2 (en) 2016-11-25 2021-07-13 Samsung Electronics Co., Ltd. Multi-lens based capturing apparatus and method
KR102646437B1 (ko) * 2016-11-25 2024-03-11 삼성전자주식회사 멀티 렌즈 기반의 촬영 장치 및 방법
GB2564386A (en) * 2017-06-30 2019-01-16 Sony Interactive Entertainment Inc Content modification device and method
US11037363B2 (en) 2017-06-30 2021-06-15 Sony Interactive Entertainment Inc. Content modification device and method
GB2564386B (en) * 2017-06-30 2021-08-04 Sony Interactive Entertainment Inc Content modification device and method
CN111527749A (zh) * 2017-12-20 2020-08-11 镭亚股份有限公司 交叉渲染多视图摄影机、系统、及方法
WO2020224719A1 (fr) * 2019-05-03 2020-11-12 Carl Zeiss Jena Gmbh Dispositif d'affichage et procédé de représentation simultanée d'images différentes dans au moins deux yeux

Also Published As

Publication number Publication date
GB201106111D0 (en) 2011-05-25
GB201114032D0 (en) 2011-09-28
WO2012140397A3 (fr) 2012-12-27

Similar Documents

Publication Publication Date Title
JP4764305B2 (ja) 立体画像生成装置、方法およびプログラム
KR101612100B1 (ko) 3차원 디스플레이에 대한 렌더링 개선
JP4861309B2 (ja) ２．５ｄグラフィックスをレンダリングするためのゴースト・アーチファクト削減
JP5340952B2 (ja) 三次元投影ディスプレイ
Schmidt et al. Multiviewpoint autostereoscopic dispays from 4D-Vision GmbH
EP1704730B1 (fr) Procede et appareil destines a generer une image stereoscopique
US7643025B2 (en) Method and apparatus for applying stereoscopic imagery to three-dimensionally defined substrates
WO2012140397A2 (fr) Système d'affichage tridimensionnel
JP6517245B2 (ja) 三次元画像を生成するための方法及び機器
US20120182403A1 (en) Stereoscopic imaging
EP3057066A1 (fr) Génération d'une imagerie tridimensionnelle à partir d'une image bidimensionnelle à l'aide d'une carte de profondeur
CN108141578B (zh) 呈现相机
KR101334187B1 (ko) 다시점 렌더링 장치 및 방법
KR20110090958A (ko) 이미지 속성들에 대한 오클루젼 데이터의 생성
US8094148B2 (en) Texture processing apparatus, method and program
AU2018249563B2 (en) System, method and software for producing virtual three dimensional images that appear to project forward of or above an electronic display
CA2540538C (fr) Imagerie stereoscopique
KR100764382B1 (ko) 컴퓨터-생성 집적영상시스템에서의 이미지 매핑장치 및 그방법
CN108450031A (zh) 图像捕获装置
KR101208767B1 (ko) 곡면 투사를 이용한 입체 영상 생성 방법, 장치 및 시스템, 이를 위한 기록 매체
Knorr et al. From 2D-to stereo-to multi-view video
WO2009109804A1 (fr) Procédé et appareil de traitement d'image
KR101567002B1 (ko) 컴퓨터 그래픽스 기반의 스테레오 플로팅 집적 영상생성시스템
KR101425321B1 (ko) 적응형 렌즈 어레이를 구비하는 3차원 집적 영상 디스플레이 시스템 및 적응형 렌즈 어레이에 대한 요소 영상 생성 방법
Thatte et al. Real-World Virtual Reality With Head-Motion Parallax

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12719778

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12719778

Country of ref document: EP

Kind code of ref document: A2