WO2015120032A1 - Réduction des artefacts de transition entre vues dans les affichages automultiscopiques - Google Patents

Réduction des artefacts de transition entre vues dans les affichages automultiscopiques Download PDF

Info

Publication number
WO2015120032A1
WO2015120032A1 PCT/US2015/014434 US2015014434W WO2015120032A1 WO 2015120032 A1 WO2015120032 A1 WO 2015120032A1 US 2015014434 W US2015014434 W US 2015014434W WO 2015120032 A1 WO2015120032 A1 WO 2015120032A1
Authority
WO
WIPO (PCT)
Prior art keywords
image content
view image
view
shearing
views
Prior art date
Application number
PCT/US2015/014434
Other languages
English (en)
Inventor
Piotr K. DIDYK
Song-Pei Du
Frederic Durand
Wojciech Matusik
Original Assignee
Massachusetts Institute Of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/531,548 external-priority patent/US9756316B2/en
Application filed by Massachusetts Institute Of Technology filed Critical Massachusetts Institute Of Technology
Publication of WO2015120032A1 publication Critical patent/WO2015120032A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/349—Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking
    • H04N13/351—Multi-view displays for displaying three or more geometrical viewpoints without viewer tracking for displaying simultaneously
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/302—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays
    • H04N13/31—Image reproducers for viewing without the aid of special glasses, i.e. using autostereoscopic displays using parallax barriers

Definitions

  • Multi-view autostereoscopic (or automultiscopic) displays may provide an immersive, glasses-free three dimensional (3D) experience and therefore have the potential to become the future of television and cinema.
  • Automultiscopic displays may reproduce both binocular and motion parallax cues. Such displays may show a different image depending on a viewer's position and/or direction. This is typically achieved by adding a parallax barrier (see, Ives, F. E., "Parallax Stereogram and Process of Making Same," U.S.
  • Patent 725,567, April 1903 which is incorporated by reference herein in its entirety
  • a lenticular screen see, Lippmann, G., "Epreuves Reversibles Dormant La Sensation Du Relief," Journal of Physics 7, 4, 821-825, November 1908, which is incorporated by reference herein in its entirety
  • Some of the spatial display resolution may be exchanged for angular resolution. This enables glasses-free 3D and provides motion parallax effect.
  • due to the limited angular resolution of such displays they suffer from view transitions, artifacts, and hot-spotting (e.g., image quality may be affected by the viewing position).
  • Some embodiments may include a method, corresponding system, and
  • Some embodiments may include a computer-implemented method that may comprise storing multi-view image content (including but not limited to one or more multi- view images) in an electronic memory.
  • the method may also perform at least one of reducing and/or removing the visibility of one or more artifacts from the multi-view image content by modifying the multi-view image content.
  • the method may modify the multi-view image content based upon at least one of: shearing the multi-view image content globally, shearing the multi-view image content locally, and/or stitching the multi-view image content.
  • the method may provide one or more updated multi-view images with improved visibility as compared with the multi-view image content, at least with respect to the one or more artifacts.
  • shearing (globally and/or locally) of the multi-view image content may be performed in one or more primary domains and/or on one or more light fields (including but not limited to one or more light fields and/or one or more epipolar-plane images or EPIs) that may be associated with the multi-view image content.
  • stitching the multi-view image content may be performed in one or more gradient domains that may be associated with the multi-view image content.
  • the method may further comprise modifying the multi- view image content based upon shearing the multi-view image content globally, shearing the multi-view image content locally, and stitching the multi-view image content.
  • Shearing the multi-view image content globally may include repositioning a plurality of views of the multi-view image content.
  • the depth of one or more scenes of the multi-view image content may be encoded by one or more slopes of one or more lines that may correspond to one or more points in the one or more scenes.
  • a perceived depth may be associated with the one or more slopes of the one or more lines that pass through the intersections of a line corresponding to a given point in the scene and/or the lines
  • the method may reposition the plurality of views of the multi-view image content and/or may adjust the one or more slopes of the multi-view image content at a transition.
  • repositioning the plurality of views of the multi-view image content may include adjusting a slope of the multi-view image content at a transition.
  • the one or more slopes may include one or more depths.
  • the method may include shearing the multi-view image content locally.
  • Shearing the multi-view image content locally may include dividing the multi-view image content into a plurality of portions of the multi-view image content, and/or repositioning a plurality of view of each of the portions of the multi-view image content.
  • the method may include stitching the multi-view image content, including propagating transitions in the multi-view image content into different views of the multi-view image content in different regions.
  • the multi-view image content used by the method may include multi-view frames across a time domain.
  • the method may select a sample of multi-view frames from the time domain.
  • the method may also perform at least one of reducing and/or removing the visibility of the one or more artifacts from the sample of multi-view frames by modifying the multi-view image content.
  • the method may also perform at least one of reducing and/or removing the visibility of the one or more artifacts from non-selected multi-view frames by interpolating changes from the nearest multi-view frames in the time domain.
  • Some embodiments may include a computer-implemented system.
  • the system may include a memory storing multi-view image content (including but not limited to one or more multi-view images).
  • the system may also include an artifact removal module configured to perform at least one of reducing and/or removing the visibility of one or more artifacts from the multi-view image content by modifying the multi-view image content. Modifying the multi-view image content may be based upon at least one of: shearing the multi-view image content globally, shearing the multi-view image content locally, and/or stitching the multi-view image content.
  • the artifact removal module may be further configured, based upon the modification of the multi-view image, to provide one or more updated multi-view images with improved visibility as compared with the multi-view image, at least with respect to the one or more artifacts.
  • shearing (globally and/or locally) of the multi-view image content may be performed in one or more primary domains and/or on one or more light fields (including but not limited to one or more light fields and/or one or more epipolar-plane images or EPIs) that may be associated with the multi-view image content.
  • stitching the multi-view image content may be performed in one or more gradient domains that may be associated with the multi-view image content.
  • the artifact removal module may be further configured to modify the multi-view image content by shearing the multi-view image content globally, shearing the multi-view image content locally, and stitching the multi-view image content.
  • the system may shear the multi-view image content globally including repositioning a plurality of views of the multi-view image content.
  • the depth of one or more scenes of the multi-view image content may be encoded by one or more slopes of one or more lines that may correspond to one or more points in the one or more scenes.
  • a perceived depth may be associated with the one or more slopes of the one or more lines that pass through the intersections of a line corresponding to a given point in the scene and/or the lines
  • the system may reposition the plurality of views of the multi-view image content and/or may adjust the one or more slopes of the multi-view image content at a transition.
  • repositioning the plurality of views of the multi-view image content may include adjusting a slope of the multi-view image content at a transition.
  • the one or more slopes may include one or more depths.
  • shearing the multi-view image content locally may include dividing the multi-view image content into a plurality of portions of the multi-view image content, and repositioning a plurality of views of each of the portions of the multi-view image content.
  • stitching the multi-view image content may include propagating transitions in the multi-view image content into different views of the multi-view image content in different regions.
  • the multi-view image content may include multi-view frames across a time domain.
  • Some embodiments of the system may include a selection module configured to select a sample of multi-view frames from the time domain.
  • the artifact removal module may be configured to perform at least one of reducing and/or removing the visibility of the one or more artifacts from the sample of multi-view frames by modifying the multi- view image content and perform at least one of reducing and/or removing the visibility of the one or more artifacts from non-selected multi-view frames by interpolating changes from the nearest multi-view frames in the time domain.
  • Some embodiments are directed to a non-transitory computer readable medium having stored thereon a sequence of instructions which, when loaded and executed by a processor coupled to an apparatus, causes the apparatus to: store multi-view image content (including but not limited to one or more multi-view images); perform at least one of reducing and/or removing the visibility of one or more artifacts from the multi-view image content by modifying the multi-view image content based upon at least one of shearing the multi-view image content globally, shearing the multi-view image content locally, and stitching the multi-view image content; and provide one or more updated multi-view images, based upon modification of the multi-view image content, with improved visibility as compared with the multi-view image content, at least with respect to the one or more artifacts.
  • multi-view image content including but not limited to one or more multi-view images
  • perform at least one of reducing and/or removing the visibility of one or more artifacts from the multi-view image content by modifying the multi-view image content
  • shearing (globally and/or locally) of the multi-view image content may be performed in one or more primary domains and/or on one or more light fields (including but not limited to one or more light fields and/or one or more epipolar-plane images or EPIs) that may be associated with the multi-view image content.
  • stitching the multi-view image content may be performed in one or more gradient domains that may be associated with the multi-view image content.
  • the instruction may further cause the apparatus to further modify the multi-view image content by shearing the multi-view image content globally, shearing the multi-view image content locally, and stitching the multi-view image content.
  • the instruction may further cause the apparatus to shear the multi-view image content globally including repositioning a plurality of views of the multi-view image content.
  • the depth of one or more scenes of the multi-view image content may be encoded by one or more slopes of one or more lines that may correspond to one or more points in the one or more scenes.
  • a perceived depth may be associated with the one or more slopes of the one or more lines that pass through the intersections of a line corresponding to a given point in the scene and/or the lines corresponding to left-eye and right-eye views.
  • the apparatus may reposition the plurality of views of the multi-view image content and/or may adjust the one or more slopes of the multi-view image content at a transition.
  • the instruction may further cause the apparatus to reposition the plurality of views of the multi-view image content including adjusting a slope of the multi-view image content at a transition.
  • the one or more slopes may include one or more depths.
  • Some embodiments may transform input image data (e.g., input light fields) by modifying the input image data (for non-limiting example, to make it more repetitive). Some embodiments may transform the input image data by global and/or local shearing, and optionally followed by stitching of repeated fragments of the image. The method (and system) of some embodiments may transform the input image data by reducing
  • Some embodiments may also provide functional improvements to the quality of images. Some embodiments may reduce artifacts, including but not limited to
  • Some embodiments provide a functional improvement to display (and/or representation) of images by improving visual quality of input images by global and/or local shearing. Some embodiments may provide a further improvement of image visual quality by stitching the resulting globally and/or locally sheared image. Several experiments and results to follow illustrate that some embodiments may exhibit substantial functional improvements that enhance the visual quality of images.
  • FIG. 1 A is a diagram illustrating a non-limiting example input light field that may be used as an input to some embodiments ("Big Buck Bunny" ⁇ by Blender Foundation).
  • FIG. 1 B is a diagram illustrating performing global and local shearing operations on an input light field of FIG. 1A, according to some embodiments.
  • FIG. 1 C is a diagram illustrating performing overlapping replicas of the light field of FIG. IB to determine an optimal stitching cut, according to some embodiments.
  • FIG. ID is a diagram illustrating stitching along the determined cut of FIG. 1 C in a gradient domain and reconstructing the light field, according to some embodiments.
  • FIG. 2 is a diagram illustrating a four- view stereoscopic display presenting a scene to a user through a parallax barrier, according to some embodiments.
  • FIGs. 3A-3C show a scene (with an embedded screen) presented at a panel through a parallax barrier (FIG. 3 A), a corresponding light field of the scene (FIG. 3B), and a light field produced by the screen (FIG. 3C), according to some embodiments.
  • FIGs. 4A-B illustrate a scene (FIG. 4A) and its corresponding light field (FIG. 4B) in the two-plane parameterization, according to some embodiments.
  • FIG. 5 is a diagram illustrating an example of a stereoscopic image with and without depth reversal, according to some embodiments ("Big Buck Bunny" ⁇ by Blender
  • FIG. 6 is a diagram illustrating a light field produced by an automultiscopic display presenting three objects at different depths with three different stereoscopic viewing locations indicated by pairs of dashed lines, according to some embodiments.
  • FIG. 7 is a diagram illustrating an original light field and a sheared light field, according to some embodiments.
  • FIG. 8 is a diagram illustrating global and local shearing, according to some embodiments.
  • FIG. 9 is a diagram illustrating a corresponding cost function according to some embodiments.
  • FIG. 10 is a diagram illustrating examples of light fields produced by some embodiments.
  • FIG. 1 1 illustrates the process of stitching by overlapping light field copies to find the optimal cut, according to some embodiments ("Big Buck Bunny" ⁇ by Blender
  • FIG. 12 is a diagram illustrating a raw light field and the results of a method employed by some embodiments.
  • FIG. 13 illustrates a comparison of four views generated using a full method according to some embodiments, as compared against four corresponding views where global shear is applied ("Big Buck Bunny" ⁇ by Blender Foundation).
  • FIG. 14 illustrates example images used in user experiments, according to some embodiments ("Big Buck Bunny" ⁇ by Blender Foundation).
  • FIG. 15 is a graph illustrating results of viewer preferences for global shearing compared with viewer preferences for the full method (including global shearing, local shearing, and stitching), according to some embodiments.
  • FIG. 16 is a graph further illustrating results of viewer preferences for global shearing compared with viewer preferences for the full method (including global shearing, local shearing, and stitching), according to some embodiments.
  • FIG. 17 illustrates images that are produced with a resulting depth which contains less depth errors than an original image, according to some embodiments ("Big Buck Bunny" ⁇ by Blender Foundation).
  • FIG. 18 is an illustration of view expansion according to some embodiments.
  • FIG. 19 is a flow diagram illustrating a process employed by some embodiments.
  • FIG. 20 is a block diagram, according to some embodiments of the present invention.
  • FIG. 21 is a detailed block diagram, according to some embodiments of the present invention.
  • FIG. 1 A is a diagram 100 illustrating an example three-dimensional (3D) input light field 102 that may be used as an input to some embodiments of the present invention.
  • the example light field 102 includes one or more segments 104.
  • an input light field 102 may include but is not limited to including, an epipolar-plane image (EPI), e.g. a two-dimensional (2D) slice of a three-dimensional (3D) input light field, and/or an input image frame.
  • EPI epipolar-plane image
  • Some embodiments may include an input light field that is associated with holographic stereography, a holographic stereogram,
  • FIG. IB is a diagram illustrating performing the method (and system) 150 of the present invention including a global (and local) shearing operation on an input light field 112 (e.g., an EPI which is also the segment 104 of FIG. 1A), according to some embodiments.
  • an input light field 112 e.g., an EPI which is also the segment 104 of FIG. 1A
  • some embodiments may then perform local shearing on the global sheared light field output, thereby outputting a globally and locally sheared output 1 14.
  • An advantage of some embodiments is that the output 1 14 may align the scene around the screen plane and provide an improved structural alignment between the first view 1 18 and the last view 1 16 shown on the display, (e.g., or element 1 18, and / formulate, or element 1 16, respectively).
  • the method (and system) 150 of the present invention may overlap replicas of the light field in order to determine an optimal stitching cut 122. Then, as illustrated in FIG. ID, some embodiments may stitch 132 along the cut 138 in a gradient domain and reconstruct the light field 134. According to some embodiments, a portion of the reconstructed light field
  • FIG. 2 is a diagram 200 illustrating a four- view stereoscopic display (e.g., flat screen 204) presenting a scene to a user through a parallax barrier 202 partitioned into views 228 and viewing zones 224.
  • a sequence of images e.g., image scenes 1 (element 252), 2 (element 254), 3 (element 256), 4 (element 258), shown in varying shades in FIG. 2 are shown within a primary field of view 208 to provide an adequate binocular parallax to the user. Beyond the primary field of view 208, the same sequence of images may repeat, forming additional viewing zones 224 (e.g., second field of view 206 and third field of view 210), which may extend the effective field of view, according to some embodiments.
  • additional viewing zones 224 e.g., second field of view 206 and third field of view 210
  • At least one advantage (and/or functional improvement) of such solutions is that they may provide immersive glasses-free 3D for multiple users in front of the screen.
  • such "stereo free-viewing" may be preferable to enable 3D displays to succeed (e.g., considering for non- limiting example, a family watching a 3D television at home).
  • a problem with automultiscopic displays may arise when a viewer's left eye and right eyes fall into different view zones (referring to viewpoint B, or element 230 in FIG. 2).
  • the black pair of eyes 220 entirely within the primary field of view 208 (also referring to viewpoint A, or element 232) and the black pair of eyes 220 entirely within the third field of view 210 may be positioned to view the image with a correct depth.
  • the pair of eyes 222 that crosses the border of the second field of view 206 and primary field of view 208 may be positioned to view a depth reversal and may view artifacts.
  • the "left" eye 222a may be view scene 4 (element 258) and the "right” eye 222b may be viewing scene 1 (element 252), but scene 4 (element 258) is supposed to be to the right of scene 1 (element 252), not to the left of scene 1 (element 252).
  • the view may not continue across the boundary of the field of view.
  • transitions may be considered as an intrinsic defect of multi-view
  • Augmented Reality may be based on hardware extensions, including head-tracking and dynamic parallax barriers.
  • existing solutions may reduce the problem, such existing solutions are suitable only for a small number of viewers (one to three viewers).
  • the additional hardware and the need for real-time processing which may depend on the current viewer's position, may make these existing approaches difficult to implement in commercial devices such as three dimensional televisions (3DTVs).
  • Some embodiments of the present invention include a method (and system) to reduce the visual effect of these transitions.
  • the method may optimize input images in order to improve the perceived quality, including but not limited to, in places where the transitions normally occur.
  • the results of the method for static images and video sequences using both parallax barriers and lenticular sheets may improve the image quality in places where transitions normally occur.
  • a user study e.g., experiment
  • advantages of the optimized content created by some embodiments is shown to follow.
  • some embodiments may include an optimization that does not require knowledge about a viewer's position, which may provide an advantage in that it makes the technique suitable for an arbitrary number of observers. Some embodiments also do not require hardware modifications and may be used as a preprocessing step to displaying an image.
  • a method (and system) employed in some embodiments may be related to light field processing and manipulation techniques and may employ techniques for seamless image and video compositing.
  • multi-view content may include enough degrees of freedom to improve its quality by modifying the displayed views.
  • some embodiments may analyze light fields produced by lenticular and/or parallax-barrier displays.
  • the light fields produced by such screens may have a repetitive structure. This may induce visual artifacts in the form of view discontinuities, depth reversals, and/or excessive disparities when viewing position is not optimal.
  • some embodiments demonstrate that light fields reproduced on automultiscopic displays may include enough degrees of freedom to improve the visual quality of displayed images.
  • Some embodiments may include a method that may modify light fields using global and/or local shears, followed by stitching, in order to improve the continuity of the light fields when displayed on a screen. Some embodiments enhance visual quality significantly, which is demonstrated herein in a series of user experiments with an automultiscopic display as well as lenticular prints.
  • a light field may include a continuous function that represents radiance emitted from a scene, which are described in Levoy, M., and
  • Light Field Rendering in Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, ACM, 31-42, August 1996, which is incorporated by reference herein in its entirety.
  • Light fields may be aliased due to the discrete nature of acquisition and display stages.
  • Several techniques are developed that may correctly reconstruct light fields from recorded data (see, e.g., Isaksen, A., McMillan, L., and Gortler, S, J., "Dynamically Reparameterized Light Fields," in Proceedings of the 27 th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison- Wesley Publishing Co., 297-306, July 2000; and Stewart, J., Yu, J., Gortler, S.
  • Content depth manipulation may further adjust content to a particular device (see e.g., Zwicker, M., Matusik, W., Durand, F., and Pfister, H., "Antialiasing for Automultiscopic 3D Displays," in Proceedings of the 17th Eurographics Conference on Rendering Techniques, Eurographics Association, 73-82, June 2006; Didyk, P., Ritschel, T., Eisemann, E., Myszkowski, K., Seidel, H.-P., and Matusik, W., "A Luminance-Contrast- Aware Disparity Model and Applications," ACM Trans.
  • Content depth manipulation may focus on depth manipulations to achieve an optimal trade-off between blur introduced by interview antialiasing and presented depth.
  • retargeting techniques may change the size of a displayed light field, thereby better adjusting light fields to different screens (see, e.g., Birklbauer, C, and Bimber, O., "Light- Field Retargeting," Wiley Online Library, Computer Graphics Forum, 31 , 295-303, May 2012, which is incorporated by reference in its entirety herein).
  • resolutions in light field reproduction are addressed by techniques (see, Tompkin, J., Heinzle, S. Kautz, J., and Matusik, W., "Content- Adaptive Lenticular Prints," ACM Trans. Graph. 32, 4, 133 : 1-133: 10, July 2013, which is incorporated by reference in its entirety herein) that increase the resolution of lenticular prints by optimizing lenslet arrays based on the input content.
  • lightfields may also provide a great flexibility in the context of stereoscopic content production.
  • Existing techniques Kim, C, Hornung, A., Heinzle, S., Matusik, W., and Gross, M., "Multi-Perspective Stereoscopy from Light Fields," ACM Trans. Graph. 30, 6, 190, December 2011, which is incorporated by reference in its entirety herein
  • the light field produced by an automultiscopic display may preferably be continuous.
  • some embodiments may employ image stitching techniques, (see, e.g., Levin, A., Zomet, A., Peleg, S., and Weiss, Y., "Seamless Image Stitching in the Gradient Domain,” Computer Vision- ECCV, 3024, 377-389, May 2004; Jia, J., and Tang, C.-K., "Image Stitching Using Structure Deformation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 4, 617- 631, April 2008; Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y., "Drag-and-Drop Pasting," ACM Transactions on Graphics (TOG), 25, 631-637, July 2006; and Eisemann, M., Gohlke, D., and Magnor, M., "Edge
  • some embodiments may employ image stitching techniques to light fields which is novel and unique. Creating continuous light fields is also related to work on video textures (see, e.g., Schodl, A., Szeliski, R., Salesin, D. H., and Essa, I., "Video Textures," Annual Conference on Computer Graphics, SIGGRAPH '00, 489-498, July 2000; and Agarwala, A., Zheng, K. C, Pal, C, Agrawala, M., Cohen, M., Curless, B., Salesin, D., and Szeliski, R., "Panoramic Video Textures," ACM Trans. Graph, (TOG), 24, 821-827, July 2005, which are incorporated by reference in their entirety herein), where the goal is to create sequences, which may be played continuously and indefinitely, and video retargeting (see, e.g.,
  • the aforementioned techniques may employ gradient based compositing (see, e.g., Perez, P., Gangnet, M., and Blake, A., "Poisson Image Editing,” ACM Trans. Graph. 22, 3, 313-318, July 2003; and Agarwala, A., "Efficient Gradient-Domain Compositing Using Quadtrees,” ACM Trans. Graph. (TOG), 26, Article No.
  • gradient based compositing see, e.g., Perez, P., Gangnet, M., and Blake, A., "Poisson Image Editing," ACM Trans. Graph. 22, 3, 313-318, July 2003; and Agarwala, A., "Efficient Gradient-Domain Compositing Using Quadtrees," ACM Trans. Graph. (TOG), 26, Article No.
  • a standard auto stereoscopic display may include a regular two- dimensional (2D) panel and an additional component (e.g., a parallax barrier and/or a lenticular screen) that may introduce a view dependence to pixels of images (i.e., only a subset of the pixels may be visible from a particular location). Introducing a view
  • a special mask e.g., a parallax barrier
  • a parallax barrier may be placed atop the screen and may occlude certain regions of the screen depending on the viewing location, referring back viewing locations of elements 222 and 220 in FIG. 2 (see also, e.g., Ives, U.S. Pat. No. 725,567, "Parallax stereogram and process of making the same, which is incorporated by reference in its entirety herein).
  • the special barrier may include a lenticular sheet, which may redirect the light instead of blocking the light as a parallax barrier does (see, e.g., Lippmann "Epreuves reversibles dormant la sensation du relief, Journal of Physics 7, 4, 821-825, November 1908, which is incorporated by reference in its entirety herein).
  • the selectivity introduced by the additional component enables encoding different views that may be visible from the corresponding locations. Additionally, if the views are displayed densely enough (e.g., with a high angular resolution), each eye may receive a different view, which may lead to stereoscopic viewing.
  • such screens may display a small number of views.
  • a high-end automultiscopic display such as the PHILIPS BDL5571VS/00 may provide 28 views. This may enable reproducing a small part of the light field observed in the real world, which may impact the perceived quality of the automultiscopic display. While such a screen may be a parallax display, the same observations may hold for lenticular based systems.
  • a light field is a function that may describe light traversing a scene.
  • a four- dimensional function L may describe a light field produced by automultiscopic displays.
  • the four-dimensional function L may be parameterized using two parallel planes (s, t) and (u, v).
  • Such a parametrization L (s, t, u, v) may correspond to the image value obtained by intersecting a scene with a ray originating from the first plane at the location (s, t) and passing through the second plane at the location ( «, v).
  • epipolar-plane images may be two-dimensional (2D) slices through a three-dimensional (3D) and/or four-dimensional (4D) light field (e.g., the parameters t and v may be constant and/or fixed) and/or may correspond to a stack of one- dimensional (ID) skylines captured from different viewing locations along a horizontal direction.
  • each given point in the scene may correspond to a line that has a slope and/or slant that may encode the depth.
  • FIGs. 3A-3C show a scene 302 presented at a panel 310 through a parallax barrier 312, a corresponding light field 320 of the scene, and a light field 340 produced by the screen.
  • Rays n (element 350) and V2 (element 352) show the relationship between the scene 302 and the light field representation 320.
  • the screen may be embedded into the portion of the scene 302 and aligned with the M-axis 360. Note that the scene light field 320 in FIG.
  • 3B depicts a portion of the scene and may extend further along the s (element 362) and u (element 360) axes.
  • the display may produce a light field 340.
  • the screen may reproduce a portion (e.g., part) of the original light field, which is shown by the dashed lines 370 in light field 320/ As illustrated in FIG. 3C, beyond this range, the screen may create replicas of the light field, which may result in discontinuities 342 and 344 at the border of each replica in the light field 340.
  • FIGs. 3A-3C may more easily show the concept of a light field, by the light field extending further along the s-axis 362 and M-axis 360.
  • FIG. 3 A illustrates the screen embedded in the scene 302.
  • the signal shown in these cones is also repeated at other locations along the s-axis 362, at the second viewing angle 306 and third viewing angle 308, although the signal may not correspond to these locations. This may create repetitions in the light field created by the screen.
  • the boxes 322, 324, 326 (first light field fragment 322, second light field fragment 324, and third light field fragment 326) in FIGs. 3 A-3C demonstrate how a fragment of the original light field 320 is encoded in the panel 310, and how the fragment forms replicas in the screen light field 340.
  • the repetitive structure of this light field 340 may create discontinuities 342 and 344, which may significantly affect the quality of perceived images.
  • FIGs. 4A-B illustrate a scene 402 and its corresponding light field 460 in a two- plane parameterization, according to some embodiments. Images observed on the screen (of the scene) may correspond to linear cuts through the epipolar-plane images. While views located on the -axis 362 may correspond to horizontal lines 452, 454, viewing positions that are removed from the -axis 362 may correspond to slanted lines 456, 458. Slants encode the distance to the scene 402. Therefore, two viewing positions located the same distance away from the scene 402 may result in parallel lines. Please refer to Applicant's Supplementary Video (which is incorporated by reference in its entirety herein, and available on the Internet at people.csail.mit.edu, under the directory "pdidyk," followed by the sub-directory
  • FIG. 4A is a diagram 400 illustrating a scene 402 viewed at various points.
  • FIG. 4B is a diagram 450 illustrating a light field 460 corresponding to the scene 402 viewed at the various points.
  • the images observed on an automultiscopic screen may correspond to a cut through the created light field.
  • the view may be a horizontal cut, as shown by the corresponding horizontal cuts 452, 454, in FIG. 4B.
  • FIGs. 4A-4B also depict a
  • FIG. 5 is a diagram 500 illustrating a stereoscopic image with 502 and without 504 depth reversal 508.
  • FIG. 5 illustrates an EPI (epipolar-plane image) 506 and cuts 510, 512, 520, 522 through the light field of the EPI 506.
  • the depth reversed image 502 may include artifacts that are not present in the image without depth reversal 504.
  • the depth reversed image 502 may correspond to the cut (the two parallel lines 510, 512 that cross the boundary between the view zones 530, 532) through the light field of the EPI 506.
  • the image without depth reversal 504 may correspond to the two parallel black lines 520, 522 that do not cross the boundary between the view zones 530, 532.
  • the repetitive structure of the light field produced by an automultiscopic display may lead to visual artifacts.
  • a view corresponds to a slanted line in the EPI
  • the view may cross several replicas of the original light field. This may create a discontinuity in the perceived image at locations that correspond to the boundaries of the replicas.
  • an observer moves, such artifacts may be increasingly apparent as the observer changes its location.
  • the above-mentioned scenario may also have a significant influence on depth perception.
  • the depth of the scene may be encoded in the slopes of the lines that correspond to the same points in the scene.
  • the perceived depth may be related to the slope of the line that passes through the intersections of the line corresponding to a given point in the scene with lines corresponding to the left-and right-eye view (FIG. 6, case (b), element 604).
  • the perceived depth may be correct.
  • the estimation of the slope may be incorrect.
  • the sign of the slope may change, creating depth reversal or extensive disparities, which may lead to a viewing discomfort (see, e.g., Shibata, T., Kim J., Hoffman, D., and Banks, M., "The Zone of
  • the depth reversal may be observed in the entire image and/or in some parts of the image.
  • element 606 some embodiments may include a stereoscopic image with and without depth reversal.
  • FIG. 6 is a diagram 600 illustrating a light field produced by an automultiscopic display presenting objects at different depths with different stereoscopic viewing locations indicated by pairs of dashed lines 602, 604, 606 (where each line in a given pair of the pairs 602, 604, 606 corresponds to a view for one eye of the user). Insets on the right present close ups at the light field.
  • the solid lines 608, 610, 612, 614, 616, 618, 620 represent slant that may correspond to perceived depth.
  • the boxes 620 and 630 indicate correct 620 and incorrect 630 depth reproduction, respectively. As illustrated in FIG.
  • both eyes may look at the same replica and the depths estimated by the observer may be correct.
  • both eyes may see different replicas for the thick lines 640 and the other thick lines 650. Therefore, the depth estimated by the user, as shown by the black lines 614, 616, 618, may be incorrect, as the depth is reversed.
  • Plane u (element 360) is also depicted in FIG. 6.
  • the thick lines 660 may indicate a perceived slope 610 in which depth estimation is the same for both eyes.
  • Modifying multi-view content may reduce artifacts caused by the discontinuities in a light field produced by an automultiscopic display. Continuity of the light field at transitions may be improved by applying subtle modifications to the input content, which may hide display imperfections. In some embodiments, discontinuities in a light field may be removed if the multi-view content is carefully designed or modified. For non-limiting example, according to some embodiments, a scene may employ a repetitive structure.
  • FIG. 7 is a diagram illustrating an original light field 702 and a sheared light field 706, according to some embodiments that include the method (and system) 150 of the present invention.
  • An original light field 702 may be produced by an automultiscopic display for a scene with a periodic pattern located at a certain depth. However, the original light field 702 may not line up optimally when repeated 704. In some embodiments, by applying a horizontal shear to the original light field 702, a new light field without transitions 706 may be obtained. In addition, in some embodiments, the sheared light field 706 may line up when repeated 708. As illustrated in FIG. 7, for a planar pattern, the discontinuities in the original light field 702 can be removed by applying a horizontal shear 706.
  • FIG. 8 is a diagram illustrating a method (and system) 150 that includes global and local shearing, according to some embodiments.
  • An input light field 802 for global shearing may be globally sheared using the method (and system) described in the present application to produce the output 804 of the global shear.
  • some embodiments may apply the global shearing method (and system) 150 to apply the shear (e.g., adjust the depth and/or change the slope in the primary domain, i.e., the domain of light fields and/or EPIs) by the amount shown in the arrow 810 (but not so limited to that amount) to the input light field 802. Then, some embodiments may apply local shearing to the output 804.
  • Local shearing first divides the output of the global shear 804 (also input to local shear 806) into a grid and then shears each portion of the grid locally by the amount shown in the arrows 812 (but not so limited by that amount) within element 806. As such, local shearing may produce an output 808 of the local shear, which may further remove artifacts and/or transitions.
  • global shear may be defined by one value s, which may encode the amount of shear that is applied to the last view of the light field shown on a screen to match the first view.
  • the method (and system) 150 may apply the shear to the entire 3D light field, and may compute the optimal shear on 2D views using the following formula (Equation 1):
  • the method (and system) may employ a matching function (see, e.g., Mahajan, D., Huang, F.-C, Matusik, W., Ramamoorthi, R., and Belhumeur, P., "Moving Gradients: A Path-Based Method for Plausible Image
  • V7 is a gradient of image / and ⁇ ( ⁇ , x, y) represents the standard deviation in a 9x9 neighborhood of pixel (x, y) in view /.
  • some embodiments may iterate over integer values in the range between s min and s max and choose the value that results in the smallest value of the matching function Q.
  • the optimization in Equation 1 may determine a large global shear that may minimize the matching error between the first and the last view.
  • some embodiments may further refine the light field using local shears (including, but not limited to. small local shears). Instead of computing the amount of shear for each pixel of I n , some embodiments may define a regular grid (e.g., having dimensions m x m), and find optimal shears for these grid points. Finding an improved (and/or best) shear for every point separately may result in discontinuities, which may introduce significant compression and stretching to the light field. Therefore, the some embodiments may determine improved shear magnitudes that may vary smoothly across different locations.
  • Some embodiments may find an optical flow between two views and minimize differences between them using a warp guided by the flow field.
  • some embodiments may restrict local warps to be small, which may results in matching similar structures instead of the same objects. Finding a dense correspondence between views may also introduce an additional problem of disocclusions, which may lead to significant compression and stretching artifacts during the warping.
  • some embodiments define a regular grid (e.g., 20 x 20), and find the optimal shears for the grid points. As such, some embodiments may find improved shear magnitudes that vary smoothly across different locations.
  • an additional step may be performed in that the coarse grid may be warped to improve the continuity of the light field.
  • the problematic regions may be filled in using the neighboring signal.
  • the above-mentioned problem of finding the optimal local shear may be formulated as a minimum graph cut.
  • some embodiments may create multiple nodes (i , s), where s may span a range of integer values from [s m ' in , s max] an d may correspond to different magnitudes of shear considered at each location.
  • the edges in the graph may be between s) and s+l), and may encode the cost of the shear s at the position
  • some embodiments may add to the graph a source and a target node (S, T), which may be connected to s m ' in ) and s m ' ax ) respectively.
  • some embodiments may adapt forward edges (see, e.g., Rubinstein, M., Shamir, A., and Avidan, S., "Improved Seam Carving for Video Retargeting," ⁇ CM Trans. Graph. 27, 3, 16: 1-16:9, August 2008, which is incorporated by reference in its entirety herein) and may add additional edges with an infinite cost (FIG. 9, element 952).
  • the amount of shear at the position (i,j) may be defined by the location of the cut, e.g., if the cut goes through edge s) ⁇ s+l), the optimal shear for position may be s.
  • some embodiments may first propagate the optimal shears from / chest to the views using linear interpolation, assuming that View Ij may receive zero shear. Then, in some embodiments, each given view may be separately sheared by warping the grid together with the underlying view.
  • FIG. 9 is a diagram 900 illustrating the cost function E(i,j, s), also element 902, according to some embodiments.
  • the cost function E(i,j, s), also element 902 is illustrated in three dimensions (element 980 representing the i axis, element 982 representing the j axis, and element 984 representing the s axis, respectively).
  • a continuous cut 904 illustrates local warping.
  • the corresponding weights 960, 962, 964, 966 on edges in the graph 952 are shown on the right.
  • forward edges dashex
  • forward edges 970 may be represented in three or more dimensions in some embodiments. Forward edges 970 may be added in the s-i (elements 984, 980) and s-j (elements 984, 982) planes.
  • the shearing techniques mentioned above may align the structure of the repeating light field fragments. However, sharp differences may remain visible. Some embodiments may apply an additional compositing of repeating light field structures in a gradient domain. Some embodiments use image/video stitching and/or retargeting techniques (see, Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y., "Drag-and-Drop Pasting,” ⁇ CM Transactions on Graphics (TOG), 25, 631-637, July 2006; Jia, J., and Tang, C.-K., "Image Stitching Using Structure Deformation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 30, 4, 617— 631 , April 2008; Rubinstein, M., Shamir, A., and Avidan, S., "Improved Seam Carving for Video Retargeting," ACM Trans.
  • some embodiments first create two copies of the original light field and overlap the two fields by m views along the s direction. Then, some embodiments may find a cut through the overlapping part, which may provide a surface where both replicas may fit better (and/or best). This cut, similarly to finding improved shears, may be found by using a graph cut technique, according to some embodiments.
  • some embodiments may first transform the overlapping light field volume into a graph, where each voxel (s, u, v) may correspond to a node.
  • the edges between (s, u, v) and (s+l , u, v) may encode the cost of the cut between these two voxels.
  • a goal of this cost may be to penalize significant differences (significant with respect to a programmable and/or pre-defined threshold) in gradients between the overlapping replicas, expressed as (Equation 3):
  • V SU L is the (s, u) component of the light field gradient
  • n is the total number of views
  • m is the number of views that are overlapped.
  • various elements of Equation 3 including but not limited to (s, u, v) and (n— m + 1 + s, u, v), as well as (s+l, u, v) and (n - m + 2 + s, u, v), may be positions that are directly overlapping.
  • some embodiments may add forward edges with an infinite cost and a source and/or a target node to perform a minimal graph cut.
  • Some embodiments after finding the optimal cut of the graph, may stitch gradients of the overlapping light field replicas along the cut, and may compute the full light field by reconstructing each EPI separately using Poisson reconstruction (see, e.g., Perez, P., Gangnet, M., and Blake, A., "Poisson Image Editing," ACM Trans. Graph. 22, 3, 313-318, July 2003, incorporated by reference herein in its entirety).
  • FIG, 11 illustrates a method (and system) 150 of the present invention of stitching by overlapping light field copies to find the optimal cut, according to some embodiments.
  • the width of the overlap m may control the number of views that are affected by the method of the present invention.
  • the method (and system) 150 of the present invention described above may apply to static light fields, according to some embodiments.
  • some embodiments as described above may extend to shearing and stitching videos by including a computation of a minimal graph cut for a 4D volume and Poisson reconstruction in 3D (see three dimensions 360, 362, 364 of FIG. 11).
  • a computation may be performed for every k-t frame, rather than for each given frame of the video.
  • the shearing as well as cut may be linearly interpolated for the remaining frames between every k-t frame. Besides improved performance, such a solution may provide temporally coherent results in some embodiments.
  • k 50.
  • FIG. 10 is a diagram illustrating non-limiting examples of light fields produced by the method (and system) 150 of some embodiments.
  • input light fields 1002a-c are globally sheared to produce corresponding global sheared light fields 1004a-c.
  • an additional step of the method (and system) 150 e.g., local shearing and stitching
  • global shearing that results in global shearing output 1004a-c may be included in some
  • some embodiments may have global shearing, local shearing, and stitching, resulting in output light fields 1006a-c.
  • FIG. 10 in order to further show the above-mentioned visual (e.g., image) effects, below each light field 1002a-c, 1004a- c, and 1006a-c, is a corresponding zoomed-in magnification (elements 1012a-c, 1014a-c, and 1016a-c, respectively), which is also represented as a first set of boxes (boxes, 1012a, 1014a, 1016a), a second set of boxes (boxes, 1012b, 1014b, 1016b), and a third set of boxes (boxes, 1012c, 1014c, 1016c), respectively.
  • FIG. 12 is a diagram illustrating an example embodiment of a raw light field 1202 and the results of the method (and system) 150 employed by some embodiments of the present invention
  • processing a raw light field 1202 may include shearing globally 1204, also shearing locally 1206 (in addition to globally shearing 1204), and/or also including stitching 1208 (in addition to both global and local shearing 1206).
  • Zoomed in representations of global shearing 1204, global and local shearing 1206, and global (and local) shearing combined with stitching 1208 are illustrated in FIG. 12 as 1204b, 1206b, 1208b, respectively.
  • FIG. 12 illustrates examples of some embodiments showing the results after each step.
  • FIG. 12 demonstrates that each of these steps 1204, 1206, 1208 improves the continuity of the resulting light field.
  • Some embodiments may not completely remove transitions because doing so may include flattening the scene. However, due to stitching across many views, the remaining transitions may be distributed across many frames, which may make them less apparent.
  • Some embodiments may not display one or more views from the original light field on the screen because it may use some of them to perform overlap. To remedy this deficiency, some embodiments may employ view synthesis techniques that may be used to create additional views for the purpose of the overlap.
  • the global shear 1204 of the original light field 1202 reduces excessive disparities and provides an improved alignment of the lines 1210 (to the left of each image 1204, 1206, 1208).
  • local shearing 1206 significantly improves the discontinuities between the lines 1220 (in the middle of images 1204b, 1206b, 1208b). In some embodiments, this may be achieved by locally sacrificing the continuity of the lines 1210, which may not influence significantly the overall quality across the views.
  • the complete technique 1208 may match the lines 1220 and the lines 1230 to further improve the continuity between the replicas.
  • the Poisson reconstruction is applied in in the above-mentioned examples in FIG. 12.
  • the method (and system) 150 may be performed on a variety of light fields of images and/or videos. Compared to the original light field 1202, and to one where global shear is applied 1204, the full technique 1208 may provide smoother results. In many high frequency regions, the method (and system) 150 may find local repetitive structures, and eliminate one or more transitions. In some embodiments, the stitching may propagate transitions optimally into different views in different regions, making them less pronounced.
  • FIG. 13 illustrates a comparison of four views generated using a method (and system) 150 according to some embodiments, as compared against four corresponding views where global shear is applied, As such, FIG. 13 illustrates a comparison of four views generated using a global shearing method 1304 of some embodiments, as compared with four views generated using the full technique (global and local shearing combined with stitching) 1302 of some embodiments.
  • the method (and system) 150 may apply the shear to align both light fields in the same way around the screen.
  • the method (and system) 150 may provide smoother transitions with less discontinuity and depth reversal, and fewer diagonal strips.
  • implementation may take 1 minute (including 5 seconds for shearing and stitching and 55 seconds for Poisson reconstruction, in a non-limiting example).
  • the Poisson reconstruction may be performed for each epipolar plane image separately.
  • processing 80 frames of a multi-view video in resolution 800 x 540 may take almost 1 hour, in which the bottleneck may be the Poisson reconstruction.
  • the performance of the method (and system) 150 of the present invention is improved.
  • Some embodiments may be highly parallelizable, e.g., every shot may be processed separately. Also, for slowly changing scenes, computation may be performed for fewer frames. As such, some embodiments alone, and/or in some
  • embodiments combination with a GPU implementation may reduce the computation time significantly.
  • user experiments are conducted that include a performance comparison of some embodiments, which include automatic global shear compared against manual adjustment done by users (FIG. 14), and then are evaluated (FIG. 15) using the full technique (e.g., global and local shearing combined with stitching) according to some embodiments.
  • the following experiments include 16 participants with normal and/or corrected-to-normal vision without stereoblindness.
  • FIG. 14 includes illustrations 1400 of non-limiting examples from the experiment: "Couch” (element 1450) and “Train” (element 1440) are captured using a camera array, and “Bunny” (element 1410) and “Teapot” (element 1460) are rendered.
  • each light field may include 100 views.
  • User experiments may evaluate the quality improvement provided by the method (and system) 150 of the present invention, according to some embodiments.
  • the 16 participants with normal or corrected-to-normal vision that are tested for stereoblindness are included in the experiment.
  • Four different light fields of 100 are processed using some embodiments.
  • Each stimuli presents to the participants two versions of the content produced by some embodiments: one with the full technique applied (global shearing, local shearing, and stitching) and the other with global shearing. While the shear is also a novel part of the method (and system) 150, it may be applied in the user test to make both stimuli comparable in terms of presented depth range.
  • the global shear may adjust the position of the entire scene with respect to the screen plane.
  • a similar, manual correction is a common practice to reduce the need of interview antialiasing (see, Zwicker, M., Matusik, W., Durand, F., and Pfister, H., "Antialiasing for Automultiscopic 3D Displays," in Proceedings of the 17th Eurographics Conference on Rendering Techniques, Eurographics Association, 73-82, June 2006; and Didyk, P., Sitthi- Amorn, P., Freeman, W., Durand, F., and Matusik, W., "Joint View Expansion and Filtering for Automultiscopic 3D Displays," ACM Trans.
  • the sequence is displayed using a 42-inch 8-view automultiscopic NEWSIGHT JAPAN display.
  • the participants are asked to sit 1.5 meters (1.5 m) away from the screen, which is preferably the optimal viewing distance for this display, and adjust the global depth of the scene until an improved (and/or the best) viewing quality is achieved.
  • the average adjustment for each scene is computed.
  • the same content is processed using global shear.
  • the resulting adjustments are compared with the results from the user study.
  • the small differences and relatively high inter-subject variability (Table 1) suggest that the difference between the global shear and the manual adjustments is small.
  • the full technique global shearing, local shearing, and stitching is compared to the global shear as both of the embodiments provide an automatic solution.
  • Table 1 illustrates statistics for the manual adjustment of the content.
  • the ⁇ adjustment (in number of pixels, px) represents the difference between correction provided by an embodiment with global shear and manual adjustment provided by users. The difference is expressed as a change of the disparities between neighboring views and measured for Full high definition (HD) resolution. Additionally, standard deviation ( ⁇ ) and standard error of the mean (SEM) are illustrated in Table 1.
  • FIG. 15 Another experiment (results shown in FIG. 15), compares the results obtained using an embodiment that includes global shear against sequences produced using an embodiment having the full technique (global shearing, local shearing, and stitching) of the method (and system) 150 of the present invention.
  • 8 viewing positions are tested.
  • participants are asked to stay in one location which is marked on a floor. They are presented with both versions of the content, and may switch between them using a keyboard.
  • the task for each participant is to judge which version of the content (e.g., the embodiment using global shear or the embodiment using the full method) provides an improved viewing experience.
  • Each of the observers perform the task for each location and for the videos.
  • the arrangement of the experiment and the results for the viewpoints are shown in FIG. 15.
  • the scenario of FIG. 15 may be considered as a common scenario for watching such image content.
  • FIG. 15 illustrates the corresponding results of this experiment for
  • the percentage values in FIG. 15 may correspond to the percentage of people who chose the full technique 1512 (including global shear, local shear, and stitching) and the content processed using global shear 1510. The places for which the results are not statistically significant are indicated 1520 in FIG. 15.
  • FIG. 16 A similar experiment is shown in FIG. 16, with 4 different static light fields ("Train,” element 1440, “Couch,” element 1450, “Teapot,” element 1460, and “Bunny,” element 1410 of FIG. 14).
  • both versions may be downsampled to 18 views, printed at 720 DPI and glued to lenticular sheets with 40 lenses per inch with a viewing angle of 25.
  • both versions are printed on a single sheet. Additional sheets may be produced where the versions are swapped to provide a random order of methods.
  • each stimulus may be shown to every participant, who may be asked to perform free viewing (i.e., they may take each lenticular sheet and look at it from different angles and different distances). The participants are allowed to take as much time as they want to view the two versions of output images (1610 and 1612). Afterwards, the
  • FIG. 16 is a graph illustrating results of the above-mentioned experiment. In order to further validate that the results of this experiment are statistically significant, a binomial test is computed for each scene. For the examples the obtained p- value may be below 0.05, which confirms that some embodiments functionally improve the visual quality of images.
  • Some embodiments may also combine such manipulations with depth remapping methods and inter-view antialiasing. For real-time applications, such as games, this may improve the performance. Even without such depth remapping methods and inter-view antialiasing already has a wide range of applications, as it does not depend on a specific type of display device.
  • Some embodiments may take two or more steps to process the light field, including but not limited to shearing (global and/or local) and stitching.
  • shearing global and/or local
  • stitching due to stitching across many views (please refer to the above mentioned section titled "Light Field Stitching"), the remaining artifacts may be distributed across different views.
  • the artifacts that remain after the techniques (1204, 1206, 1208) are applied may be small and local.
  • an approach may be followed (Filippini, H. R., and Banks, M.
  • depth may be computed using cross correlation for two original views, and the same views processed using some embodiments. While the original content may produce pronounced depth reversals, some embodiments may provide views for which resulting depth may contain less structured depth errors.
  • distributing transitions across different views may affect the sweet-spot viewing.
  • the stitching may be performed on a small part of the light field near viewing zone boundaries.
  • m nIA
  • the resulting light field may include 3/4 of the views, and the stitching step may affect 1/4 of the views.
  • view synthesis techniques may be used to create additional views for the purpose of stitching (FIG. 18).
  • the display e.g., number of views
  • the viewing conditions e.g., range of viewing distances
  • different sets of views may be used.
  • several versions of the content may be prepared and delivered to the client.
  • the correct version may be sent alone.
  • Some embodiments may benefit from repetitive patterns, which are common in many image scenes, including but not limited to natural scenes such as trees, grass, ground, clouds and/or man-made image scenes.
  • the original light field 1810 (which may include 100 views) may be expanded 1820 (for non-limiting example, to 200 or more views) using image-based warping of the method (and system) 150 of the present invention.
  • the resulting light field 1830 may have the same number of views as the input content.
  • an embodiment that uses local shearing may find improved matches and improve the quality of the light field. Some embodiments may reduce the transitions problem by placing these objects close to the screen plane, which is also desirable for eye strain reduction.
  • Some embodiments may not apply directly to multi-layer displays; however, some embodiments may be used to expand their field of view. Other embodiments may combine such manipulations with depth remapping methods and interview antialiasing as well as to improve performance for realtime applications. Some embodiments are beneficial not only for 3DTV applications and 3D visualizations, but also for large scale projector-based cinema systems.
  • FIG. 19 is a flow diagram illustrating a process 1900 employed by some embodiments that include the method (and system) 150 of the present invention.
  • the process 1900 may begin by receiving an input light field (1902).
  • the process 1900 then may apply a global shear to the light field (1904).
  • the shear may change (e.g., adjust) the slope of one or more lines associated with the input light field (1902), which is another way to express that the shear may also change the depth of the scene.
  • the process 1900 then may apply a local shear to the light field (1906).
  • the process 1900 may perform the local shear by dividing the light field into a grid, and applying local shears to each portion of the grid.
  • the process 1900 then may apply stitching to the light field (1908).
  • FIG. 20 is a block diagram 2000 illustrating an example embodiment of the present invention.
  • a memory 2002 and processor 2004 may be operatively coupled with an artifact removal module 2006.
  • the artifact removal module 2006 may be configured to remove one or more artifacts from the multi-view content by modifying the multi-view content.
  • the artifact removal module 2006 is programmed and/or otherwise configured to execute the process 1900 of FIG. 19 described in detail above.
  • the memory 2002, processor 2004 and artifact removal module 2006 may also be coupled with a rendering system 2008.
  • the rendering system 2008 may be configured to render the multi-view content at a display or print the multi-view content for use with a lenticular sheet, for non-limiting example.
  • FIG. 21 is a high-level block diagram 2100 of an embodiment of the present invention method (and system) 150 that may remove artifacts from an input image frame (and/or input image sequence and/or input image light field) according to the principles of the present invention.
  • the computer-based system 2100 contains a bus 2106.
  • the bus 2106 is a connection between the various components of the system 2100.
  • Connected to the bus 2106 is an input/output device interface 2128 for connecting various input and output devices, such as a keypad, controller unit, keyboard (generally 2124), mouse/pointing device 2126, display, speakers, touchscreen display (generally display device 2118), etc. to the system 2100.
  • the input/output device interface 2128 provides an interface for allowing a user to select image (and/or video) display parameters and aspects using any method as is known in the art.
  • a central processing unit (CPU) 2102 is connected to the bus 2106 and provides for the execution of computer instructions such as those of artifact removal module 2006 and process 1900 discussed above.
  • Memory 21 10 provides volatile storage for data used for carrying out computer instructions.
  • Storage or RAM 2108 provides nonvolatile storage for software instructions such as an operating system.
  • the system 2100 also comprises a network interface 2122, for connecting to any variety of networks, including wide area networks (WANs), local area networks (LANs), wireless networks, mobile device networks, cable data networks and so on.
  • WANs wide area networks
  • LANs local area networks
  • wireless networks mobile device networks
  • cable data networks and so on.
  • steps of the processes described above and/or any additional processes that may be related to those described above may be stored as computer executable instructions in, for example a memory area 2104 that is operably and/or communicatively coupled to the processor 2102 and to a GPU 2120 by a system bus 2106 or similar supporting data communication line.
  • a "memory area” as used herein, refers generally to any means of storing program code and instructions executable by one or more processors to aid in storing multi-view image content in an electronic memory, removing one or more artifacts from the multi-view image content, and/or modifying the multi-view image content including shearing the multi-view image content globally, shearing the multi-view image content locally, and/or stitching the multi-view image content.
  • the instructions executable by one or more processors, based upon the modification of the multi-view image content may provide one or more updated multi-view images in which the one or more artifacts are removed and/or reduced in visibility.
  • the memory area 2104 may include one, or more than one, form of memory.
  • the memory area 2104 may include random access memory (RAM) 2108, which may include non-volatile RAM, magnetic RAM, ferroelectric RAM, and/or other forms of RAM.
  • RAM random access memory
  • the memory area 2104 may also include read-only memory (ROM) 2110 and/or flash memory and/or electrically erasable programmable read-only memory (EEPROM).
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • Any other suitable magnetic, optical and/or semiconductor memory, such as a hard disk drive (HDD) 21 12, by itself or in combination with other forms of memory, may be included in the memory area 2104.
  • HDD 2112 may be coupled to a disk controller 21 14 for use in transmitting and receiving messages to and from processor 2102.
  • the memory area 2104 may also be or may include a detachable or removable memory 21 16 such as a suitable cartridge disk, CD-ROM, DVD, or USB memory.
  • the memory area 2104 may in some embodiments effectively include cloud computing memory accessible through network interface 2122, and the like.
  • the above examples are exemplary only, and thus, are not intended to limit in any way the definition and/or meaning of the term "memory area.”
  • a CPU 2102 sends a stream of two-dimensional (2D) and/or three-dimensional (3D) video images (including, but not limited to the three dimensions of an x-axis, a y-axis, and time) to GPU 2120 via a system bus 2106 or other communications coupling.
  • GPU 2120 employs the above-described methods, algorithms and computer-based techniques as programmed in memory area 2104 to generate images exhibiting removed and/or reduced artifacts for display on display device 21 18.
  • the GPU 2120 forms a picture of the screen image and stores it in a frame buffer. This picture is a large bitmap used to continually update and drive the screen image on display device 21 18.
  • embodiments may include a stream of three- dimensional and/or four-dimensional video images (including, but not limited to three dimensions including an x-axis, a y-axis, and time, or four dimensions including a x-axis, y- axis, z-axis, and time).
  • the display device 2118 may be, without limitation, a monitor, a television display, a plasma display, a liquid crystal display (LCD), a display based on light emitting diodes (LED), a display based on organic LEDs (OLEDs), a display based on polymer LEDs, a display based on surface-conduction electron emitters, a display including a projected and/or reflected image, or any other suitable electronic device or display mechanism.
  • LCD liquid crystal display
  • LED light emitting diodes
  • OLEDs organic LEDs
  • polymer LEDs based on polymer LEDs
  • surface-conduction electron emitters a display including a projected and/or reflected image, or any other suitable electronic device or display mechanism.
  • the display device 2118 may include a touchscreen with an associated touchscreen controller.
  • the above examples are exemplary only, and thus, are not intended to limit in any way the definition and/or meaning of the term "display device.”
  • artifacts may be referred to as visual artifacts and/or image artifacts.
  • a scene and/or image scene
  • a screen including but not limited to an embedded screen
  • a pixel may refer to an image pixel.
  • multi-view image content may be referred to as a multi- view image and/or one or more multi-view images.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

L'invention concerne des affichages automultiscopiques permettant une visualisation 3D sans lunettes en assurant une parallaxe à la fois binoculaire et de mouvement. À l'intérieur du champ de vision de l'affichage, différentes images sont observées en fonction de la direction de visualisation. Lors d'un déplacement hors du champ de vision, les images observées peuvent se répéter. Des champs lumineux produits par des affichages automultiscopiques lenticulaires et à barrière de parallaxe peuvent présenter une structure répétitive avec des discontinuités significatives entre les champs de vision. Cette structure répétitive induit des artefacts visuels sous la forme de discontinuités de vues, d'inversions de relief et de disparités extensives. Pour surmonter ce problème, un procédé selon l'invention modifie le contenu d'image présenté des champs lumineux et le rend plus répétitif. Dans le procédé, un champ lumineux est raffiné en utilisant un cisaillement global et local, puis les fragments répétitifs sont raccordés. Le procédé réduit les discontinuités dans le champ lumineux affiché et conduit à des améliorations de la qualité visuelle. Les avantages du procédé sont démontrés en utilisant un affichage automultiscopique doté d'une barrière de parallaxe et d'impressions lenticulaires.
PCT/US2015/014434 2014-02-07 2015-02-04 Réduction des artefacts de transition entre vues dans les affichages automultiscopiques WO2015120032A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201461937371P 2014-02-07 2014-02-07
US61/937,371 2014-02-07
US14/531,548 US9756316B2 (en) 2013-11-04 2014-11-03 Joint view expansion and filtering for automultiscopic 3D displays
US14/531,548 2014-11-03

Publications (1)

Publication Number Publication Date
WO2015120032A1 true WO2015120032A1 (fr) 2015-08-13

Family

ID=53778398

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2015/014434 WO2015120032A1 (fr) 2014-02-07 2015-02-04 Réduction des artefacts de transition entre vues dans les affichages automultiscopiques

Country Status (1)

Country Link
WO (1) WO2015120032A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107038720A (zh) * 2015-12-23 2017-08-11 汤姆逊许可公司 具有可调视差方向的三维渲染
US9756316B2 (en) 2013-11-04 2017-09-05 Massachusetts Institute Of Technology Joint view expansion and filtering for automultiscopic 3D displays
US9967538B2 (en) 2013-11-04 2018-05-08 Massachussetts Institute Of Technology Reducing view transitions artifacts in automultiscopic displays

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US725567A (en) 1902-09-25 1903-04-14 Frederic E Ives Parallax stereogram and process of making same.
EP1235439A2 (fr) * 2001-02-23 2002-08-28 Sharp Kabushiki Kaisha Méthode et dispositif de transformation d'images stéréoscopiques
US20040233275A1 (en) * 2003-03-20 2004-11-25 Seijiro Tomita Stereoscopic image picking up and display system
AP1903A (en) 2002-04-22 2008-10-16 Sekisui Rib Loc Australia Pty Ltd Composite strip windable to form a helical pipe and method therefor
US20110109629A1 (en) * 2007-08-29 2011-05-12 Setred As Rendering improvement for 3d display
EP2323416A2 (fr) * 2009-11-11 2011-05-18 Disney Enterprises, Inc. Édition stéréoscopique pour la production vidéo, la post-production et l'adaptation d'affichages
US20130286170A1 (en) * 2010-12-29 2013-10-31 Thomson Licensing Method and apparatus for providing mono-vision in multi-view system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US725567A (en) 1902-09-25 1903-04-14 Frederic E Ives Parallax stereogram and process of making same.
EP1235439A2 (fr) * 2001-02-23 2002-08-28 Sharp Kabushiki Kaisha Méthode et dispositif de transformation d'images stéréoscopiques
AP1903A (en) 2002-04-22 2008-10-16 Sekisui Rib Loc Australia Pty Ltd Composite strip windable to form a helical pipe and method therefor
US20040233275A1 (en) * 2003-03-20 2004-11-25 Seijiro Tomita Stereoscopic image picking up and display system
US20110109629A1 (en) * 2007-08-29 2011-05-12 Setred As Rendering improvement for 3d display
EP2323416A2 (fr) * 2009-11-11 2011-05-18 Disney Enterprises, Inc. Édition stéréoscopique pour la production vidéo, la post-production et l'adaptation d'affichages
US20130286170A1 (en) * 2010-12-29 2013-10-31 Thomson Licensing Method and apparatus for providing mono-vision in multi-view system

Non-Patent Citations (42)

* Cited by examiner, † Cited by third party
Title
AGARWALA, A.: "Efficient Gradient-Domain Compositing Using Quadtrees", ACM TRANS. GRAPH. (TOG, vol. 26, no. 94, July 2007 (2007-07-01)
AGARWALA, A.; ZHENG, K. C.; PAL, C.; AGRAWALA, M.; COHEN, M.; CURLESS, B.; SALESIN, D.; SZELISKI, R.: "Panoramic Video Textures", ACM TRANS. GRAPH., (TOG, vol. 24, July 2005 (2005-07-01), pages 821 - 827
BIRKLBAUER, C.; BIMBER, 0.: "LightField Retargeting", WILEY ONLINE LIBRARY, COMPUTER GRAPHICS FORUM, vol. 31, May 2012 (2012-05-01), pages 295 - 303
BROOKES, A.; STEVENS, K. A.: "The Analogy Between Stereo Depth and Brightness", PERCEPTION, vol. 18, no. 5, February 1989 (1989-02-01), pages 601 - 614
CHEN, B.; OFEK, E.; SHUM, H.-Y.; LEVOY, M.: "Interactive Deformation of Light Fields", ACMPROCEEDINGS OF THE 2005 SYMPOSIUM ON INTERACTIVE 3D GRAPHICS AND GAMES, vol. 367, April 2005 (2005-04-01), pages 139 - 146
DIDYK ET AL.: "Joint view expansion and filtering for automultiscopic 3D displays", ACM TRANSACTIONS ON GRAPHICS (TOG, vol. 32, no. 6, November 2013 (2013-11-01), pages 221
DIDYK, P.; RITSCHEL, T.; EISEMANN, E.; MYSZKOWSKI, K.; SEIDEL, H.-P.; MATUSIK, W.: "A Luminance-Contrast-Aware Disparity Model and Applications", ACM TRANS. GRAPH., vol. 6, November 2012 (2012-11-01), pages 184 - 1,184-10
DIDYK, P.; SITTHI-AMORN, P.; FREEMAN, W.; DURAND, F.; MATUSIK, W.: "Joint View Expansion and Filtering for Automultiscopic 3D Displays", ACMTRANS. GRAPH, vol. 32, no. 6, November 2013 (2013-11-01), pages 221 - 1,221-8
DU, S.-P.; DIDYK, P.; DURAND, F.; HU, SHI-MIN; MATUSIK, W.: "Improving Visual Quality of View Transitions in Automultiscopic Displays", JOURNAL OF ACMTRANSACTIONS ON GRAPHICS, vol. 33, no. 6, November 2014 (2014-11-01)
EISEMANN, M.; GOHLKE, D.; MAGNOR, M.: "Edge-Constrained Image Compositing", PROCEEDINGS OF GRAPHICS INTERFACE 2011, CANADIAN HUMAN-COMPUTER COMMUNICATIONS SOCIETY, May 2011 (2011-05-01), pages 191 - 198
EISEMANN, M.; GOHLKE, D.; MAGNOR, M.: "Edge-Constrained Image Compositing", PROCEEDINGS OF GRAPHICS INTERFACE 2011, CANADIAN HUMAN-COMPUTERCOMMUNICATIONS SOCIETY, May 2011 (2011-05-01), pages 191 - 198
FILIPPINI, H. R.; BANKS, M. S.: "Limits of stereopsis explained by local cross-correlation", JOURNAL OF VISION, vol. 9, no. 1, January 2009 (2009-01-01), pages 8,1 - 18
HORN, D. R.; CHEN, B.: "Lightshop: Interactive Light Field Manipulation and Rendering", ACMPROCEEDINGS OF THE 200 SYMPOSIUM ON INTERACTIVE 3D GRAPHICS AND GAMES, April 2007 (2007-04-01), pages 121 - 383
IMPROVING VISUAL QUALITY OF VIEW TRANSITIONS IN AUTOMULTISCOPIC DISPLAYS, September 2014 (2014-09-01), Retrieved from the Internet <URL:http://www.youtube.com/ under>
ISAKSEN, A.; MCMILLAN, L.; GORTLER, S. J.: "Proceedings of the 2ih Annual Conference on Computer Graphics and Interactive Techniques", July 2000, ACM PRESS/ADDISON-WESLEY PUBLISHING CO., article "Dynamically Reparameterized Light Fields", pages: 297 - 306
JIA, J.; SUN, J.; TANG, C.-K.; SHUM, H.-Y.: "Drag-and-Drop Pasting", ACM TRANSACTIONS ON GRAPHICS (TOG, vol. 25, July 2006 (2006-07-01), pages 631 - 637
JIA, J.; SUN, J.; TANG, C.-K.; SHUM, H.-Y.: "Drag-and-Drop Pasting", ACMTRANSACTIONS ON GRAPHICS (TOG, vol. 25, July 2006 (2006-07-01), pages 631 - 637
JIA, J.; TANG, C.-K.: "Image Stitching Using Structure Deformation", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 30, no. 4, April 2008 (2008-04-01), pages 617 - 631
KIM, C.; HORNUNG, A.; HEINZLE, S.; MATUSIK, W.; GROSS, M.: "Multi-Perspective Stereoscopy from Light Fields", A CM TRANS. GRAPH, vol. 30, no. 6, December 2011 (2011-12-01), pages 190
KONRAD, J.; AGNIEL, P.: "Subsampling Models and Anti-Alias Filters for 3-D Automultiscopic Displays", IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 15, no. 1, January 2006 (2006-01-01), pages 128 - 140
KWATRA, V.; SCHODI, A.; ESSA, I.; TURK, G.; BOBICK, A.: "Graphcut Textures: Image and Video Synthesis Using Graph Cuts", ACM TRANS. GRAPH., vol. 22, no. 3, July 2003 (2003-07-01), pages 277 - 286
LEVIN, A.; ZOMET, A.; PELEG, S.; WEISS, Y.: "Seamless Image Stitching in the Gradient Domain", COMPUTER VISION-ECCV, vol. 3024, May 2004 (2004-05-01), pages 377 - 389
LEVOY, M.; HANRAHAN, P.: "Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques", August 1996, ACM, article "Light Field Rendering", pages: 31 - 42
LIPPMANN, G.: "Epreuves Reversibles Donnant La Sensation Du Relief", JOURNAL OF PHYSICS, vol. 7, no. 4, November 1908 (1908-11-01), pages 821 - 825
LIPPMANN: "Epreuves reversibles donnant la sensation du relief", JOURNAL OF PHYSICS, vol. 7, no. 4, November 1908 (1908-11-01), pages 821 - 825
MAHAJAN, D.; HUANG, F.-C.; MATUSIK, W.; RAMAMOORTHI, R.; BELHUMEUR, P.: "Moving Gradients: A Path-Based Method for Plausible Image Interpolation", ACM TRANS. GRAPH., vol. 28, no. 3, August 2009 (2009-08-01), pages 42
MASIA, B.; WETZSTEIN, G.; ALIAGA, C.; RASKAR, R.; GUTIERREZ, D.: "Display Adaptive 3D Content Remapping", COMPUTERS & GRAPHICS, vol. 37, no. 8, July 2013 (2013-07-01), pages 983 - 996
PEREZ, P.; GANGNET, M.; BLAKE, A.: "Poisson Image Editing", ACM TRANS. GRAPH., vol. 22, no. 3, July 2003 (2003-07-01), pages 313 - 318
PETERKA, T.; KOOIMA, R. L.; SANDIN, D. J.; JOHNSON, A. E.; LEIGH, J.; DEFANTI, T. A.: "Advances in the Dynallax Solid-State Dynamic Parallax Barrier Autostereoscopic Visualization Display System", IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, vol. 14, May 2008 (2008-05-01), pages 487 - 499
RUBINSTEIN, M.; SHAMIR, A.; AVIDAN, S.: "Improved Seam Carving for Video Retargeting", ACM TRANS. GRAPH. 27, vol. 3, no. 16, August 2008 (2008-08-01), pages 1 - 16
RUBINSTEIN, M.; SHAMIR, A.; AVIDAN, S.: "Improved Seam Carving for Video Retargeting", ACM TRANS. GRAPH., vol. 27, no. 3, August 2008 (2008-08-01), pages 16 - 1,16-9
RUBINSTEIN, M.; SHAMIR, A.; AVIDAN, S.: "Improved Seam Carving for Video Retargeting", ACMTRANS. GRAPH. 27, vol. 3, August 2008 (2008-08-01), pages 16 - 1,16-9
SCHODL, A.; SZELISKI, R.; SALESIN, D. H.; ESSA, I.: "Video Textures", ANNUAL CONFERENCE ON COMPUTER GRAPHICS, SIGGRAPH '00, July 2000 (2000-07-01), pages 489 - 498
SHIBATA, T.; KIM J.; HOFFMAN, D.; BANKS, M.: "The Zone of Comfort: Predicting Visual Discomfort with Stereo Displays", JOURNAL OF VISION 11, vol. 8, no. 11, 29 July 2011 (2011-07-29), pages 1 - 11
SHIBATA, T.; KIM J.; HOFFMAN, D.; BANKS, M.: "The Zone of Comfort: Predicting Visual Discomfort with Stereo Displays", JOURNAL OF VISION, vol. 11, no. 8, July 2011 (2011-07-01), pages 11 - 1,11-29
SONG-PEI DU ET AL: "Improving visual quality of view transitions in automultiscopic displays", ACM TRANSACTIONS ON GRAPHICS (TOG), ACM, US, vol. 33, no. 6, 19 November 2014 (2014-11-19), pages 1 - 9, XP058060831, ISSN: 0730-0301, DOI: 10.1145/2661229.2661248 *
STEWART, J.; YU, J.; GORTLER, S. J.; MCMILLAN, L.: "Proceedings of the 14th Eurographics workshop on Rendering, Eurographics Association", June 2003, article "A New Reconstruction Filter for Undersampled Light Fields", pages: 150 - 156
TOMPKIN, J.; HEINZLE, S.; KAUTZ, J.; MATUSIK, W.: "Content-Adaptive Lenticular Prints", ACM TRANS. GRAPH. 32, vol. 4, no. 133, 10 July 2013 (2013-07-10), pages 1 - 133
YE, G.; STATE, A.; FUCHS, H.: "A Practical Multiviewer Tabletop Autostereoscopic Display", 2010 9TH IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR), October 2010 (2010-10-01), pages 147 - 156
YI, S.-Y.; CHAEAND, H.-B.; LEE, S.-H.: "Moving Parallax Barrier Design for Eye-Tracking Autostereoscopic Displays", 3DTV CONFERENCE: THE TRUE VISION - CAPTURE, TRANSMISSION AND DISPLAY OF 3D VIDEO, May 2008 (2008-05-01)
ZHANG, Z.; WANG, L.; GUO, B.; SHUM, H.-Y.: "Feature-Based Light Field Morphing", ACMTRANS. GRAPH., vol. 21, no. 3, July 2002 (2002-07-01), pages 457 - 464
ZWICKER, M.; MATUSIK, W.; DURAND, F.; PFISTER, H.: "Antialiasing for Automultiscopic 3D Displays", PROCEEDINGS OF THE 17TH EUROGRAPHICS CONFERENCE ON RENDERING TECHNIQUES, EUROGRAPHICS ASSOCIATION, June 2006 (2006-06-01), pages 73 - 82

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9756316B2 (en) 2013-11-04 2017-09-05 Massachusetts Institute Of Technology Joint view expansion and filtering for automultiscopic 3D displays
US9967538B2 (en) 2013-11-04 2018-05-08 Massachussetts Institute Of Technology Reducing view transitions artifacts in automultiscopic displays
CN107038720A (zh) * 2015-12-23 2017-08-11 汤姆逊许可公司 具有可调视差方向的三维渲染

Similar Documents

Publication Publication Date Title
US20180249145A1 (en) Reducing View Transitions Artifacts In Automultiscopic Displays
US9031356B2 (en) Applying perceptually correct 3D film noise
Zinger et al. Free-viewpoint depth image based rendering
EP2150065B1 (fr) Procédé et système pour rendu vidéo, produit de programme informatique associé
US8957892B2 (en) Stereo composition based on multiple camera rigs
Schmidt et al. Multiviewpoint autostereoscopic dispays from 4D-Vision GmbH
US9756316B2 (en) Joint view expansion and filtering for automultiscopic 3D displays
US20060012675A1 (en) Three dimensional interaction with autostereoscopic displays
US20130057644A1 (en) Synthesizing views based on image domain warping
US20050231505A1 (en) Method for creating artifact free three-dimensional images converted from two-dimensional images
US20130187910A1 (en) Conversion of a digital stereo image into multiple views with parallax for 3d viewing without glasses
US20130033586A1 (en) System, Method and Apparatus for Generation, Transmission and Display of 3D Content
US20150348273A1 (en) Depth modification for display applications
KR20110090958A (ko) 이미지 속성들에 대한 오클루젼 데이터의 생성
WO2012140397A2 (fr) Système d&#39;affichage tridimensionnel
CN116982086A (zh) 先进立体渲染
Du et al. Improving visual quality of view transitions in automultiscopic displays
WO2015120032A1 (fr) Réduction des artefacts de transition entre vues dans les affichages automultiscopiques
JP4320271B2 (ja) 立体画像表示方法
EP2822279B1 (fr) Représentation de tapisserie autostéréoscopique
Miyashita et al. Perceptual Assessment of Image and Depth Quality of Dynamically Depth-compressed Scene for Automultiscopic 3D Display
KR20170033293A (ko) 입체 비디오 생성
Schild et al. Integrating stereoscopic video in 3D games
Boev et al. GPU-based algorithms for optimized visualization and crosstalk mitigation on a multiview display
Tan et al. A system for capturing, rendering and multiplexing images on multi-view autostereoscopic display

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15706575

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15706575

Country of ref document: EP

Kind code of ref document: A1