WO2015100490A1 - Reconfiguration of stereoscopic content and distribution for stereoscopic content in a configuration suited for a remote viewing environment - Google Patents

Reconfiguration of stereoscopic content and distribution for stereoscopic content in a configuration suited for a remote viewing environment Download PDF

Info

Publication number
WO2015100490A1
WO2015100490A1 PCT/CA2014/051228 CA2014051228W WO2015100490A1 WO 2015100490 A1 WO2015100490 A1 WO 2015100490A1 CA 2014051228 W CA2014051228 W CA 2014051228W WO 2015100490 A1 WO2015100490 A1 WO 2015100490A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
stereoscopic
viewing
user
version
Prior art date
Application number
PCT/CA2014/051228
Other languages
French (fr)
Inventor
Chang SU
Ngoc Lân NGUYEN
Nicholas Routhier
Original Assignee
Sensio Technologies Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sensio Technologies Inc. filed Critical Sensio Technologies Inc.
Publication of WO2015100490A1 publication Critical patent/WO2015100490A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/816Monomedia components thereof involving special video data, e.g 3D video
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/172Processing image signals image signals comprising non-image signal components, e.g. headers or format information
    • H04N13/178Metadata, e.g. disparity information
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194Transmission of image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/366Image reproducers using viewer tracking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/23439Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements for generating different versions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/254Management at additional data server, e.g. shopping server, rights management server
    • H04N21/2541Rights Management
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/482End-user interface for program selection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6587Control parameters, e.g. trick play commands, viewpoint selection

Definitions

  • This invention relates generally to the field of video content distribution and more particularly to the field of stereoscopic content distribution.
  • the field of video content distribution is a rapidly-expandmg one.
  • Traditional physical media such as Blu-ray DiscsTM are giving way to electronic forms of distribution.
  • Services like NetflixTM, iTunesTM or YouTubeTM allow users to rent or buy movies directly online without having to go to the store and purchase a physical medium. Instead, these services allow users to download (either on-the-fly by streaming or by downloading a video file(s) ) video content in programs such as movies or TV shows.
  • access to the program is typically limited by an amount of time or a number of viewings. For example, a streaming may only provide access to the rented content during the time of the rental, or a downloadable file may only be playable during that time.
  • 3D content present unique challenges in a content distribution setting.
  • 3D content is such that a user is presented with different images in the left and right eye so as to allow the user to perceive the content in three dimensions.
  • Such content may be called stereoscopic.
  • stereoscopic content is generally defined by a configuration that is adapted to a particular viewing environment.
  • the content may be viewed by different customers in different viewing environment, it has not been possible to provide high-quality 3D due to mismatch between ideal and real viewing environments. Summary
  • a method for managing access to viewable stereoscopic content in a digital content library by a remote user application for viewing at a remote viewing environment that is characterised by a set of user viewing parameters.
  • the method comprises the steps of determining the presence in the digital content library of a first version of a stereoscopic content in a first stereoscopic configuration, the first stereoscopic configuration corresponding to a first set of viewing parameters.
  • the method further comprises the step of determining the presence in the digital content library of a second version of the stereoscopic content in a second stereoscopic configuration, the second stereoscopic configuration corresponding to a second set of viewing parameters.
  • the method further comprises the step of receiving digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters.
  • the method further comprises the step of receiving from the remote user application a request for the stereoscopic content.
  • the method further comprises the step of selecting on the basis of the digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application.
  • the method further comprises the step of provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
  • a method for accessing viewable stereoscopic content from a digital content library by a user application for a viewing device being part of a viewing environment, the viewing environment characterised by a set of user viewing parameters comprises the step of transmitting to a content management server a request for a particular stereoscopic content.
  • the method further comprises the step of transmitting to the content management server digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter, the digital viewing parameter data being used for identifying a particular stereoscopic configuration corresponding to the viewing environment.
  • the method further comprises the step of receiving the stereoscopic content in the particular stereoscopic configuration.
  • the method further comprises the step of causing the stereoscopic content in the particular stereoscopic configuration to be displayed on a display associated with the viewing environment.
  • a method for providing stereoscopic video-on-demand content to a remote user operating a remote user application comprises the step of at a content management server, providing the remote user a list of stereoscopic programs available in a digital content library for display on a viewing device to the remote user.
  • the method further comprises the step of for a selected stereoscopic program in the list of stereoscopic programs, providing a regular version and a child-safe version, the regular version being an original configuration of the stereoscopic program and the child-safe version being a reconfigured version of the program reconfigured to adapt the program to a child interocular distance.
  • the method further comprises the step of selecting on the basis of digital viewing parameter data received from the remote user application one of the regular version and the child safe version of the selected stereoscopic.
  • the method further comprises the step of causing the selected version of the selected stereoscopic program to be transmitted to the remote user application for display on the viewing device.
  • a method for managing access to viewable stereoscopic content in a digital content library by a remote user application at a remote viewing environment characterised by a set of user viewing parameters comprises the step of receiving a stereoscopic content in a first stereoscopic configuration, the first stereoscopic configuration corresponding to a first set of viewing parameters.
  • the method further comprises the step of receiving from a remote user application a digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter.
  • the method further comprises the step of determining on the basis of the digital viewing parameter data whether the first stereoscopic configuration is suitable for viewing in the remote viewing environment.
  • the method further comprises the step of upon determining that the first stereoscopic configuration is not suitable for viewing in the remote viewing environment, performing a reconfiguration operation to generate a second stereoscopic configuration corresponding to the at least one viewing parameter from the set of user viewing parameter.
  • the method further comprises the step of provoking the transmission of the stereoscopic content in the second stereoscopic configuration to the remote user application.
  • a graphical user interface implemented with a viewing device for presenting to a user of the viewing device access to stereoscopic content in a digital content library offering for viewing at the viewing device.
  • the graphical user interface comprises a first pane comprising a plurality of first visual elements, each of the first visual elements being representative of a category of stereoscopic program and for each first visual element, a first input element associated with the visual element, the first input element being operable by the user using an input device to select the category of stereoscopic program associated with the first visual element associated with the first input element.
  • the graphical user interface further comprises a second pane comprising a plurality of second visual elements, each of the second visual elements being representative of a stereoscopic program and for each second visual element, a second input element associated with the visual element, the second input element being operable by the user using the input device to select the stereoscopic program associated with the second visual element associated with the second visual element.
  • the graphical user interface further comprises a third pane comprising a visual element displaying textual information about a particular stereoscopic program and an third input element associated with the particular stereoscopic program, the third input element being operable to select for viewing the particular stereoscopic program.
  • the graphical user interface further comprises a version visual element indicating the availability of a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters, the version visual element further providing for at least one of the plurality of versions information regarding the set of viewing parameters respective to corresponding stereoscopic configuration.
  • the graphical user interface further comprises a version input element operable by the user using the input device to select a version from amongst the plurality of versions of the particular stereoscopic program.
  • a content management system for managing access to viewable stereoscopic content in a digital content library by a remote user application at a remote viewing environment characterised by a set of user viewing parameters.
  • the content management entity comprises a stereoscopic content database comprising a set of records of stereoscopic content in a digital content library, the digital stereoscopic content database comprising for at least one of the records of stereoscopic content the identification of a plurality of versions of the stereoscopic content, each of the plurality of versions being in a different stereoscopic configuration, each stereoscopic configuration corresponding to a different set of viewing parameters.
  • the content management entity further comprises a communication interface system for communicating with a remote entity, the communication interface being suitable for receiving digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters.
  • the content management entity further comprises processing logic configured for accessing the records of stereoscopic content in the stereoscopic content database, accessing the digital viewing parameter data received from the remote entity, selecting on the basis of the received digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application; and provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
  • a method for permitting access by a remote user application to viewable stereoscopic content in a configuration adapted for a set of viewing parameters characterizing a remote viewing environment comprises establishing communication with a remote user device. The method further comprises transmitting to the remote user device a registration form comprising queries prompting the inputting of registration information by a user at the user device, the queries including at least one query prompting the input of at least one viewing parameter. The method further comprises receiving from the remote user device the registration information, the registration information comprising digital viewing parameter data comprising the at least one viewing parameter.
  • the method further comprises causing the association of the registration information with a unique user account at a content management system for selection by the content management system on the basis of the digital viewing parameter data of a version of stereoscopic from amongst a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters.
  • a registration system for permitting access by a remote user application to viewable stereoscopic content in a configuration adapted for a set of viewing parameters characterizing a remote viewing environment.
  • the registration comprises a communication interface system for establishing bidirectional communication with a remote user device.
  • the system further comprises processing logic configured to cause the transmission using the communication interface system to the remote user device a registration form comprising queries prompting the inputting of registration information by a user at the user device, the queries including at least one query prompting the input of at least one viewing parameter.
  • the processing logic is further configured to process registration information comprising digital viewing parameter data comprising the at least one viewing parameter received by the communication interface system from the remote user device to cause the association of the registration information with a unique user account at a content management system for selection by the content management system on the basis of the digital viewing parameter data of a version of stereoscopic content from amongst a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters.
  • a content access system for accessing viewable stereoscopic content from a digital content library for viewing in a viewing environment, the viewing environment characterised by a set of user viewing parameters.
  • the system comprises a communication interface system for communicating with a content management system.
  • the system further comprises processing logic configured to cause the transmission using the communication interface system to the content management system of a request for a particular stereoscopic content.
  • the processing logic is further configured to cause the transmission using the communication interface system to the content management system of digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter, the digital viewing parameter data being used for identifying a particular stereoscopic configuration corresponding to the viewing environment.
  • the processing logic is further configured to process a received stereoscopic content received at the communication interface system in the particular stereoscopic configuration in response to the request to cause the received stereoscopic content to be displayed on a display associated with the viewing environment.
  • a system for distributing stereoscopic video-on-demand content comprises a content management server having a digital stereoscopic content database comprising a set of records of stereoscopic content held in a digital content library.
  • the system further comprises a digital content library storing the stereoscopic content.
  • the system further comprises a remote user application in communication with the content management server and the content storage server at a remote viewing environment characterised by a set of user viewing parameters.
  • the remote user application is operative to send a request for a particular stereoscopic content from the content management server and send to the content management server digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters.
  • the content management server selects on the basis of the digital viewing parameter data one of a plurality of possible versions of the particular stereoscopic content each version having a respective stereoscopic configuration each corresponding to a respective set of viewing parameters.
  • the content management server causes the particular stereoscopic content to be transmitted to the remote user application in the selected version.
  • Figure 1 shows a solution of view-environment violation problem e.g. for the broadcasting industry with stereoscopic content reformater
  • Figure 2A shows the effect of image re-formatting on an X-Z plane
  • Figure 2B show the effect of the image re-formatting illustrated in Figure 2A but on the Y-Z plane;
  • Figure 3 A shows a geometric model depicting on the X-Z plane the display of a point in 3D on two different displays
  • Figure 3B shows a geometric model depicting on the X-Z plane the display of a different point in 3D on the two displays of Figure 3 A;
  • Figure 3C shows the geometric model of Figure 3 A on the Y-Z plane
  • Figure 3D shows the geometric model of Figure 3B on the Y-Z plane
  • Figure 4 shows the mam structure of the proposed re-formatting algorithm, according to an exemplary embodiment
  • Figure 5 shows an image tube and its transforming in a view space
  • Figure 6 shows a comfortable zone of the perceived depth in a view space
  • Figure 7 the proposed comfortable zone adaptive depth transforming algorithm, according to an exemplary embodiment
  • Figure 8 is a schematic diagram illustrating virtual stereoscopic content acquisition parameters for reformatting stereoscopic content for presentation on an intended stereoscopic display of a different size
  • Figure 9 is a schematic diagram illustrating projection differences of a scene object onto a second plane
  • Figure 10 illustrates a stereoscopic display displaying an object A in 3D
  • Figure 11 illustrates a large stereoscopic display and a small stereoscopic display displaying a same image comprising an object B to a user at a same distance from the display;
  • Figure 12 illustrates three viewers in three different positions relative to a stereoscopic display viewing a same object
  • Figure 13 shows a stereoscopic display showing two left-right vie-pairs of an object
  • Figure 14 shows a single viewer viewing an object D on a stereoscopic display
  • Figure 15 is a conceptual illustration of a stereoscopic viewing environment for stereoscopic content
  • Figure 16 is a block diagram illustrating non-limiting embodiment of a stereoscopic content distribution system
  • Figure 17 is a block diagram illustrating another view of a non-limiting embodiment of a stereoscopic content distribution system;
  • Figure 18 is a conceptual illustration of a stereoscopic content database according to a non-limiting embodiment;
  • Figure 19 is a process/data flow according to a non-limiting embodiment
  • Figure 20a is an illustration of one view of a graphical user interface according to a non-limiting embodiment
  • Figure 20b is an illustration of another view of the graphical user interface of Figure 20a.
  • Figure 21 is a block diagram illustrating another non-limiting embodiment of a stereoscopic content distribution system. Detailed Description
  • the images viewed in each eye differ by parallax providing the user a perception of depth.
  • the configuration determines which viewpoint of a scene will be seen by each eye.
  • the configuration of stereoscopic content is typically determined at capture by certain capture parameters. If only two images are captured in a stereoscopic scene, as is the case for typical content generated with stereoscopic camera pairs, transported over a two-image stereoscopic format and displayed on a display system providing two views to a user, one for each eye, such as an active shutter display or a polarized passive display, then the configuration of the content is determined by the parameters of the stereoscopic cameras.
  • the configuration of the content which determines which two viewpoints of a scene the eyes of the viewer will see can also be determined by capture parameters.
  • the stereoscopic configuration of stereoscopic content can also be affected by factors other than pure capture parameters, for example, the content may be re-configured during post-processing.
  • not all stereoscopic content is actually "captured" in the sense that they are caught on camera; much content nowadays is generated by computer and rendered in 3D in a particular stereoscopic configuration.
  • the stereoscopic configuration of content is responsible for perception of depth by the user since it determines which viewpoints of a scene the user perceives and since the viewpoint perceived creates the parallax effect that translates into depth perception.
  • the actual depth perception created by a particular stereoscopic configuration depends upon the viewing environment.
  • a same stereoscopic content in one particular stereoscopic configuration will appear differently in two different viewing environments. In particular the depth perceived by a user will be different, such that the content may look proportionally correct in one viewing environment may look stretched or compressed in the Z-direction (depth direction) in another viewing environment.
  • Viewing environments may be defined by certain viewing parameters as illustrated in Figure 15.
  • the viewing parameters are any parameters that may change from one viewing environment to another and that may affect the three-dimensional perception of video content.
  • a viewing environment in terms of viewer distance (VD), the interocular distance of the viewer (IOD), the display resolution and the display dimensions, for example a diagonal size from corner-to-diagonally-opposed-corner, or a height and width.
  • stereoscopic content is captured for one particular viewing environment, typically the cinema room.
  • stereoscopic content is captured as a stereoscopic pair of images. Dual cameras are used for such capture to obtain a left-eye and right-eye view of the captured scene which is to be represented to the eventual viewer in such a way as the viewer's left eye sees the left-eye perspective and right eye sees the right-eye perspective.
  • Capture parameters including camera focal distance, intercamera separation and camera angle of convergence, are selected with particular viewing environment in mind, to create a stereoscopic configuration of the captured content that provides an accurate perception of depth in that particular viewing environment
  • the target viewing environment will typically be a big-screen theater presentation with a centrally located viewer of typical IOD.
  • the 3D effect may be distorted as a result of the disparity not representing the same relative depth as in the original viewing space.
  • the resulting 3D image may exhibit stereoscopy that is uncomfortable for viewing under the new viewing parameters as it may require overconvergence or divergence of the viewer eyes under the new viewing parameters.
  • This is not limited to merely made-for-theater movies played on the home screen, but is an effect that can occur whenever a stereoscopic video is captured for a given set of viewing parameters and viewed under a second set of viewing parameters.
  • 3D video captured without proper care for viewing parameters may also exhibit such problems when viewed on a 3D display.
  • Stereoscopic reconfiguration is the generation of new viewpoints, or modification of existing viewpoints to create a new configuration for stereoscopic content in which the viewer sees a different viewpoint of a scene in at least one eye from the one seen in the original configuration.
  • a form of stereoscopic reconfiguration may be called stereoscopic re-formatting. It should be understood that examples comprising reconfigurating or a reconfigurator may use reformatting or a reformater, for example of the type described herein.
  • one goal for stereoscopic reconfiguration is to allow the automatic real- time reconfiguration at the viewing-end of any received stereoscopic content to adapt it to the particular viewing environment in which it is being viewed.
  • generating new viewpoints of a scene can be very challenging, particularly when the only existing information on the scene is contained in a stereoscopic pair of viewpoints provided by the original content.
  • stereoscopic content contains only two images, one for the left eye and one for the right eye, it can be very difficult to accurately re-create new views of the scene that are visually accurate-looking.
  • Simpler reconfiguration schemes attempt to simulate new viewpoints by simple image-shifting or pixel-shifting techniques.
  • One very effective reformatting technique has been invented by the Applicant and that scheme that generates highly accurate reconfigured stereoscopic image pairs from original stereoscopic image pairs. The technique is particularly efficient and requires low resources to generate high-quality images. This will now be described.
  • the image tube model can be considered an abstract dataset defining in virtual terms spatial relationships in the image, for example the spatial relationship of things in the image, such as the spatial relationships of pixels (particularly left-eye pixels and corresponding right-eye pixels -that is pixels illustrating a same part of the scene in the left-eye image and the right-eye image) in the image.
  • the virtual terms of this spatial relationship may be for example, the eyes of the viewer and position of a perceived point and position of pixels on a screen and the spatial relationship may be defined in terms of lines intersecting eyes and pixels on the screen and/or intersection points of these lines and each other, the eyes and/or the screen, which in virtual terms may be a virtual screen (and virtual eyes) according to intended viewing parameters.
  • a new disparity map between the re-formatted stereo sub-images can be obtained thus the new stereo image pair can be synthesized.
  • a post-processing containing occlusion processing and distortion control is then applied to the synthesized stereo image pair to obtain high-quality re-formatted stereoscopic contents.
  • the shooting conditions of a 3D scene are often strictly selected according to the view environment under which the 3D scene will be rendered. This is relatively easy for 3D movie production, since the environments of different theaters are similar, or at least, do not vary greatly. Therefore, a 3D movie shot according to an optimized estimation of the viewing parameters typical in a theater environments can be shown in most theaters, and provides audiences with satisfying 3D experience. But for the 3D contents made for broadcasting industries, this 3D production strategy will not work.
  • the viewing parameters may be quite different. They may be a home theater, or a bed-room with a small 3DTV, a mobile 3D display device such as mobile DVD player, or even a head mounted display with separate displays (or portions of a display) for each eye, etc.
  • the 3D contents made for a specific view environment may be rendered in a quite different view environment under different viewing parameters.
  • view-environment violation When view-environment violation occurs, audiences may suffer serious visual discomfort and visual fatigue, e.g., headache, dizziness, eye-strain, etc. Much research has been done to analyze the causative factors of visual discomfort and visual fatigue, and recent research shows that view-environment violation is an important factor that degrades the 3D experiences. To build a robust 3D broadcasting system, the view-environment violation problem must be solved.
  • Depth-image based render techniques promises to make the 3D contents compatible independently to view-environments, but the techniques are not currently available in multimedia markets due to 1) the technical obstacles in depth acquisition, and 2) the technical obstacles of view synthesis. In practice, most 3D contents are made for a specific view environment.
  • a reasonable solution of the view-environment violation problem is to convert the 3D contents made for a view environment to a new 3D contents which are suitable for being rendered in a different view environment.
  • Figure 1 shows this solution for broadcasting a 3D contents produced for a specific view environment. Note that in the receiver ends, view environments are different.
  • the 3D contents of real -world scenes are firstly obtained by 3D production 1105, which usually consists of stereo imaging systems and relevant post- processing.
  • 3D production 1105 usually consists of stereo imaging systems and relevant post- processing.
  • the configurations of the stereo imaging system including both the inner parameters such as the focal length of cameras and the external parameters such as the base-line between the cameras and angle of convergence, are carefully selected according to the desired view-environment.
  • the 3D contents are distributed via diverse channels, for example, by air, or by DVD, etc... using, in this example, existing distribution technology.
  • a stereoscopic content reformater 1110 may be integrated into each rendering device.
  • the 3D rendering device can be configured according to the actual view environments, thus the received 3D contents can be re-formatted to the new contents which are suitable for rendering in the current view environment. With the stereoscopic reformater, the received 3D contents can be well rendered in different view environments, and bring high-quality 3D experiences to different audiences.
  • the perceived visual quality of 2D contents are mostly related to image quality For example, the noise level, the sharpness, and the contrast.
  • a high-quality image scaler can obtain satisfying re-formatted 2D contents for divers rendering devices.
  • the perceived visual quality is not only related to the image quality of each sub-image in a stereo image pair, but also related to the stereoscopic distortion, and more important, related to the perceived depth.
  • the image quality of each view of a stereo image pair affects the 3D visual quality.
  • the quality of a 2D image may be degraded by following artifacts: 1) noise, 2) block artifacts caused by codecs, and 3) unnatural blur.
  • Noise existing in the two views can degrade the quality of 3D contents, however, the human vision system (HVS) has higher tolerance to noise in 3D contents than in the 2D case. Therefore, such noise affects the 3D visual quality less.
  • Block artifacts caused by codecs may degrade the 3D visual quality, and eventually lead to visual discomfort such as eye-strain.
  • the most important causative factors of visual discomfort in 3D experience is the unnatural blur.
  • Image blur may greatly affect the HVS in stimulating accommodation and convergence thus lead to visual discomfort in some strong motion image regions.
  • Depth is a unique feature of 3D contents comparing to 2D contents, and it is also a very important causative factors of visual discomfort in 3D experiences.
  • a depth comfort zone in terms of spatial characteristics, in terms of relative spatial characteristics (e.g. relative to the viewer and the screen) or in terms of viewer-perspective with consideration for the limit of perceptual depth and/or the depth of focus.
  • a viewer can obtain comfortable 3D experience and set the comfort zone so.
  • the application rules of the comfortable zone of the perceptual depth which suggest that the perceived depth should be always within the comfortable zone. Accordingly, it the proposed re-formator may ensure that all re-generated depth should be within the comfortable zone of the current view-environment.
  • VDF Visual Discomfort and/or Fatigue
  • a 3D re-formator recreates a stereo image pair according to the current view environment, some of its behaviors are similar to stereo imaging. So some geometrical distortion caused by stereo imaging may also occur in re-formatting processing.
  • the proposed 3D re-formator minimizes the geometrical distortion.
  • Another advantage of the proposed re-formator is that it is robust to occlusion and recovering. Since the depth may be greatly changed after re-formatting processing, image occlusion and recovering regions may also changed. This is similar to the case that a viewer will see different occlusion parts of two objects in different locations when the viewer moves toward or backward to the objects in real -world. Unnatural occlusion will decrease the depth perception and even lead to visual discomfort.
  • a traditional 2D image represents a real-world scene by pixels.
  • Each pixel is a sample of a 3D real-world point which is projected to the imaging plane of a digital camera by an imaging system.
  • a real-world point can be well described by the corresponding pixel in an image with three elements, namely, the coordinate of X-axis, denoted x , the coordinate of Y-axis, denoted ⁇ , and the intensities of the pixel, denoted i , which is usually a vector consists of the illuminance intensity and the chromatic intensities.
  • P and P are the pixel of an image and the real-world point, respectively, we have
  • X , Y , and Z are the coordinates of P in real-world coordinate system.
  • a real-world point P is represented by two corresponding pixels, one belongs to the left view, denoted P , and the other belongs to the right view, denoted P « .
  • P the left view
  • P « the right view
  • the difference between the locations of L and P « can be considered that only exists along the X-axis.
  • is the disparity between Pi and P « in pixel-wise, for the convenience of discussion, we call &x the disparity; L and 1r are the intensities of and P « , respectively. Theoretically, should be equal to 1r . However, in practice, they are usually not equal to each other because of the difference between the characteristics of the cameras and the calibrations.
  • the disparity ⁇ in (2) is the most important factor for depth perception in 3D experiences. It generates the screen parallax, denoted P , according to the parameters of a given view space, and P lead to the perception of the depth information via human vision system
  • Figures 2 A and 2B sho When view a 2D image, each perceived point is located on the convergence plane, namely, the screen plane, therefore, no depth information can be perceived.
  • the essential of 2D re-formatting processing is to determine a mapping rule which maps a perceived spatial point P on the screen 1 to a corresponding spatial point P' at the new location on screen 2. Since both of P and P' are located on the convergence planes, the perceived depth of P and P' are v and v , respectively. Without knowing ⁇ v of and the desired ⁇ v of a high-performance image scaler can complete the 2D re-formatting processing very well.
  • the depth of a spatial point is related to both the pixel-wise disparity and the screen parallax.
  • the pixel-wise disparity is related to the generation of the stereo image pair, namely, the parameters of the stereoscopic shooting system, including the baseline between the two cameras, the arrangement of the two cameras, the distance between the cameras and the objects of interest, the focal length of the camera, and the camera resolution, etc.
  • the screen parallax is related to both of the parameters of shooting systems and the parameters of the view space where the stereo image pair is rendered, as shown in (3). Further more, in addition to the screen parallax, the perceived depth of a spatial point is also greatly related to many view space parameters, including the view distance, and the human eye separation. We will discuss this in detail later.
  • the perceived depth may change seriously due to the changes of the view-space parameters.
  • a way to render the stereoscopic contents shooting for in ⁇ v is to 1) obtain the disparity map between ⁇ and ⁇ R of each stereo image pair, and 2) apply some adaptations to the disparity map to avoid the perceived depth resulting in visual discomfort.
  • this disparity-adaptation based strategy is not reliable for generating appropriate depth.
  • the relationship between the disparity (or screen parallax) and the perceived depth is not linear. It depends on the parameters of the current view space, even human eye separation may significantly affect the perceived depth.
  • disparity-adaptation based algorithms linearly adjust the disparity maps.
  • the linearly changed disparities will lead to non-linearly changed depth map. Serious perceived depth distortions may occur when rendering the adapted stereoscopic
  • Figure 3 models the perception of a spatial point in different view spaces, note that in this report, the spatial point perception includes the perceptions of both the depth information
  • the proposed re-formatting strategy may involve finding a spatial point transition rule to move a spatial point , which can be comfortably perceived in ⁇ ° , to an appropriate position
  • FIG. 4 shows the main structure of the proposed re-formatting algorithm, according to one exemplary embodiment.
  • the proposed stereoscopic content re-formatting algorithm reformats a given stereoscopic video, which is shot for a view space , to a new stereoscopic sequence, which can be rendered in a new view space for comfortable 3D experiences, with four steps.
  • the perceived depth of each stereo image pixel pair that can provide audiences with comfortable 3D experiences in ⁇ v are firstly computed, and then a new disparity map which will achieve the new perceived depth map of ⁇ v is reconstructed.
  • the new disparity map the two views of the image pair in ⁇ v can be easily synthesized to form a new stereo image pair.
  • the proposed image-tube transforming method is sent to the post-processing to further improve its visual quality, including occlusion and recovering processing, distortion rectification, etc.
  • the perceived depth comes from screen parallax, and it plays a pivotal rule in stereoscopic re-formatting tasks.
  • ⁇ p is the perceived depth
  • is the eye separation
  • P is the screen parallax
  • is the view distance, namely, the distance between the eye plane and the screen plane in a view space.
  • the expression in (10) is convenient and direct to present depth information, however, it only can express the depth information. This is far away enough for re-formatting tasks.
  • (10) only expresses the static depth information, yet in most re-formatting tasks, the perceived depth in different view spaces are not the same, i.e., the perceived depth is a dynamic factor in different view spaces.
  • Second, (10) does not express the relationship between the perceived spatial points and different view spaces.
  • a sample example is that (10) cannot express how to adapt ⁇ to obtain the similar ⁇ p in a new view space.
  • the third, (10) is not convenient to reconstruct a specific new depth in a new view space, since the relationship between P and ⁇ p is non-linear.
  • an image tube can be considered as a passage that light travels between different points. It represents both the static parameters of a view space, e.g., the view distance, and the dynamic relationship between different parameters of the view space, e.g., how the perceived depth changes if the screen location changes.
  • a stereoscopic image including both its intensity information and its 3D spatial representations, can be conveniently expressed by a bench of image tubes. With the image tubes, potential changes of the perceptions of stereoscopic contents can be conveniently obtained from image tube transforming.
  • an image tube is defined as comprising a line family, which comprises three lines, namely, 1) the projective line between the left eye and the spatial point, 2) the projective line between the right eye and the spatial point, and 3) the line presenting the rendering plane.
  • a line family which comprises three lines, namely, 1) the projective line between the left eye and the spatial point, 2) the projective line between the right eye and the spatial point, and 3) the line presenting the rendering plane.
  • Figure 5 shows an example of the image tube of a point P in a view space. Note that in Figure 5, the solid lines represent the real image tube , and the dash lines represent a virtual tube ⁇ , which corresponds to after image tube transforming. From (11) and Figure 5, we can see that with the proposed image tube conception, the perceptions of stereoscopic contents in a given view space and any dynamic properties of 3D content perception can be easily presented.
  • Disparity is an important factor for perceiving depth in a view space.
  • disparity estimation is used for generating high-quality re-formatted stereoscopic contents.
  • the precisely estimated depth will generate the high-quality image tube, and eventually, after image-tube transforming and depth re-construction, we can get high-quality re-formatted contents.
  • LSO local stereoscopic optimization
  • GSO global stereoscopic optimization based method
  • the proposed re-formatting algorithm employs an efficient, hardware friendly, and robust disparity estimation method but any suitable disparity estimator may be used.
  • the quality of the estimated disparity maps may affect the quality of the re-formatted stereoscopic contents as errors in the disparity map may be propagated to the later processing, and finally result in visual impacts in the final outputs.
  • Occlusion parts in a stereo image pair are the image regions which are visible in one view but invisible in the other view. Therefore, the disparities of the corresponding pixel pairs existing in occlusion regions cannot be directly determined. Without occlusion-adaptive algorithms, the estimated disparities of the occlusion regions are determined by video contents and searching ranges instead of local-optimum criterion. Mistakenly estimated disparities may lead wrong perceived depth prediction, and eventually lead to visual impacts in the final outputs.
  • occlusion adaptive re-formatting strategy To improve the quality of the re-formatted stereoscopic contents, we propose an occlusion adaptive re-formatting strategy in this proposal.
  • the main idea of the proposed strategy is to detect the occlusion regions between the two views first, then assess the reliability of the estimated disparities according to the obtained occlusion mask.
  • the disparities belonging to the occlusion regions are regarded as non-reliable disparities.
  • the non-liable disparities are refined by disparity refining processing.
  • the solution described herein may be used for reformatting to convert between view spaces but also to transform an image within a same view space, for example to correct for uncomfortable filming, to tailor the 3D experience to individual preferences, to correct errors in a 2D-to-3D converted film, to adapt a film to limitations/increased capabilities of a particular display, or merely to adjust depth for flattening/deepening or otherwise altering an image or portions thereof.
  • the first step of image tube transforming is to perform spatial point translation to move each spatial point in ⁇ ° , where k is the index of the pixel pair of the input stereo image pair, to the corresponding positions in .
  • k is the index of the pixel pair of the input stereo image pair
  • 2D linear scaling we may adopt 2D linear scaling to complete the job. For the purposed of this example, we assume that an image is fully displayed within the effective screen size and that no cutting or extending (e.g., add black bars around the image boundaries) is applied to the image to be displayed. To a spatial point P
  • the second step of image tube transforming is depth stretching, which will change the perceived depth in ⁇ " .
  • 3D contents are usually produced for specific view spaces. Therefore, the stereoscopic contents made for ⁇ ° may generate serious visual discomfort in if the two view spaces are quite different.
  • a widely adopted way to solve the problem is to adapt the disparities.
  • the perceived depth is not only related to the disparities but also related to view space parameters. When the view space changes, the perceived depth changes non-linearly. Therefore, directly adapt disparities usually generate relatively high depth distortions.
  • the comfortable zone - 0 ⁇ 2 ⁇ determines the limits of the perceived depth of a view space.
  • the minimum depth for comfortable 3D perception is expressed as the comfortable zone for foreground, denoted f
  • the maximum depth for comfortable 3D perception is expressed as the comfortable zone for background, denoted * .
  • FIG. 7 shows the proposed comfortable zone adaptive depth transforming algorithm.
  • the comfortable zone of denoted ' is firstly computed according to the parameters of ⁇ v .
  • the minimax of the perceived depth of ⁇ is located, denoted ⁇ f and ⁇ b , representing the minimum depth (the foreground) and the maximum depth (the background) of " ⁇ 0 ⁇ , respectively. Then, we check if the original
  • the new depth of ⁇ in ⁇ v can be a f d o (k) foreground
  • the third step of the image tube transforming algorithm is to new tube computation in which obtain a new image tube according to the parameters of ⁇ v and the transformed depth ⁇ d v (k) ⁇ 0 ⁇ 6 ⁇ a ⁇ me mc j ex j m ma y 3 ⁇ 4 e different from k , which is the tube index in i io , since the proposed algorithm also takes the case that the image resolution is also changed into account. When the image resolution is changed, the number of image tubes in different view spaces are different. In this report, we compute with image tube interpolation.
  • * _ 1 2,..., ⁇ M - g ⁇ e numDer 0 f me neighboring tubes of " ⁇ .
  • the disparity reconstruction is relatively simple.
  • the screen location of each pixel of the left view in has been determined by the given parameters of and the desired image resolution, denoted ' ⁇ ' .
  • the screen location of each pixel of the right view in ⁇ v denoted ⁇ r ⁇ Xr ' ⁇ v ⁇
  • m Since the human eyes are generally located on the X-axis of and P R will have the same Y coordinates. (Note that the prior computation of a depth map is not necessarily required to reconstruct disparity.)
  • a stereo image pair L and R shot for view space ° can be re-formatted as a new stereo image pair L and with the proposed algorithm described previously.
  • the direct output of the algorithm is preferably processed to remove visual artifacts.
  • the artifacts may be mainly caused by two factors. One is the occlusion between the left and the right views, and the other is the distortions caused by image tube transforming.
  • the occlusion parts between the two views of a stereo image pair are a challenge for real-world applications. Since the pixels of the occlusion parts only exist in one of the two views, the disparities of the pixels cannot be directly estimated from disparity estimation using traditional disparity estimation techniques. However, without the well-estimated disparities, the visual quality of the occlusion parts in the re-formatted image pairs will be lower. Therefore, it is a benefit for the post-processing in the proposed algorithm to properly address occlusion region refining.
  • the comfortable is also relatively big, Z-scaling factors may be relative small; yet if the view distance is small, and the comfortable zone is also small, Z-scaling factors may be relatively big.
  • the Z-scaling factors are different for the foreground and the background pixels.
  • to maintain the comfortable zone we need to increase the depth of some foreground points, i.e., adopting the Z-scaling factor that is bigger than 1.0 for the foreground points, and decrease the depth of some background points, i.e., adopting the Z-scaling factor that is smaller than 1.0 for the background points.
  • scaling factors in X-Y plane is relatively big, and since small view distance is usually adopted to the small screen, the Z-scaling factors will also be relatively big; and for the big-size screen in , the X-Y scaling factors are relative small, and since big view distance is usually adopted to big screens, the Z-scaling factors will also be small. It is similar to the case that a small screen is used in ⁇ 0 . This will greatly maintain the naturalness of the stereoscopic contents. Second, different scaling factors in different direction may result in some unnaturalness. However, it has been shown that in 3D experiences, the tolerance to unnaturalness of the human vision system significantly increased companng to the 2D experiences.
  • the image tube of each corresponding pixel pair is computed from the geometric properties of the current view space, including the view distance, screen size, and the natural characteristics of audiences.
  • a stereo image pair is converted to an image tube set in the current view space.
  • the obtained image tube set is transformed to a new tube set according to the parameters of a new view space.
  • the transformed image tube set may have different number of tubes from the original tube set depending on the configuration of the new view space.
  • the non-linearly changed screen parallax of each corresponding pixel pair is computed from each transformed image tube, thus the pixel-wise disparities can also be obtained from the new view space configuration.
  • the new disparity map With the new disparity map, the new stereoscopic image pair can be synthesized.
  • the quality of the synthesized image pair is refined by a set of post-processing, which contains occlusion processing, distortion refining, and sharpness enhancement if it is necessary.
  • the algorithm proposed can be implemented in software code. Additionally, this algorithm has specifically been designed so as to be readily implementable in a hardware embodiment with minimal use of costly resources. It is designed to be executable quickly enough to be executed in real time in practical implementations.
  • the above algorithm is already suited for adaptation to a view-and-disparity-map format or a view-and-depth-map format or for a 2-views- and-(l or 2)-depth-map.
  • we may use such formats to replace the disparity estimation step by obtaining directly the disparity map of the image, allowing us to compute image tubes therefrom.
  • the above algorithm can take advantage of such formats to reduce the computational burden while the remainder of the algorithm, as described above, allows the generation of stereoscopic image pairs (or any other number of different views) tailored to desired viewing parameters or with a desired depth effect.
  • the image tube transformation may be made to respect several different kinds of constraints.
  • the image tube transformation may set hard limits on the maximum and/or minimum depth of objects in the image
  • the image tube transformation may be used to ensure a certain proportionality between the resizing in the X-Y plane and the resizing in the depth direction.
  • an appropriate modification of the depth can be selected based on the X-Y scaling undergone.
  • knowing the image tube in the original viewspace; for any particular X- Y scaling factor and starting depth the proper scaling effect in the Z-direction can be ascertained using known computational methods or by referring to a lookup table comprising pre-computed values.
  • the algorithm provides proportional image reformatting with a safeguard for view environment violation whereby when the transformed image tubes would images to points going beyond the comfort zone, the image is further scaled in the Z-direction before proceeding to re-construction and view synthesis.
  • the image tubes indicating points violating the view-environment may simply be transformed to bring the points back within the comfort zone. But this, however, may lead to an awkward flattening of the image about the edge of the comfort zone.
  • the whole set of image tubes may be scaled by depth together.
  • the scaling may be non-linear (e.g. logarithmic) such that the image tubes in violation are subject to the greatest change in depth while those that represent points that are closest to the plane of convergence are the least modified.
  • the object of interest in a video is usually featured at or near the plane of convergence, this object will be the least affected by the scaling, while remaining proportional. It is to be understood that Z-scaling to avoid view environment violation may be done even in cases where Z-scaling for proportionality isn't performed.
  • the reformater allows for reformatting according to different criteria, e.g. proportionality, comfort zone adherence, and (as will be seen later) child safety.
  • scaling can be performed to reconcile the different criteria where reformatting according to one criterion would cause a conflict of another criterion, however such scaling may cause distortion.
  • a reformater is implemented in a viewing system allowing for a selection of those criterions (e.g. proportionality, comfort and child safety) that are desired.
  • a user may be provided via a graphical user interface implemented by a processor in the viewing system, a button interface or elsehow a choice of which of the criteria the user wants the reformater to base reformatting on.
  • An additional option can be no reformatting, which may be selected separately or merely by virtue of not selecting a criterion.
  • the viewing system is a television
  • the user may access a menu using a remote control, the menu providing a set of visual indicators of a reformatting criteria and a selection indicator, e.g. in the form of a radio button.
  • the user may select one criterion and by providing an input, e.g.
  • the user may indicate that this is the criterion according to which the reformater is to reformat.
  • the processor sets the parameters according to which the reformater is to operate and causes the reformater to operate in the manner described herein according to the constraints.
  • the user may select a hierarchical order for the criteria, each causing a different transformation of the image tubes in inverse order, that is, the most important one being performed last and the least important one first.
  • the reformater will first transform the image tubes according to the view space, then transform them to fit the comfort zone, then transform them to maintain proportionality and finally transform them to ensure child safety.
  • no scaling is perform to reconcile the different criteria.
  • the last transformation is sure to be applied throughout the image, but the others may have had their effects modified by subsequent transformations.
  • the reformatting system is used to correct for such near-frame problems after capture. This may be implemented in real-time in a display system or in a post- production setting. It may allow for more liberal capturing (e.g. filming) and for more satisfying transition from very large screens to more modest ones.
  • the image tubes are analyzed to identify the presence of points near the edge that would have a large depth in the new viewspace. If such points are found, the set of image tubes may be transformed to shift in the Z-direction to bring the points near the edges closer to the plane of convergence.
  • the scaling may be linear or non-linear. In this case, the scaling may be applied (e.g.
  • reformatting may be done, not only to accommodate a new viewspace ⁇ , but also to accommodate other modifications of the image as may be, for example, indicated by a user at an input.
  • the reformater is implemented on a viewing system comprising a display and a user input though which a user may input reformatting parameters. Reformatting parameters may include a shift in the Z-direction to implement a so-called reformater Z-shift.
  • the viewing system is a television and the input is provided by way of a graphical user interface through which the user may provide input using a remote control.
  • the user may input an indication of a desire to shift in the Z-direction, for example by first selecting a Z-shift option in a menu and using arrow buttons to indicate a shift inwards or outwards from the screen.
  • the reformater transforms every image tube to impart the change of depth required.
  • the subsequent image synthesis reconstructs the images and the result is a 3D scene that moves inwards and outwards of the display according to user input.
  • the user may input a Z-shift value using a numerical interface.
  • zooming is problematic in 3D for reasons that can be understood in light of the above description. While in 2D zooming merely involves scaling the image in the X and Y direction (in the case of a zoom-in, this may involve abandoning or cropping out parts of the image that would, once scaled, fall outside of the edge of the view area/display, in the case of a zoom out, it may result in an image smaller than the view area/display and thus a black or empty border around it). However, as we know such manipulation causes non-linear shift in the resulting depths in a 3D image pair. Merely resizing a stereoscopic pair of images is therefore not satisfying.
  • the reformater serves to implement a 3D zoom.
  • the so-called reformater 3D zoom allows for proportional 3D zooming.
  • the reformater 3D zoom is implemented on a viewing system comprising a user input.
  • the reformater follows the same steps as described in the main embodiment with some modifications.
  • the image tubes are transformed to respect the new viewing parameters (viewspace) but are more over transformed to reflect the zoom in the X-Y plane.
  • the image tubes are modified to reflect a proportional change in depth.
  • the scaling of the image to use with the disparity map is altered by the amount as the scaling in the X-Y plane performed on the image tubes to reflect the transformation resulting from the zoom.
  • the zooming action is a zoom-in
  • some of the original image will fall outside of the frame of the display and consequently not be displayed.
  • the excluded portions of the image can be removed from the originating images during the X-Y scaling in image reconstruction, such that although all the image tubes were transformed according to the zoom, only those that pertain to points that will be visible in the new images are actually used in synthesis according to the above method.
  • some the image may shrink to less than the size of the display such that an empty (black) border surrounds the image.
  • the zooming operation is performed on the totality of the image tubes therefore as the image is zoomed in or out, the whole visual information is available if it can fit in the display.
  • X-Y zooming of the originating images may be used in advance in order to compute those image tubes which will remain and which are worth transforming.
  • comfort-zone based reformatting as described above may further be applied to ensure that the zoomed image suits the viewing parameters it is viewed under.
  • the end-result of zooming in may cause objects to move outwards towards the viewer up to the limit of what may comfortably be seen and then squish together at that depth. IT will be appreciated that this and other interesting effects may be applied by transformation of the image tubes.
  • 2D TVs typically have zoom functionality for adapting to different input image sizes.
  • a 3D TV performs similar zoom functionality but using the reformater 3D zoom when the input feed is 3D.
  • a reformater 3D zoom may be implemented elsewhere than on a viewing system, for example in postproduction equipment. Likewise it will be appreciated that the reformater 3D zoom may also be operated non in direct response user input. 3D zooming may be used, for example automatically, to correct for detected problems around the edges of a display (described above) or in other non-user-input directed manners.
  • the reformater 3D zoom and reformater Z-shift provide powerful 3D manipulation tools with minimal computational burden. Since these tools rely on the lightweight reformater design provided above, they are, like the reformater, implemented in hardware for real-time application in a display system chip. They are, of course also be implemented in software for a variety of applications including software video processors, video editing tools and post- production equipment.
  • the combination of reformater 3D zoom and reformater Z-shift enables a user to blow up and shrink down and move into and out of a 3D image at will. This is combined with X and Y direction translation allows an unprecedented full range of motion to the user.
  • X and Y direction translation is performed by translational module that causes stereoscopic image pairs to shift together in the X and Y direction by altering the display timing of their pixel lines according to known methods.
  • the viewing system is a tablet comprising a 3D viewing display displaying 3D imagery to the user.
  • the user interacts with the viewing system by a user interface, in this case a touch screen interface.
  • a user interface in this case a touch screen interface.
  • the user is able to zoom into and out of the 3D image, Z-shift in and out of it, and move it up, down, left and right.
  • the well-known pinch zoom is used to indicate that zoom in or out of the 3D image is requested and the image is zoomed by the reformater 3D zoom accordingly.
  • Dragging a single finger over the touch schreen indicates a desire to cause a shift in the image in the dragging direction and the image is moved accordingly while a two finger drag indicates a request to Z-shift into (dragging in one direction) and out of (in the opposite direction) the 3D image and the image is Z-shifted by the reformater Z-shift accordingly.
  • other inputs may be used to indicate a request for Z-shifting such as twirling a single finger in one direction (e.g. clockwise) to shift inwards and twirling it in the other (e.g. counter clockwise) to shift outwards.
  • Known multi-touch technology may be used to implement this input interface.
  • the viewing system is a 3D television and the user interface is implemented using a combination of a remote control and a GUI.
  • the remote control may comprise a 3D zoom in button, a 3D zoom out button, a Z-shift in button, a Z-shift out button, a move left button, a move right button, a move up button, a move down button or any subset thereof.
  • the buttons are used by the user to input a request to zoom/shift/translate the image accordingly.
  • a processor in the viewing system receives the request and causes the reformater 3D zoom, reformater Z-shift or translational module to perform the requested transformation. The processor then processes the resulting images for display and displays them on the display of the viewing system.
  • stereoscopic content capture for applications such as remote inspection, remote guidance and entertainment, typically employs a pair of spatially rotated cameras spaced apart to provide an observer with views of a scene from two points of view.
  • Left and right stereoscopic image pairs are acquired and presented to the observer as stereoscopic content in the form of overlapped image fields of view which provide comfortable stereoscopic observation of un-occluded objects within a limited region.
  • the parallax between the corresponding images on the stereoscopic display causes the observers gaze to converge at points behind or in front of the plane of the stereoscopic display where the observer perceives the three dimensional image.
  • stereoscopic content presentation or playback typically employs a stereoscopic display having a size and resolution.
  • comfortable stereoscopic perception depends on the display characteristics. Due to a limited resolution of the display, objects exhibiting a parallax less than can be represented on the display due to pixel size are perceived as very far, or at infinity. Conversely, objects exhibiting excessive parallax may suffer from extreme disparity and be perceived as double.
  • depth perception depends on a variety of viewing parameters. Viewing parameters affect the perception of depth for a given 3D program. These may include the size of the display and distance of a viewer from the display. They may also include horizontal angle of a viewer relative to the display, height of the display relative to the viewer, and any other measure of position of the viewer and the display relative one another. They may also include more subtle consideration such as viewer eye spacing and viewer eye conditions such as far/near sightedness prescription.
  • the depth of field of the observer's eyes covers comfortably only a finite distance in front and behind the display.
  • the region of comfortable stereoscopic observation which can be displayed on a stereoscopic display may not cover the entire region over which the left and right stereoscopic pair image overlap. This has been described above.
  • the parallax of objects in the scene must not exceed a maximum parallax. That is to say, the disparity (the distance between equivalent points in the left and right images presented to a viewer) must not exceed a certain maximum in either direction (overconvergence or divergence) for comfort.
  • the maximum/minimum amounts depends upon the viewing parameters, as a disparity of X pixels will represent a different distance on different screens and a disparity of X cm will demand a different angle of convergence of the eyes at different distances.
  • Stereoscopic content presentation of a display having a different size than the size for which the content was intended introduces a scaling factor to parallax which in turn caused a change in the perceived depth of objects in the scene.
  • the observer may only fuse the stereoscopic image pair with effort straining the eyes. For objects outside the depth of field of the observer's eyes, image fusion is impossible leading to double vision. Undue eye strain is experienced when objects have excessive parallax.
  • Figure 10 illustrates a stereoscopic display displaying an object A in 3D.
  • the projection plane represents here the convergence plane which is the plane occupied by the screen in a typical (theater projection screen, LCD or plasma) screen-based stereoscopic display.
  • the stereoscopic display shows two images of the object A, one visible only to the left eye and one visible only to the right eye. These two images are not in the same location such that a viewer looking at the stereoscopic display sees the object A in two different places with his left and right eyes.
  • Figure 10 shows the eyes of a viewer (at constant eye spacing c) according to three different positions of the viewer: a mid-range distance (Distance A) from the screen, a far- back distance (Distance B) from the screen, and an up close distance (Distance C) from the screen. From each position, the viewer's left and right eyes see the object A at a different angle. The dotted lines joining the left and right eyes to the left and right perspective images of object A represent the angle of view of the object for each eye. The depth of the object A is perceived at the point where these two lines meet.
  • Figure 11 illustrates a large stereoscopic display and a small stereoscopic display displaying a same image comprising an object B to a user at a same distance from the display. On the smaller display, the left and right perspective views of object B are closer together, resulting in the appearance of object B being closer to the display. Images having been captured for viewing parameters other than those under which they are viewed may appear more flat in the Z (depth) direction.
  • the left and right perspective views of object B are further apart because the overall image is larger, which leads to a perception that object B is further away from the display.
  • Images having been captured for viewing parameters other than those under which they are viewed may appear to have greater depth variations in different circumstances.
  • the size of the screen may affect the perceived 3D effect.
  • the perceived position of the object if too close, may be outside of the comfortable range of comfortable for a viewer's eyes, leading to discomfort or double vision.
  • FIG. 12 illustrates three viewers in three different positions relative to a stereoscopic display viewing a same object. Each perceives the object at a different point in space.
  • the perceived position of an object was shown as varying as a function of the display or viewer.
  • the perceived position of an object also depends on the manner in which the object was stereoscopically captured.
  • Parameters at capture affecting the parallax, 3D effects, or generally the depth perception are capture parameters.
  • camera pairs are positioned in such a way as to capture left and right eye perspectives of images as they would appear to a viewer viewing the display on which it will be displayed at a particular position. Camera positioning and orientation are examples of capture parameters.
  • the choice of spacing (separation) between the cameras of the camera-pair and the angle of convergence affect the stereoscopy (3D effect) of the resulting image at a particular viewing parameter.
  • the capture parameters may also include focus and the position and/or orientation of the cameras other than the spacing and angle of convergence. For example, if the cameras have been imperfectly aligned and have a certain vertical offset or an uneven and/or vertical angle relative one another, these may affect the depth perception for the viewer (e.g. in a negative way).
  • Figures 14 and 15 illustrate some of the effects that can result from varying the parameters.
  • Figure 13 shows a stereoscopic display showing two left-right vie-pairs of an object D at two different position ("placements") on the display.
  • the two placements may be the results of different capture parameters. Filming a scene with first capture parameters would result in object C having placement 1 on the display. In this placement a viewer at the position labeled "viewer 1" would perceived object C at the position illustrated. If the viewer where to move to the location labeled "viewer 2" in the Figure, we know from our earlier discussion that object C would appear in a different place in space. However, where the image originally captured with a second different set of capture parameters that resulted in object C being located at placement 2, shown, the perceived position of object C would be the exact same for this viewer as was for the first viewer with the first set of capture parameters.
  • the two viewer position may represent different viewing parameters.
  • the image tubes may be transformed as described above to provide a placement of object C at placement 2 where it would originally have been showed (without reformatting) at placement 1, for example.
  • Figure 14 shows a single viewer viewing an object D on a stereoscopic display. Two placements of object D are shown, according to two different capture parameters. Using first capture parameters to capture an image comprising object D results in the first placement of object D on the display, which results in object D being perceived by the viewer as being at a first location, close to the display. Using second capture parameters to capture the image results in a second placement of object D on the display, which results in object D being perceived by the viewer as being at a second location, further from the display. Thus in this example, the choice of capture parameters affects the perceived depth of the object D.
  • capture parameters affect the perceived location in 3D space of the various objects in a stereoscopic image.
  • the capture parameters are computed to provide the right depth perception for given viewing parameters.
  • movie programs will be filmed using camera positioning tuned specifically for a central viewer watching it on a movie theater screen.
  • Stereoscopic system camera positioning for the acquisition of stereoscopic content is dictated to a great extent by the display on which the stereoscopic content is intended to be displayed in order to accommodate a range of viewer interocular distances (the distance between the eyes).
  • the extent of the region of space which can be displayed in three dimensions without causing undue eye strain to the observer can be controlled by the acquisition system's geometrical (and optical) parameters. Therefore stereoscopic (video) content is captured for output/playback on an intended stereoscopic display having a particular size.
  • the cameras of the stereoscopic content acquisition system are spaced an initial distance apart and oriented towards the object of interest, also known as toe-in or angle of convergence.
  • the 3D effect may be skewed such that objects appear unnaturally flattened or stretched in the depth direction or otherwise distorted.
  • Spheres may appear deformed like a rugby ball or objects may appear as flat sheets with varying depths. Such may be the effect of viewer positioning in a sub- optimal place.
  • the gradation of the depth perception may be quite pronounced.
  • Reformatting can be performed, for example, by applying a geometric transformation to images captured at first (original/real) capture parameters to generate transformed images approximating or simulating the images that would have been generated at second (other/simulated) capture parameters.
  • Such a transformation can be used to change stereoscopic images that were captured with capture parameters intended for viewing parameters other than those at which the stereoscopic images will actually be viewed, in such a way as to recreate images optimized for the real viewing parameters.
  • This may simulate images that were captured with capture parameters optimized for the real viewing parameters.
  • a movie filmed for the big screen may be reformatted to be optimized for viewing on a screen typical of home 3D TV's by a viewer at a typical at-home viewing distance.
  • Reformatting can also be performed to correct mistakes or imperfection in the original capture parameters. For example, if it is determined that the camera positioning (e.g. angle of convergence or spacing) was not ideal for the intended viewing parameters for a particular video, the ideal capture parameters may be found, as discussed below, and the images of the video may be reformatted to transform the images of the video such that they are as if they were captured at the ideal capture parameters.
  • the above described image tube- based solution can be used to modify the image in the view space it will be viewed in to correct the effects of improper capture of the images. This is done by selecting a transformation of the image tubes such that the resulting reformatted image respects the comfortable viewing zone of the viewer. Additional intelligence can readily be built into the system so as to modify the location of objects in the 3D viewing space, e.g. to respect certain three dimensional proportions. Indeed, since the 3D location of a point in the view space can be readily derived from the image tubes, and since the image tubes, as shown above can be transformed to adopt virtually any other 3D location, it follows that an image can be reformatted not only to respect a comfortable viewing zone but also for many other purposes as will be described below.
  • a program may be filmed with deliberately non-optimized capture parameters.
  • the camera placement e.g. angle of convergence and/or spacing
  • the video so captured is then reformatted such that the capture parameters (now simulated) are optimized for each scene.
  • Optimal capture parameters for each scene may be determined in any suitable manner. For example an expert may compute the optimal camera positioning using a cameral placement algorithm or other known techniques on site, while being relieved of the necessity to actually set the cameras accordingly.
  • test patterns may be implemented into the scene (in a manner ideally invisible or unrecognizable to the viewer) to be used for identifying distortion.
  • test patterns of a single frame may be implemented within a video and, provided that the frame rate is high enough, this would be generally invisible to the viewer.
  • a test pattern may be included at the beginning or end of every scene, to be used to compute actual and/or optimal capture parameters for the scene and to thereafter be removed from the video during editing. Test patterns will be discussed further below.
  • the ideal capture parameters vary (e.g. linearly) during a scene, they may be computed at several discrete points in the scene and a function of change may be computed such that for each frame (or for each discrete sets of any number of frames, if it is desired to change the capture parameters in larger steps than single frames) the optimal capture parameters may be found from the computed function and the frames (or set thereof) may be reformatted accordingly.
  • the ideal capture parameters may be computed for each individual frame.
  • a program is captured using capture parameters that are not necessarily ideal, but may be for example, simpler, easier and/or more cost effective or accessible to implement.
  • Ideal capture parameters are then determined.
  • the program then undergoes a reformatting step to reformat the program according to the determined ideal capture parameters. This determining ideal capture parameters and reformatting may be done for the entire program, scene-by-scene, for discrete portions of scenes or even in a dynamic manner, changing (e.g. linearly) even within scenes.
  • the identifying ideal capture parameters and reformatting the program may be done several times for a same program to optimize the program for different viewing parameters.
  • the IMAXTM version, movie big-screen version and Blu-ray (or VOD, etc .. ) version of a program may be produced all at once. It is not necessary for the original capture parameters to be non-ideal for each version produced.
  • the capture parameters may be optimized during filming for the big-screen, and be reformatted during production to optimize the program for IMAX and typical Blu-rayTM-connected displays.
  • a film may also be produced using this method in several formats, for a variety of typical viewing parameters.
  • a film may be offered in Blu-rayTM or VOD (or through other distribution means) in a selection of optimized viewing parameters such that the viewer/purchaser may select his screen size and/or layout.
  • Blu-rayTM there may be several optimization of a film on a same Blu-rayTM.
  • filming may refer to actual filming with cameras or generating/rendering CGI animation or games.
  • the term “camera” as used herein (even when qualified as “original” or “actual” or “real") may be virtual camera positioned computed for the purposed of rendering a 3D graphics scene.
  • a method for reformatting an input stereoscopic content for proper presentation of three dimensional content on a second stereoscopic display having a different format than that of a first stereoscopic display for which the input stereoscopic content was generated, each stereoscopic display having a corresponding predetermined stereoscopic camera pointing angle and a predetermined intercamera separation comprising: projecting each first stereoscopic channel image from a first virtual image plane perpendicular to a line of sight of a corresponding first camera onto a second virtual image plane perpendicular to a line of sight of a corresponding second camera to form a corresponding second stereoscopic channel image and re-pixelating each said second stereoscopic channel image.
  • a method for reformatting an input stereoscopic content for proper presentation of three dimensional content on a second stereoscopic display having a different format than that of a first stereoscopic display for which the input stereoscopic content was generated, each stereoscopic display having a corresponding predetermined stereoscopic camera pointing angle and a predetermined intercamera separation comprising: projecting each first stereoscopic channel image from a first virtual image plane perpendicular to a line of sight of a corresponding first camera located at a focus point of said first camera along said line of sight of said first camera onto a second virtual plane perpendicular to a corresponding second camera line of sight, said second virtual plane being located at a focus point of said second camera along said corresponding central line of sight to form corresponding second stereoscopic channel image and re-pixelating each said second stereoscopic channel image.
  • input stereoscopic content including left and right stereoscopic image pairs, is provided along with original camera separation and original camera pointing angle parameters employed in capturing, generating or initial formatting of the stereoscopic content for an original stereoscopic display of an original size.
  • original camera and original display parameters can be predetermined in advance, or provided in metadata with the input stereoscopic content.
  • original camera parameters can be specified once for the entire stereoscopic video program or on a scene- by-scene basis whereas original display parameters can typically be provided once for example at the beginning of the program.
  • original camera parameters are derived from the input stereoscopic content. Deriving original camera parameters from stereoscopic content is being described elsewhere, and is beyond the scope of the present description.
  • a preliminary step of determining the original camera parameters from the input stereoscopic content subjects stereoscopic image pairs to a preprocessing step, for example to determining geometric aspects of the imaged scene.
  • the determination of the original camera parameters can be performed once for the entire stereoscopic program, once per scene, for each stereoscopic image pair or can include averaging over multiple stereoscopic image pairs.
  • the stereoscopic program can dictate the type and extent of such preprocessing, for example a static stereoscopic video scene would require less preprocessing whereas an action stereoscopic video scene would require more.
  • the angle between the central lines of sight of the original cameras show in Figure 14 is referred to as the angle of convergence and is a primary stereoscopic content acquisition parameters; another primary stereoscopic content acquisition parameter is the distance between the cameras.
  • the point in plan view where the central lines of sight converge is referred to as the point of convergence. More generically, employing a dual camera stereoscopic content acquisition system convergence is achieved over a line, for cameras having long focal length lenses over a substantially straight vertical line of convergence.
  • comparing a display size of a stereoscopic display on which the input stereoscopic content is intended to be presented with the original display size can be employed to trigger the reformatting of the input stereoscopic content in order to minimize stereoscopic viewing discomfort as presented herein:
  • the original camera parameters and original stereoscopic display specification defines for each channel a first virtual image plane (L oam i, L cam2 ) illustrated in Figure 8, perpendicular to the line of sight of a corresponding camera, the first virtual image plane passing through a vertical convergence axis line (D conv ) defined by the original camera separation and the original camera pointing angle(s) (6 ca mi, e ca m2) where the central lines of sight of the cameras intersect.
  • D conv vertical convergence axis line
  • the display parameters of the stereoscopic display on which the stereoscopic content is intended to be presented require a second camera separation and a second camera pointing angle for comfortable viewing of the stereoscopic video content.
  • each side image of the original stereoscopic image pair is projected onto a second virtual plane perpendicular to the corresponding (same side) second camera line of sight passing through the convergence axis.
  • the above described image processing does not use information from the other stereoscopic channel to provide content reformatting for a given stereoscopic channel.
  • the corresponding image projections create a new second stereoscopic image pair.
  • the projection may be orthogonal, which projection imparts a twist and shift to the original image in reformatting the corresponding new image.
  • the twists and shifts are complementary between the reformatted right and left images.
  • each original stereoscopic image is orthogonally projected onto a second virtual plane perpendicular to the corresponding second camera's central line of sight and located along the line of sight at the point of focus of the original camera. Accordingly an original directorial intent is more faithfully reproduced while providing a substantial reduction in eye strain.
  • Examples where the points of focus of the left and right cameras are not the same as the point of convergence include the cameras non being ideally set up or done purposefully to achieve some a visual effect, for example poorly focussed eyes, e.g. to simulate being tired, euphoric or disoriented.
  • the second virtual planes of the left and right images may not intersect the point of convergence of the two cameras. Projecting an original image from the first virtual plane of the original image to the new second virtual plane which is located proportionally spaced from the (new virtual) point of convergence to mimic the same (error) effect as in the original input stereoscopic content.
  • the actual point of focus of the cameras represents a secondary parameter which may or may not be taken into account, depending on whether the system is assumed to be ideal or not.
  • the invention is not limited to the above assumption that cameras are placed side by side along a horizontal line and oriented towards one another at a slight angle of convergence to mimic the eyes of the viewer.
  • Other possible secondary parameters include camera positions in a three dimensional world relative one another. It is possible that due to error or intent the original cameras were not perfectly placed, including one camera being recessed or brought forward. The pointing angles of the two cameras would not be perfectly equal, etc. So long as information regarding such secondary parameters is provided (or can be inferred/determined), the second virtual plane of the left and right images can be defined based on the central line of sight and the focus of each camera. In the absence of one or two secondary parameters assumptions can be made.
  • the location and orientation of the second virtual plane can be derived from the vector of the line of sight assuming that the focus is at the point of convergence where it crosses the right camera's central line of sight.
  • the point of convergence could be set as the middle of the shortest line linking the two central lines of sight.
  • the original stereoscopic content can be reformatted not only to adapt it for presentation on a different sized stereoscopic display on which the stereoscopic content is intended to be displayed, but rather/also to correct imperfections in the original content capture.
  • the original cameras need not be imperfectly set up, any virtual secondary parameters can be employed to achieve desired effects.
  • the original left and right stereoscopic images provided as input undergo a transformation to simulate new camera parameters for the intended presentation to reduce viewing discomfort by reducing eye strain.
  • the images are reformatted (transformed) to project them on the second virtual plane in which they would have been had the original images been captured by the virtual cameras having the second (virtual) camera parameters.
  • Any suitable projection technique may be used to reformat the original input images.
  • the goal is to achieve as accurately as possible what the images would have looked like had the images been taken with the second virtual cameras. Given that objects in the imaged scene are three dimensional, there are regions of space which would be occluded from one camera's viewpoint but visible in the other. Consider for example a vertical cylinder having rainbow colors drawn all around it.
  • each original input stereoscopic image has some information which remains after the transformation however which should be absent (occluded) in the transformed stereoscopic image when compared to a corresponding ideal virtual image.
  • each original input image lacks some information (which is occluded in the original input image) that would be present in an ideal virtual image.
  • occlusions For small camera angle differences and small positioning differences between the original and virtual cameras, which usually lead to the type of eye strain being mitigated herein, there is typically very little excess/missing image data (occlusions).
  • an object in the real scene which lies between the two planes illustrates aspects of the proposed solution.
  • the line connecting the object and the first original camera intersects the first original plane is where the object is located in the original image.
  • the line connecting the second virtual camera and the object intersect the second virtual plane is where the object should be located in an ideal second (simulated) image. Since orthogonal projection is used, the actual location of the object on the new second image is orthogonally above the point on the first original image where the object is located in the first image.
  • the transformation may yield some less-than-ideal results for background objects and foreground objects away from the focal plane of the camera, desirably these objects are out of focus and likely to attract less of the observer's attention.
  • a reasonable approximation of an image taken with the virtual camera parameters is to use a projection as the transformation step as described hereinabove.
  • the application of an orthogonal projection to each original input image onto the corresponding virtual image plane can be done using any suitable projection technique.
  • the pixels of each original image can be mapped onto the corresponding second virtual plane by providing each pixel of the original image a new position in an area of the second virtual plane orthogonally above the corresponding original image.
  • the transformed pixels may be stretched or compressed by the mapping (depending on whether the second stereoscopic display is larger or smaller than the original stereoscopic display).
  • the invention is not limited to the manner of implementing orthogonal, projections other suitable techniques may be used.
  • the original image is morphed to appear stretched out and shifted (with respect to the central line of sight of the virtual camera, which may not be in the center of the new virtual image).
  • An area in the second virtual plane of the second virtual camera corresponds to an area where the virtual camera would have captured the required image. This is a rectangular area centered around the central line of sight of the second virtual camera, referred to hereinafter as the virtual image area.
  • the mapping on the second virtual plane is located orthogonally above the original image in the original image virtual plane but may not cover entirely the virtual image area.
  • the projected pixels can be shifted to cover the entire virtual image area (assuming it is larger in the second virtual plane).
  • the projected pixels are left as projected and re-pixelisation is performed by applying a pixel grid over the virtual image and assigning values to pixels according to the value of the projected pixel mapping at grid coordinates.
  • each grid pixel is assigned at least one value from: brightness, chroma, at least one pixel component (RGB, YUV, etc.) in the projected mapping that is in the center of that pixel.
  • an average (or other weighted value) in the projected mapping is assigned for the area of the pixel to the pixel.
  • the resulting image has black bands on either one or two edges because the virtual image area was not fully filled by the projection mapping. While the left eye and right eye of the viewer will not see whole images, because of left and right image overlap providing a substantial region over which the second left and right images can be fused to perceive the stereoscopic effect, it will appear that whole images are being displayed. This is because the black band(s) of the left reformatted image will overlap with a filled-out portion of the right reformatted image and vice versa. Thus the reformatted images will appear to the viewer as whole images. There will however be no stereoscopic effect in the black band(s) region(s) since in any such area the human viewer will only see one image which lacks parallax. This lack of stereoscopic effect is limited to the sides of the reformatted stereoscopic image and not in the main area of focus.
  • orthogonal projections yield a shift and twist of the image to simulate a different position and angle of the camera, and are computationally light.
  • non-orthogonal projections can be employed for example by a direct calculation or two-step projection employing a first orthogonal projection on an intermediary plane and then employing a second orthogonal projection from that plane to the intended final plane.
  • a two-step (orthogonal or not) projection can be employed using an intermediary surface of projection which is not flat having a shape intended to impart a certain non-linear transform to the input image.
  • an intermediary plane could be used in a two-step projection approach to distort the output image or to shift it.
  • a non-flat intermediary projection surface can provide a different transform effect for each pixel by using a sufficiently detailed intermediary surface. That is, pixels could be treated differently by using a distortion surface which causes certain areas of the input image to be projected differently than others due to a varying slope of the intermediary surface.
  • pixel depths are taken into account and differentiated modifications are applied for different image areas or individual pixels in each original image.
  • Both images of each original stereoscopic pair are processed to infer imaged object depth for each pixel or other data such as a depth map (a mapping of pixel depths, sometimes -but rarely- provided with a stereoscopic video feed) is employed.
  • a depth map a mapping of pixel depths, sometimes -but rarely- provided with a stereoscopic video feed
  • An advantage of the use of projection to generate the transformed image is that this is a computationally-light technique, which may be generated in real-time.
  • a video e.g. film
  • This real-time reformatting may be used in movie theaters (e.g. of non-conventional dimension) or may be used at the home to account for the large variety of viewing parameters in homes.
  • the reformater is used to implement a child safety viewing mode on a video viewing system. It is believed that forcing the eyes to focus beyond the usual range of convergence angles may be bad for vision. In young children who are still growing and whose anatomy is still developing in particular, it is feared that such strained focusing may lead to abnormal development of the eyes and eye muscles and cause vision problems on the long term. As discussed herein, most 3D content is captured for a particular set of viewing parameters including an interocular distance. Commonly, it is assumed that the viewer will be an adult with an average interocular distance. However, if viewed by a user with a different, e.g.
  • the video viewing system comprises a display and at least one user input for enabling or disabling the child safety viewing mode.
  • the commercial video viewing system is a television and the input is controlled by the user via a graphical user interface controllable by a remote control, whereby a user can access a menu in which a visual presentation (e.g. icon) of the child safety mode option is presented to the user.
  • the user may activate child safety mode by selecting with buttons on the remote control the child safety mode icon and pressing a selection button.
  • child safety mode could be accessible via a button on the remote control.
  • the vide viewing system may be a gaming console with a child safety toggle button or a 3D-enabled mobile device (phone or tablet) with a graphical user interface including a setting menu including a display submenu accessible as visual presentation (e.g. icon and/or text heading) in the setting menu by a user pointing device such as a touch screen.
  • the display submenu may comprise a further 3D submenu similarly accessible, and the 3D submenu may comprise a visual presentation (e.g. an icon) representing the child safety mode option, which presents the user the possibility of toggling child safety mode by interacting with the corresponding visual presentation using the pointing device, e.g. by pressing on it on a touchscreen.
  • activation of the child safety mode causes a change in the viewing parameters used to compute the reformatting performed.
  • the new viewing parameters used to reformat the image comprise a smaller interocular distance more typical of young children.
  • the camera placement algorithm used to "place" the virtual cameras is a camera placement algorithm that takes interocular distance as an input and it is provided the smaller interocular distance more typical of young children.
  • a refinement of the above is possible where the input device allows for additional information to be received from the user.
  • a user may be presented an option of selecting one of a plurality of age groups for the intended viewer. There may be for example a "6 year or less” option, a "6-9 years” option, a "9-12" years option, and a "teen” option.
  • These options may be represented in a menu, e.g. a pop-up menu brought forth on the screen in response to the activation of the child safety mode. Selection of an option is input by a user using an input device, e.g.
  • a pointing device like a touchscreen on a mobile device or a remote control with arrow buttons allowing selection of an option and a select button to confirm the selection.
  • these options are presented upon activation of the child safety mode as described above.
  • activation of a child safety mode may be performed by virtue of selecting an option appropriate for children, like an age category in the above-described menu. In this case, there needn't be a specific "child safety mode" button, but rather the menu above may represent the whole age gamut including "adult”. Child safety mode may be activated by user input constituting of the activation of an option suitable for children.
  • the 3D image may tend to be flattened towards the plane of convergence (the plane of the screen in typical one-screen displays). While this leads to an inferior 3D experience, it is not considered harmful, and in any event is far safer than when the interocular distance is set too high. In the latter case, the 3D effects may appear exaggerated, leading to parallax well beyond the comfortable viewing zone and even eye divergence.
  • a user may be presented a visual messages instructing the user as to how to best select the interocular distance-related option.
  • Such instructions may include instructions on how to measure interocular distance (for the below example) and/or other visual indication displaying a message instructing the user as to select an option.
  • the message may warn the user that when in doubt, it is safer to select the option corresponding to the younger age/closer interocular distance.
  • the options may be displayed directly as ranges of interocular distance.
  • the instructions on how to measure such distance will be particularly useful. While the above refers to a child safety mode, it will be appreciated that this indeed represents a user safety mode, as it can apply not just to children but to anybody that does not have the interocular distance used by the content creator in the calibration of the capture system.
  • the system may allow for the user to input information corresponding to the interocular distance, either by inputing the actual interocular distance of the viewer through a numerical input interface or by providing other information such as a date of birth through which interocular distance may be estimated.
  • the child safety mode may moreover impose additional restrictions on the reformatting. Indeed, in addition to informing the viewing parmater used in reformatting, the child safety mode may, for example, cause the reduction of depth by, e.g., implementing a smaller comfort zone. It may also cause a reduction of the image size, e.g. by using the reformater 3D zoom, to compensate for kids sitting too close to the TV. This may also be done in 2D, by simply reducing the image size, e.g. by downsampling or downscaling.
  • the reformater is combined with a viewing parameter acquisition module (or viewspace module) that identifies viewing parameters and provides them to the reformater to derive ⁇ .
  • the viewspace module stores information relative to the size of the screen and relative position (e.g. distance) of the viewer from the screen.
  • the viewspace module is implemented by a processor in a viewing system (in this example a 3D TV) that implements a graphical user interface allowing a user to input viewing parameter information for the viewspace module.
  • the viewspace module may be hard-wired with information on the size of the display but the graphical user interface may prompt the user, upon a first activation of the television to input the distance of the couch to the TV.
  • the prompt may be implemented by a visual message on the display including a schematic diagram of the measure to take and a text box may allow the user to enter on the remote control the number of, e.g., inches between couch and TV.
  • the viewspace module may use alternate means to identify viewing parameters.
  • televisions today occasionally come with front-facing cameras for VOIP applications.
  • known etechniques exist for identifying where in the frame faces are located for the purposes of focusing the camera.
  • the viewspace module is connected to a front-facing camera and uses known face- identifying techniques for determining where users are located relative to the screen Distance (depth) of the users can be ascertained using previously user-mput distance-to-screen information, using range-finding techniques or by assuming a certain interocular distance (either a common or average interocular distance or one corresponding to a previous user input, e.g.
  • a position may be ascertained, e.g. by visually identifying the indicia and comparing it's perceived size to a known size. Any other means, such as GPS tagging or local triangulation may be used.
  • the 3D image will look slightly different to each viewer since each have different viewing parameters.
  • one viewer's position may be selected as the target viewing parameters for reformatting such that at least one viewer has an ideal view.
  • the viewspace module may locate a postion geometrically centered between all the viewers as location of the target viewer in the viewing parameters.
  • the reformater may have as a criterion (on top of the ones discussed above, or alone) the respect of the comfort zone for all viewers. That is, while only one viewing location may be the ideal viewing location which depending on the viewing parameters fed to the reformater by the viewspace module, the reformater may be provided additional sets of viewspace parameters corresponding to all the viewers not in the ideal location criterion under the criterion that under no circumstance should the 3D image extend beyond the comfort zone of any viewer.
  • an image tube-based implementation performs a first transformation of image tube to the new viewspace, transforms the image tubes according to the other criteria desired (e.g.
  • the image tubes are further transformed, e.g. to reduce depth effect, to prevent it and the resulting transform is once again verified for all viewspace parameters. It should be noted that it should always be possible to satisfy all comfort zones as there is no lower limit to the 3D effect (it may be decreased asymptotically to 2D which is free from depth perception comfort issues). However, it is also possible to set limits to prevent over distortion for the sake of non-central viewers.
  • the reformater provides multiple reformatted stereoscopic image pairs targeted at different viewers, e.g. by reformatting an input stereoscopic image pair using several different instances (in sequence or in parallel) of the method described herein.
  • One of the particularly useful consequences of the foregoing, is that the present reformater makes it possible to provide a multi-view output formatted to provide images for users at different 3D positions from the screen, including depth, not just angles.
  • aS such a multiview generator implemented by the image tube-based implementation of the reformater may provide a better viewing experience on a autostereoscopic or multiview display than has so far been possible.
  • a mutliview generator comprising the reformater described herein, wherein the reformater generates a plurality of image view including at least a plurality of stereoscopic image pairs, each of said plurality of image pair being optimized for the viewing parameters (e.g. for the viewing location) of a different viewer, said image views being for display on a multiview display.
  • the plurality of image views are then displayed on the screen in such manner as to present to each of the different viewers the respectively optimized image pair using known autostereoscopic/multiview display methods.
  • the reformater may use one (or more) image and the disparity map to generate image tubes and therefrom generate stereoscopic image pairs for the users according to the method above, although disparity estimation will not, in this case, be required.
  • Viewing equipment in the home may be equipped with reformatting technology to enable an ideal 3D viewing experience in the home.
  • the camera placement-based solution like the image tube-based solution may be implemented in hardwear for real-time running.
  • Such viewing equipment may include televisions, VOD terminals, Blu-rayTM players, gaming console. They may be pre-programmed to reformat according to particular viewing parameters and may receive the original capture parameters alongside the 3D program, for example in metadata provided therefor. Alternatively, they may provide customization tools to customize the reformatting to particular viewing parameters or to a particular program.
  • a viewing equipment may identify on its own the position of the viewer and may already know, or be inputted the size of the display. Any method for locating the user may be used such as facial recognition combined with range finding, methods used in autofocussing of digital cameras, detecting sensors, transponders or reflectors in 3D eyewear, etc... If multiple viewers are present, the viewing equipment may identify an average viewing parameter for the multiple viewers, or may provide multiple instances of reformatted program if it is possible to display different views to different viewers (e.g. with shutter glasses at offset timing or by providing different views in an autostereoscopic display's different rays of visibility).
  • a user may input its viewing parameters, such as angle and distance to the display and/or display size.
  • the program may then (e.g. in real time) be reformatted for the particular viewing parameters.
  • At test pattern may also be used to determine viewing parameters.
  • a test pattern that has a clearly known shape e.g. a sphere, a cube at an oblique angle or an "x, y, z," unit vector combination also at an angle
  • the user may then use an input tool such as a remote control to vary different aspects of the image until the shape appears correctly, the viewing equipment then determines the corresponding viewing parameters either using a lookup table or by revers -determining which capture parameters would have led to the distortion created by the viewer.
  • the viewer may directly change the viewing parameters or the viewing equipment may provide the viewer with a feed that constantly changes the test pattern and the viewer may tell the viewing equipment using a user input tool such as a remote when the test pattern looks right.
  • video programs may be provided with a test pattern (e.g. the known image) at the beginning of the program, the viewer may then provide inputs to the distortion experienced and the reformater may apply changes in simulated capture parameter accordingly until the image appears undistorted (or less distorted) to the user.
  • a test pattern e.g. the known image
  • the reformater may apply changes in simulated capture parameter accordingly until the image appears undistorted (or less distorted) to the user.
  • the change in capture parameters can be directly determined for a particular video.
  • the reformater may be adjustable subjectively. That is the input of the viewing parameters may be done, not by precise manual entry or parameter values (e.g. distance-from-screen, inerocular distance, etc..) nor by visual acquisition by the viewspace module but by a user-subjective evaluation.
  • the a reformater may be implemented on a viewing system comprising a display and a user interface for allowing a viewing user to interact with the viewing system, the user interface comprising adjustment mechanisms for allowing the user to adjust reformatting parameters.
  • the reformatting parameters may be a Z-direction stretch, or may reflect viewing parameters (e.g. distance from screen, interocular distance, position relative to screen or any function thereof).
  • the viewspace module causes the display by a processor of a graphical user interface of a visualization of instructions instructin gtrhe user to adjust the image using buttons on a remote control.
  • the user uses the left and right arrow buttons to stretch or compress in the Z-direction the image until it looks subjectively right to the user.
  • the user may pause a video when a human head or another easily recognizable feature is in view and adjust the reformatting as described until the geometry, and particularly the ratio of the Z to the X and Y directions looks right.
  • a video stream may begin with one or more well-known shapes (e.g. spheres and wireframe boxes) for that exact purpose.
  • the user may have multiple control inputs for varying Z-scaling at different depths.
  • the user can uses the side arrows to compress or expand depth at near the display depth (convergence plane) and uses the up and down arrows to compress and expand depth at near-viewer depths, the reformater plotting and applying a depth-variance function satisfying both inputs.
  • the present reformater allows for the universal reformatting of images for all in a manner tailored for all viewing system such that video shot on a cell phone stereoscopic camera may be enjoyed on a large screen home theater system and movies shot for image or big screen theaters may be enjoyed at home on a TV or tablet.
  • the reformater can be used in a post-production setting to adjust a video stream in any of the manner mentioned above.
  • the reformater is useful for fixing camera calibration errors, for correcting for moving/changing camera parameters rather than to recalibrate cameras, for reformatting for different viewing parameters, e.g. target screen size, etc...
  • the stereoscopic reformatting scheme described above can be implemented in programming as described to create a suitable stereoscopic reformater in software or hardware.
  • the functionality described hereabove can be implemented, for example, in software code to create a software program capable of reformatting 3D content according to the above scheme.
  • a such software may be particularly useful for so-called "offline" reformatting, that is to reformat content in non-realtime for storage for later display under corresponding viewing parameters, as will be described in more details below.
  • This algorithm has been developed specifically to enable rapid real-time performance and implementation in hardware (e.g. FPGA or ASIC).
  • a reformater programmed according to the foregoing description may comprise the modules described and illustrated, although certain modules may be merged (e.g. the disparity map estimation and refining, depending on the refining method used) as will be understood by a skilled programmer.
  • An example of an implementation that a skilled programmer may create would be one wherein an externally -built (e.g. purchased) disparity estimation module is used (optionally, a refining module is added to it externally), a transformation engine is programmed to comprise the functionality illustrated in the "image tube generation and transforming" and "disparity re-construction and view synthesis" portions of Figure 4, and external post-processing software is used on the input of the transformation engine.
  • Reconfiguration is one method of obtaining multiple configurations of stereoscopic content.
  • Depth-image based rendering (DIBR) techniques also exist that allow generation of different configuration on the basis of a depth-image which provides the depth or disparity of pixels from a particular viewpoint Combined with at least one view-image DIBR techniques are used to generate two or more views to provide a particular stereoscopic configuration for a viewer. Thus if a depth-image is available, DIBR techniques can provide another method of generating multiple configurations of stereoscopic content.
  • the content may be generated the content into different configuration directly at capture.
  • Some capture schemes proposed employ more than two cameras to capture content.
  • two different configurations may share a viewpoint.
  • the viewpoint of a scene presented to the left eye of a viewer may be the same but the viewpoint of the seen presented to the right eye of the viewer may be different such as to provide different depth perception.
  • the viewpoint of a scene presented to the left eye of a viewer in one configuration may actually be the same as the viewpoint of the scene presented to the viewer in another configuration provided that the other viewpoint shown is appropriately selected to provide depth perception in a viewing environment.
  • Stereoscopic content made from computer-generated and rendered into images can be also rendered into multiple viewpoints to create at generation different configurations of the content.
  • FIG 16 illustrates a stereoscopic content distribution system 200 according to a non-limiting embodiment.
  • the content distribution system provides video-on- demand (VOD) service which allows a customer 215 having a smart TV 220 to rent movies that are then streamed directly to their TV 220. Only one customer is shown in this illustration but it will be understood that the system may be used for distribution to a large number of customers.
  • the system 200 has several parts that ensure the quality and non-reproducibility of the video as may be required by the video content providers such as film studios that produce stereoscopic films.
  • the system has two main portions: the content management system 205 and the content storage system 210.
  • the content management system 205 and the stereoscopic content distribution system 200 are separate entities, however as will be described further below, the two could be embodied by a same entity such as a single server.
  • the content storage system 210 is responsible for storing the digital content and in this example, for streaming it directly to the customer 215 via a content distribution network 225.
  • the content storage system 210 is also considered a content provisioning system, although it will be simply referred to as content storage system 210 for simplicity.
  • the content distribution network 225 is shown as separate from the content storage system 210 but it could also be considered to comprise the content storage system 210.
  • the content management system 205 is responsible for managing access to the stereoscopic content in the content storage system 210 by the customer 215 and by any other customers. It will now be described in more details.
  • stereoscopic content are video programs and more specifically stereoscopic films that are provided by film studios in master source files 230.
  • the master source files 230 provide the stereoscopic content in multiple stereoscopic configurations.
  • each stereoscopic configuration for a particular film is treated like a separate film and provided separately in different files.
  • the stereoscopic content is received in multiple configurations each having two views, a left- and a right-eye view which are each treated like a monoscopic film, meaning that each view has its own source file, however the two together are considered a single master source file 230 for the purpose of this description.
  • the master source files 230 may be provided by various means, such as electronically by FTP or physically in a hard disk or tape.
  • a data ingest system 235 receives the master source files 230 and performs a variety of initial functions.
  • the data ingest system 235 are first quality-checked and prequalified for ingestion. If a frame-compatible stereoscopic format is used, like Side-by-Side (SBS) or Quincunx-Side-by-Side (such as SENSIOTM HiFi 3D), the stereoscopic content is converted to that format at this stage.
  • Master source files 230 are then converted to mezzanine files which in turn are quality checked. Any metadata is created or received and is appropriately formatted at this stage.
  • SBS Side-by-Side
  • Quincunx-Side-by-Side such as SENSIOTM HiFi 3D
  • the mezzanine files are then passed to a demuxer system 240 in which the mezzanine files are compressed and transcoded to H.264.
  • a demuxer system 240 in which the mezzanine files are compressed and transcoded to H.264.
  • multiple bitrates may be applied to allow for adaptive streaming bitrates. This may lead to the creation of multiple encoded files. All associated files and metadata are then packaged to form complete deliverable.
  • the results are then passed on to a cypher encryption server 245, shown here as part of the content storage system 210 but which could be located remotely and operated by a separate data protection company.
  • a cypher encryption server 245 shown here as part of the content storage system 210 but which could be located remotely and operated by a separate data protection company.
  • There the video content is encrypted and packaged.
  • An encryptor generates an asset key that protects the whole asset.
  • One or more content keys are used to encrypt video data.
  • Each content key is placed into an entitlement control message (ECM).
  • ECM entitlement control message
  • the asset key is registered with a cypher key management server 250 and used later to create permissions.
  • Encrypted files and metadata are stored on a media streaming system 255, which may be a separate server.
  • the media streaming system is not only responsible for storing the encrypted media but also for streaming it via content distribution network 225 to the customer 215.
  • the content is accessed through an application programing interface and delivered to approved licensors.
  • Access to the stereoscopic content on the media streaming system 255 is controlled by the content management system 205.
  • An asset management system 260 is responsible for content management and user (account) management.
  • the content management employs content provisioning system API to query catalog contents (based on customer credentials) and access catalog contents.
  • the User management includes user registration and verification.
  • An application system 265 communicates directly with a remote user application on TV 220 via a network such as the internet and provides a storefront and acts as a media portal for ordering, account management, etc...
  • a reporting and analytics system 270 generates financial and usage reports that are made available on a periodic basis (e.g. monthly). These include default data on movie rentals including the quantity of movies that have been rented and the quantities of each configuration that has been provided to users.
  • a stereoscopic content distribution system 300 will now be described with focus on the distribution of different configuration versions of stereoscopic content with reference to Figure 17.
  • Certain elements of the stereoscopic content distribution system 200 of Figure 16, such as elements associated with data encryption, are absent in the stereoscopic content distribution system of Figure 17; these may be absent from this implementation or, alternatively, may be present but simply not shown.
  • the stereoscopic content distribution system 300 comprises a content management system 302, a content storage system 304, a registration system 326, which work together to provide access to stereoscopic content at a user end 306.
  • the content management system 302, content storage system 304 and registration system 326 are separate entities and more particularly are servers, but it will be appreciated that the functions of some or all of these could entities could be combined and performed by a single entity or server or could be further divided into more entities or servers as will become clear in the following description.
  • the content management system 302 is responsible for managing access to viewable stereoscopic content in a digital library 308 by a remote user.
  • the content storage system 304 is responsible for storing the stereoscopic content and therefore comprises the digital content library 308.
  • the content storage system 304 is also responsible for providing the stereoscopic content to the user end 306 and therefore also acts as a content distribution server.
  • the content storage system 304 is a server which comprises a large storage system containing the digital content library 308.
  • the digital content library 308 is a content library that contains stereoscopic content in the form of one or more stereoscopic programs such as films or TV shows.
  • the digital content library 308 comprises a multitude of stereoscopic films.
  • the digital content library 308 may comprise also non-stereoscopic programs such as 2D films, but for the purpose of this example, it will be assumed that only stereoscopic films are contained in the digital content library 308.
  • the digital content library 308 comprises a plurality of versions of the stereoscopic program.
  • Every program in the digital content library 308 is available in a plurality of versions, however this does not need to be the case.
  • Each version of a program corresponds to a different stereoscopic configuration of the program.
  • Each stereoscopic configuration corresponds to different viewing parameters or, by extension, to different viewing environments defined by those viewing parameters.
  • a version may be re-configured from an existing version/configuration not for an entire viewing environment, taking into account every viewing parameter defining it, but rather only to account for a difference in one viewing parameter (for example IOD, as discussed below).
  • each version is in a stereoscopic configuration corresponding to a respective set of viewing parameter, which set may be a plurality of viewing parameters (up to every viewing parameter defining a viewing environment) or simply one viewing parameter such as the interocular distance (IOD).
  • set may be a plurality of viewing parameters (up to every viewing parameter defining a viewing environment) or simply one viewing parameter such as the interocular distance (IOD).
  • IOD interocular distance
  • each version of a stereoscopic program is stored as a separate file in the same manner as different stereoscopic programs would normally be stored in a content library It will be appreciated that this can be done in other manners.
  • reconfiguration of stereoscopic programs can be performed on-demand in realtime on the basis of a single version stored in the digital content library 308.
  • control entity 310 in this case a processor programmed with the functions of the server.
  • the content storage system 304 is provided with an appropriate communication interface system 312 which directs communications with the content management system 302 and the transmission of stereoscopic content to the user end 306.
  • the communication line from the content storage system 304 to the user end 306 is shown as unidirectional to reflect the fact that in this example the user end 306 directs requests and other data to the content management system 302 and merely receives content from the content storage system 304, but of course some feedback, e.g. for streaming control or for audio selections and other such controls may be provided from the user end 306. Optionally even this feedback may be transmitted to the content management system 302 and forwarded by the content management system 302 to the content storage system 304.
  • the communication interface system 312 of the content storage system 304 comprises appropriate communication links to the content management system 302 and the user end 306. In this example, both are accessed via the internet and the communication interface 312 comprises a very high-bandwidth connection, however the content management system 302 can be separate but local in which case the content storage system 304 and content management system 302 can communicate through a different link also part of the communication interface system 312.
  • the content storage system 304 is treated herein like a single entity, the skilled addressee will understand that the content storage system 304 may be distributed, e.g. like in a server bank. It is moreover not uncommon for a content distribution network to comprise several server locations at different places each offering the same service to users in respective geographical regions near them. Thus the singleness of the content storage system 304 and indeed of the content management system 302 and the registration system 326 shown here are not intended to be limiting.
  • the content management system 302 comprises a storage system which in this example is local storage 314 but could also be implemented by other storage means like cloud-based solutions, processing logic 316 and a communication interface system 318.
  • the content management system 302 is a server.
  • the local storage 314 is used for storing a stereoscopic content database 320 and a user database 322. Both are stored in the same local storage here, but each could have its respective storage system.
  • the stereoscopic content database 320 comprises a set of records 321a, 321b, 321c... each corresponding to different stereoscopic content, in this example to different films.
  • Each record comprises information to be provided to the user end 306 to allow for a selection of a program at the user end 306. This may include a title, a brief description and a thumbnail.
  • each record comprises an identification the different versions of the stereoscopic content available. More particularly each record comprises an identification of a plurality of versions of the stereoscopic content, each of the plurality of versions being in a different stereoscopic configuration, each stereoscopic configuration corresponding to a different set of viewing parameters. In this example, each version is identified and associated to its respective set of viewing parameters.
  • stereoscopic content database 320 can be part of a larger content database comprising non-stereoscopic content but for the purpose of this non-limiting example, only stereoscopic content are offered.
  • the content management system 302 builds populates the stereoscopic content database 320 via communications with the content storage system 304. In particular it may receive from the content storage system 304 a list of stereoscopic programs contained in the digital content library 308. To this end the communication interface system 318 of the content management system 302 may be in communication with the content storage system 304 (via its communication interface system 312) from which it receives the information with which to build or populate the stereoscopic content database 320, including for each program an identifier of the stereoscopic content. This information may then be placed by the processing logic 316 of the content management system 302 into the records 321a, 321b, 321c, ... of the stereoscopic content database 320.
  • the identifier may comprise an address where the respective stereoscopic content may be accessed, such as an indication of a corresponding location in the digital content library 308.
  • the content management system 302 may similarly receive an identifier of each version, which may also comprise an address of the respective version of the stereoscopic content such as an indication of a corresponding location in the digital content library 308. Accordingly, each of the records 321a, 321b, 321c, ... may be provided with a location identifier for each of the versions of the respective stereoscopic content.
  • the stereoscopic content database 320 is stored in local storage 314. It may be stored there in a persistent manner, being periodically updated by pushed updates from the content storage system 304 or via queries to the content storage system 304.
  • records 321a, 321b, 321c... may contain optional credentials data (shown in brackets) which provide information on the user credentials required to access the stereoscopic content. This credential data may be used to determine which content to present to the user end 306, or which content requests from the user end 306 to honor.
  • the stereoscopic content database 320 may be present in the content management system 302 only temporarily, as will be described in the example related to Figure 19.
  • the digital content library 308 comprises a plurality of versions for each film, corresponding to different types of viewing environments.
  • a viewing environment can be defined by a large number of viewing parameters, however some viewing parameters can guessed or can be inferred or approximated from others.
  • the IOD is typically considered constant across all viewers, often being estimated to be 65mm.
  • the display resolution may be considered to be a typical resolution such as 1920x1080. Thus the IOD and display resolution may simply be guessed. Alternatively, the display resolution may be inferred from other parameters such as the display dimensions.
  • the resolution may be assumed to be ultra-high-def (4K) resolution while if the display dimensions indicate a typical television-size or laptop/desktop monitor size (say 11 "-69") the resolution may be assumed to be 1920-1080 while for smaller sizes more typical of tablets or smartphones, the resolution may be assumed to be 1280x720.
  • certain other viewing parameters may be inferred from other parameters. In particular, it is possible to obtain a reasonable estimate of the viewing distance from the display dimensions. Typically television owners will set up a couch at a certain distance from the TV, which distance tends to be related to the size of the television, bigger TVs being more typical of bigger homes.
  • VD viewer distance
  • ⁇ H laptop/desktop monitor-sized displays
  • a viewing environment for which the digital content library 308 may have a version of the film is a head-mounted display (HMD) such as the Occulus RiftTM.
  • a viewing environment may include a type of display that is an HMD or even a particular HMD, which may include particular display characteristics such as a distance from the eyes of the screen, a position of a screen (or respective left- and right-eye display portions for the HMD), a lenticular effect or type of lens, a distortion required on the 3D image (or, more particularly, individual left- and right-eye subimages), a resolution, a screen (or individual display portion) size, and other parameters.
  • HMD head-mounted display
  • a viewing environment may include a type of display that is an HMD or even a particular HMD, which may include particular display characteristics such as a distance from the eyes of the screen, a position of a screen (or respective left- and right-eye display portions for the HMD), a lenticular effect or type of lens, a distortion
  • the digital content library 308, may contain reconfigured versions of stereoscopic content for the general HMD display viewing parameter or, preferably, for individual HMD displays or display-types (e.g. Occulus RiftTM, Samsung Gear VRTM, a particular type of kit with a particular size smartphone, a generic single-screen-split-display type, etc . ).
  • display-types e.g. Occulus RiftTM, Samsung Gear VRTM, a particular type of kit with a particular size smartphone, a generic single-screen-split-display type, etc .
  • Head-mounted display include a display, which may be unitary or split into two parts that may consist of individual display screens, mounted about the head, in front of the eyes, generally in close proximity thereto, that each display for each eye a respective image.
  • a single display may be used on which the image is split down the middle such that the left side of the display presents an image for the left eye and the right side presents an image for the right eye.
  • An example of such a head-mounted display is the Samsung Gear VRTM which allows a smartphone to be mounted in front of the head of a wearer and used as a such unitary HMD display.
  • Respective left-eye and right-eye displays or display portions may overlap in a head-mounted display, with appropriate screening or multiplexing occurring before each eye to ensure each sees only the image intended for it, for example using polarization or page-flipping techniques used in TV screens or other known display techniques.
  • a configuration of stereoscopic content for display on a head-mounted display may include several modifications of the content.
  • the content may be reformatted, such as with the algorithm described herein or using any other appropriate reformatting algorithm to correct the depth-to-width (or depth to another proportion) ratio and/or to avoid over-convergence, divergence or other discomfort-causing effects.
  • the image tube model may be used.
  • the viewing parameters may include non-overlapping viewing areas for each eye. Accordingly, a constraint is placed on the reformatted image in that it should not include pixel placements where corresponding left- or right-eye pixels would be located beyond the area of their respective left- or right-eye display area.
  • this may be done by placing as a constraint in the target view parameters the determination the dimension and/or location (e.g. relative to the other display) of the display portion (e.g. the right-eye portion) for which a new image will be synthesized.
  • the constraint in one example is a threshold for a value in the image tube defining parallax such that the transformed image tubes cannot define a relationship between the eye and a point that would place the point beyond a certain angle or distance.
  • a feedback loop may be implemented at the view synthesis step whereby if any pixels are found to be placed outside of the area of a respective display portion, an indication of this, and optionally an indication of the offending pixel(s) itself and the amount and direction by which it is off-display is sent back to the image tube transforming module so that it may recalculate a new set of image tubes in an attempt to avoid the same problem or apply correction to the next image it processes.
  • a distortion effect may be applied to the image.
  • the Occulus RiftTM applies a barrel distortion to the two subimages (left- and right-eye views of the 3D image/scene) which is then corrected by the pincushion effect created by lenses in the headset.
  • the end result of the distortion and lenses is a spherical- mapped image for each.
  • the distortion may be applied by hardware within or connected to the HMD.
  • a "kit" to transform a smartphone into a HMD may include a crude cardboard frame, lenses and software to cause the smartphone to decode 3D content (e.g. in SBS format) and to apply distortion and display left and right images in respective display areas.
  • a device may not include the software or hardware required to apply a desired distortion to a 3D image/stream.
  • a distortion may be desired to cooperate with a lens to create a particular effect (e.g. barrel distortion described above, but other distortions may also be applied for other lenses and/or effects).
  • a distortion may be desired simply to create an effect desired for any type of display, although with a HMD, it may be preferable to apply distortions to account for the close proximity of the screen and/or higher visibility of pixels.
  • the application of such a distortion may be performed, using any suitable distortion-creating scheme, such as well-known barrel-distortion techniques, at the reconfiguration stage.
  • a version of 3D content in the digital content library 308 that is for a head-mounted display viewing environment may include an image distortion pre-applied to the image.
  • Viewing parameters for a version of content stored in the digital content library 308 may also include a software (or general scheme) used to display content.
  • Certain video players that can be run on HMDs may present a "virtual cinema room" whereby stereoscopic content is presented to the viewer alongside virtual context content.
  • the virtual context content may be, for example, a visual reproduction of a cinema room, with the stereoscopic (e.g. film) content displayed in on a virtual screen in the cinema room.
  • the viewing parameters may also include virtual parameters including the desired perceived distance (e.g. convergence plane) and size of the stereoscopic display.
  • the viewing parameter may be defined in such terms, or simply by a type of software decoder known to run certain parameters or by a particular known scheme (e.g. "IMAX room” or “virtual living room” or “virtual hotel room”) that corresponds to a particular set of viewing parameters. These viewing parameters may be provided as described herein by a user or by the software.
  • the digital content library 308 may include versions of content corresponding to different viewing parameters which may include HMD physical parameters as well as virtual parameters.
  • the films in the digital content library 308 are pre- reconfigured from an original version to include versions for the typical viewing parameters of handheld-sized displays ( ⁇ 11"), monitor-sized displays (11 "-26"), TV -sized displays (26"-70”) and very large TVs (>70"). It is to be understood that other versions corresponding to other configurations are possible, including for entirely different viewing parameters or simply to provide a different gradation of display size (e.g. more/fewer/different size ranges).
  • the stereoscopic content distribution system 300 may include a mean for providing a user with an estimated error or a correction factor.
  • the content management system 302 may provide an estimate error in terms of depth fidelity or a suggested correction factor in terms of other viewing parameters. For example, the content management system 302 may compute exactly how well or poorly the reformatted version of the stereoscopic will be faithful in depth if the viewing parameter (e.g. display size) at the user end differs from the one the version was ideally created for (e.g. if the 26"-70" range version was actually created for a 50" TV and the viewing environment includes 26" TV). It may provide this in terms of a depth-fidelity score.
  • the viewing parameter e.g. display size
  • the content management system 302 may compute an ideal other viewing parameter (e.g. viewing distance) with which to view the content in the particular viewing environment (e.g. on the particular viewing display) of the user or in the intended ideal viewing environment (e.g. on the 50" display the version was configured for) and may provide this information to the viewer for ideal viewing.
  • an ideal other viewing parameter e.g. viewing distance
  • the user database 322 comprises information on all users/customers of the stereoscopic content distribution system 300. For each user this information may include unique user account including a unique user identifier, credential information which may include payment information (such as a credit remaining and or information on the progress of a present payment) and/or subscription information (such as a level of subscription to unlimited access or a number of content accesses remaining under the current subscription).
  • credential information which may include payment information (such as a credit remaining and or information on the progress of a present payment) and/or subscription information (such as a level of subscription to unlimited access or a number of content accesses remaining under the current subscription).
  • the content management system 302 may optionally communicate with a payment authority to authenticate a payment from a remote user to rent or purchase stereoscopic data using either payment information (e.g. credit card details) stored as part of the user information in the user database or received directly from the user end 306.
  • payment information e.g. credit card details
  • payment information may be received from the payment authority and used by the processing logic 316 to authenticate a transaction and not stored persistently in the local storage 314.
  • the user database 322 may also comprise for each user digital viewing parameter data indicative of at least one viewing parameter from a set of user viewing parameter defining a remote viewing environment in which stereoscopic content provided by the stereoscopic content distribution system 300 is to be viewed.
  • This digital viewing parameter data is typically provided from the user end 306 as described herein.
  • the digital viewing parameter data is provided by the user end by a secondary (separate) device 324 via a registration system 326 and is stored as part of the user information 322 in the user database 322.
  • the digital viewing parameter data does not need to be persistently stored in the user database 322 and can be obtained on a per-content-request basis.
  • the user database 322 may contain a user preference, which may serve as default unless a digital viewing parameter data indicative of another set of viewing parameters is received upon request of stereoscopic content.
  • the content management system 302 also comprises a communication interface system 318.
  • the communication interface system 318 is adapted for all external communications of the content management system 302.
  • the communication interface system 318 is for communicating with the user end 306, the content storage system 304, and a registration system 326.
  • the communication interface system may comprise separate modules for communicating with each of these entities or it may do so over a single point of entry/exit, such as a high bandwidth connection to the internet.
  • the content management system 302 further comprises processing logic 316.
  • the processing logic 316 can be dedicated hardware with hard- or firm-coded instructions such as an FPGA, but is more likely a general-purpose processor controlled by software instructions tangibly stored in a computer-readable storage medium such as local storage 314 instructing the general-purpose processor to behave as described herein.
  • Communication between the processing logic 316, the local storage 315 and the communication interface system 318 can be done by any suitable manner but in the present example they are linked by internal buses. This is similarly the case in the content storage system 304 and the content access system 328.
  • the content management system 302 be implemented by a distributed network of processing resources accessing a distributed network of storage resources (e.g. a cloud) instead of the local storage 314.
  • the content management system 302 may implement a method for managing access to viewable stereoscopic content in a digital content library by a remote user application (described in more details further below) for viewing at a remote viewing environment characterized by a set of user viewing parameters.
  • the processing logic 316 is configured to access the records of stereoscopic content in the stereoscopic content database 320 and determines the presence in the digital content library 308 of a plurality of versions of the stereoscopic content each having a respective stereoscopic configuration corresponding to respective sets of viewing parameters.
  • the communication interface system 318 being adapted to communicate with the remote user application receives from the user application digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters.
  • the digital viewing parameter data are received from the user end 306 via a registration system 326, which will be described in more detail below.
  • the digital viewing parameter data is indicative of a display size provided during registration.
  • VOD Video-on- Demand
  • Registration for the stereoscopic content distribution system 300 is done via a secondary device 324 separate from the device used to access the content.
  • Many VOD services like 3DGO!TM allow access to video content directly on a smart TV.
  • a secondary device 324 separate from the TV, may be used to provide registration information.
  • Some VOD services, such as NetflixTM are multi-platform services which allow viewing on several devices.
  • the secondary device may actually also be used as a content access device implementing a remote user application / content access system 328, but for the purpose of this example, in order to better illustrate the registration system 326, the secondary device will simply be considered a separate device from the one implementing the remote user application. Also for the purpose of describing the registration system 326, it will be assumed that the content access system 328 implementing the remote user application is in a smart TV, although as will be clear in the description of the content access system 328, this does not need to be the case and should not be construed as limiting.
  • registration to VOD service is done via the world wide web (web).
  • web the world wide web
  • a user wishing to access the stereoscopic content in the digital content library 308 on his smart TV must first register online using a web browser.
  • a registration system 326 is provided.
  • the registration system 326 is a web server.
  • the skilled addressee will understand all the variations possible in this regard.
  • the registration system 326 is in communication with the content management system 302 and the secondary device 324 via a communication interface system 332 typical of a web server.
  • the registration system 326 further comprises processing logic 334 in the form of a processor acting upon instructions stored tangibly in a computer-readable storage medium, such as the storage 336, that instruct the processing logic to perform the tasks described herein thereby making is to configured.
  • the registration system 326 further comprises website and form data 338 in the storage 336, which may be local storage as shown or otherwise distributed according to any suitable storage system.
  • the communication interface 332 being suitable for a website host is capable of bidirectional communication with the secondary device 324 which is a remote user device.
  • the registration system 324 established bidirectional communication with the secondary device 324. This may be done in the classic way when a user at the secondary device 324 directs a web browser running on the secondary device 324 to an address designating the registration system 324.
  • the processing logic 334 causes the transmission of the registration website to the secondary device 324.
  • the processing logic accesses the storage 336 to extract the website data which includes form data 338 and causes its transmission to the secondary device 324 using the communication interface system 332.
  • the form data 338 comprises queries prompting the inputting of registration information by the user of the secondary device 324.
  • the registration information entered at the secondary device 324 may include unique or semi-unique identifiers such as a name, address.
  • the queries may include text boxes for entering such data.
  • the registration information may also include hardware information identifying the hardware on which the stereoscopic content will be viewed or on which the content access system 328 will run, or a software platform on which it will run. For such input a text box or roll-down menus may be used.
  • the registration information include digital viewing parameter data indicative of at least one viewing parameter characterizing the remote viewing environment at which the stereoscopic content provided by the stereoscopic content distribution system 300 will be viewed.
  • digital viewing parameter data is indicative of a dimension of the display on which stereoscopic content will be viewed.
  • the registration form contains a query prompting a user of the secondary device 324 to enter in a text box, or to select from a roll-down menu, a diagonal length of the display.
  • the registration information which comprises the digital viewing parameter data
  • the registration system 326 Upon entering the registration information by the user of the secondary device 324, the registration information, which comprises the digital viewing parameter data, is sent to the registration system 326 where it is received by the communication interface system 332 and processed by the processing logic 334 which causes the registration information to be associated with a unique user account in the user database 322 at the content management system 302, where the unique user account also comprises the digital viewing parameter data, or at least information indicative of the user parameter(s) indicated in the digital viewing parameter data, such that the content management system 302 can select on the basis of the digital viewing parameter data the version of the stereoscopic content when such stereoscopic content is requested from the user end 306 on the basis of the user viewing parameters.
  • the registration system 326 is in communication with the content management system 302 via the communication interface system 332.
  • the processing logic 334 causes the association of the registration information with a unique user account at the content management system 302 by transmitting the registration information comprising the digital viewing parameter data together to the content management system 302 for association thereat by the content management system 302.
  • the user account may be generated by the content management system 302 and associated to the registration information by the content management system 302 upon prompting by the registration system 326 by being provided with fresh registration information.
  • the processing logic 334 is configured to generate a unique user account and associating it to the registration information comprising the digital viewing parameter data and providing the unique user account details to the content management system 302 via communication interface systems 332 and 318 for storage in the user database 322 at the content management system 302. Creation of a unique user account may include generating an entry as described above for the user database 322.
  • the registration information may optionally include payment information, such as credit card details.
  • the registration form includes prompts for entering payment information including text boxes for a credit card number, name and CCD, roll down menus for the expiry month and year, and radio buttons to select a type of card.
  • the payment information is not sent to the content management system 302, but rather sent directly from the registration system 326 to a payment authority server which processes the information.
  • the payment information may be used for actual payment, or for verifying the uniqueness/authenticity of a registration request or both. If the user is registering for a fixed-fee service (e.g. with a monthly fee), the payment authority processes the payment and provides a confirmation to the registration system 326 and/or the content management system 302.
  • the registration system 326 may forward the confirmation alongside the registration information or user account to the content management system 302.
  • the payment information may for example merely serve as a unique identifier to which to tie the user account, or it may be kept on file by the payment authority for future purchases/rentals.
  • the payment authority may link the payment information to a unique payment information identifier and return this to the content management system 302 or registration system 326 which then treats it similarly to the aforementioned confirmation.
  • payment information may be simply stored and associated to the unique user account, similarly to the digital viewing parameter data, to be communicated to the payment authority to process a payment as needed by the content management system 302.
  • the payment information is provided to the content management system 302 by the registration system in the same manner as the rest of the registration information.
  • the digital viewing parameter data may include information indicative of a version of stereoscopic content.
  • the digital viewing parameter data entered at registration will not necessarily dictate the version employed every time the stereoscopic content is obtained over the stereoscopic content distribution system 300.
  • the digital viewing parameter data entered at registration may be indicative of a user preference, which serves by default for the content management system 302 to select a version but can be overridden by providing from the remote user application along with a program selection another digital viewing parameter data in the form of a actual viewing parameter(s) or a version selection.
  • the user of the secondary device 324 is not necessarily the user of the content access system 328 or indeed the viewer associated with user viewing parameters provided in the digital viewing parameter data.
  • the person registering for VOD services is not necessarily the one that will be watching the stereoscopic content every time or ever.
  • the user viewing parameters are associated to him insofar as he is entering them into the registration process but calling them associated to him should not imply that they necessarily apply to him. He could be registering for the content access for himself or for his family or for someone else.
  • the form needs not be presented in a single instance (single web page) but may be provided in multiple segments or portions, for example in a page- by -page sequence with each page comprising a different portion of the form.
  • the content management system 302 receives a request from the remote user application for a particular stereoscopic content.
  • the remote user application is implemented on a content access system 328 which may control access to, e.g., VOD content on a device such as a TV, a computer, a HMD, or a handheld mobile device.
  • the remote user application first logs into the system. This may be done by sending a log-in message providing user identification.
  • the processing logic 316 of the content management system 302 accesses the records of the stereoscopic content in the stereoscopic content database 320 and transmits the list of available stereoscopic content to the remote user application.
  • the processing logic may simply provide the complete list of available stereoscopic content (if, for example, all the films are available for rental from anyone) or it may first access the user database 322 to verify user credentials and provide only the list of films accessible to the user account associated with the remote user application.
  • the skilled addressee will understand that such communication may be performed by an application system like that of Figure 16 and may be done in several steps. For example the list of available title may be presented to a user via the remote user application in successive panels each comprising a different category of programs and may therefore be provided portion- by -portion as the user browses the catalogue.
  • the processing logic 316 of the content management system 302 accesses the stereoscopic content database 320 and sends using the communication interface system 318 to the remote user application with a selection of programs from which to select.
  • a selection of a title is made and transmitted to the content management system 302.
  • a communication interface system 330 in the content access system 328 is made to transmit an indication of a selected program to the content management system 302, which is received at the communication interface system 318 and processed by the processing logic 316 which uses this indication (e.g. and identifier) to identify the selected program.
  • the processing logic accesses the user database 322 to obtain the digital viewing parameter data and selects on the basis of the digital viewing parameter data a version of the selected stereoscopic content to be transmitted to the remote user application.
  • the processing logic 316 selects a version of the stereoscopic content from among the plurality of versions of the stereoscopic content listed in the corresponding record in the stereoscopic content database 320 by identifying a set of viewing parameters from among the respective sets of viewing parameters provided in the corresponding record that best corresponds to the user viewing parameters identified in the digital viewing parameter data.
  • Selecting a version may simply entail looking up which version corresponds to the exact user viewing parameter provided, if the user is only permitted to provide viewing parameters corresponding to existing versions of the stereoscopic content.
  • the user provided an exact display size (in terms of diagonal length) during registration but the digital content library 308 only contains four versions of each stereoscopic program corresponding to different display size range.
  • Finding the best version therefore entails finding the display size range in which the user's digital viewing parameter data falls.
  • Other methods could include finding the size closest to the one in the digital viewing parameter data if each version corresponds to a specific size. If multiple viewing parameters are present, a multidimensional closest approximation can be found or a hierarchical layered approach may be used (e.g. find the closest IOD, then the closes display size, then the closets VD).
  • the processing logic 316 provokes the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
  • the processing logic 316 does this by sending an instruction to the content storage system 304 via communication interface systems 318 and 312 to transmit the selected stereoscopic content in the selected version from the digital content library 308 to the user end 306 and particularly to the remote user application.
  • the instruction includes the necessary information for the content storage system 304 to know where to send the data, for example it may contain an address for the remote user application or the information to facilitate a handshaking between the content storage system 304 and the content access system 328.
  • the processing logic 316 may provoke the transmission of the stereoscopic content in the selected version from the digital content library 308 to the user end 306 and more particularly the remote user application by providing the remote user application with a token with which to access the selected stereoscopic content in the selected version.
  • the token may be a decryption key, whereby the remote user application can request and access any content from the content storage system 304 but will be only able to decrypt the content and version corresponding to the received decryption key, or the token may be an authorization token attesting to authorization from the content management system 302 to access the selected stereoscopic content in the selected version (and perhaps identifying it).
  • the remote user application Upon receiving the authorization token, the remote user application transmits it to the content storage system 304 via communication interface systems 330 and 312 whereupon when the control 310 of the content storage system 304 receives the authorization token, it determines that the remote user application is indeed authorized to access the selected stereoscopic content in the selected version and transmits it.
  • the transmission of the stereoscopic content in the selected version is done from the content storage system 304 to the remote user application by streaming as is common in Video-on-Demand (V OD) systems.
  • V OD Video-on-Demand
  • the transmission could be a file download from the remote user application. This could even be a file download in the case of a rental provided that the file and/or remote user application is protected by suitable mechanisms to prevent viewing of the content outside of the rental period/parameters.
  • hardware information identifying the hardware on which the stereoscopic content will be viewed or on which the content access system 328 will run, or a software information identifying a software platform on which it will run can be provided to the content management system 302.
  • such information can be used as digital viewing parameter data.
  • hardware information may be indicative of at least one viewing parameter. For example, if the hardware information includes a model number for a television or other device, it is possible to ascertain the size of an associated display.
  • the content management system 302 (or registration system 326) may have access to a lookup table (stored locally or remotely, e.g.
  • the digital viewing parameter data received from the remote user application may simply be the hardware identifier, which may have been provided to the remote user application by its user, but which in the case of a remote user application linked to the display device on which the stereoscopic content will be viewed (such as a VOD app running on a smart TV) may simply be automatically sent to the content management system 302 without requiring input from a user.
  • the digital viewing parameter data could be a software identifier indicative of a software platform insofar as a software platform may be indicative of at least one user viewing parameter such as a display size.
  • the digital viewing parameter data may be provided automatically by the remote user application without input from a user.
  • the content storage system 304 transmits the requested stereoscopic content in the selected version directly to the user end 306. It should be understood that according to a different architecture, transmission of the stereoscopic content could pass through the content management system 302.
  • the processing logic 316 could simply request the content from the content storage system 304, wherein the control 310 of the content storage system 304 is configured to unquestionably honor any such request from the content management system 302 and wherein the processing logic 316 of the content management system 302, upon receipt of the content from the content storage system 304 (via communication interface systems 318 and 312) transfers the selected stereoscopic content in the selected format to the user end 306 and more particularly to the remote user application via communication interface system 318.
  • the stereoscopic content contained in the digital content library 308 is only contained in a single version (typically the original version) and is reconfigured at the content management system 302 by a reconfiguration module.
  • a reconfiguration module An example of such real-time reconfiguration will be provided further below.
  • the user end 306, as shown in Figure 17 comprises a content access system 328 and, in this example a secondary device 324 for registration that is conceptually considered at the user end 306 but is not necessarily physically located at or near the control access system or viewing environment.
  • the content access system 328 comprises a communication interface system 330, processing logic 340 and local storage 342.
  • processing logic 340 processing logic 340
  • local storage 342 local storage
  • the content access system 328 implements a remote user application.
  • the content access system 328 is located on a smart TV, whereby the communication interface system 330 comprises the smart TV's WiFi interface and wired network interface.
  • the processing logic 340 can be dedicated hardware with hard- or firm-coded instructions such as an FPGA, but is more likely a general-purpose processor controlled by software instructions tangibly stored in a computer-readable storage medium such as local storage 342 instructing the general-purpose processor to behave as described herein.
  • the processing logic is contained within a system-on-a-chip (SOC) contained in the smart TV.
  • SOC system-on-a-chip
  • the processing logic is linked to the communication interface system 330 by an appropriate logic path such as a bus.
  • the local storage 342 is flash memory accessible by the processing logic 340 via a flash interface (not shown).
  • the content access system 328 includes a display 344, which is the display of the smart TV. It is shown in the figure as optional since the content access system 328 which controls access to the stereoscopic content offered by the stereoscopic content distribution system 300 is not necessarily on the same device as will display the stereoscopic content. Indeed, VOD applications may run on devices other than TVs, computers and handheld communication devices. Set-top boxes, Blu-rayTM players and Apple TV-type devices may be used to access the stereoscopic content in the digital content library 308 despite not having a display. Moreover even the presence of a display does not necessarily mean that the stereoscopic content is to be viewed on the device embodying the content access system 328.
  • GoogleTM Chromecast allows control of content streaming to a TV (or other such device) using another connectivity device such as a phone or computer, which itself also has a display capable of displaying video.
  • content access may be controlled using a first device (say, a mobile phone) embodying the content access system 328 while the stereoscopic content is streamed to a second device (say a television), yet the stereoscopic content may still be said to be transmitted to the content access system 328 and is received by the content access system 328 insofar as some information on the content such as video progress information is sent to the content access system 328 and that the transmission of content is effectively under control of the content access system 328.
  • a first device say, a mobile phone
  • a second device say a television
  • the display 344 which may or may not be a part of the content access system 328, is a part of the viewing environment as illustrated in Figure 16.
  • the content access system 328 implements a remote user application.
  • the remote user application takes the form of a smart TV app stored as program instructions on a computer-readable storage medium, here a flash memory, configuring the processing logic 340, here a SOC's processor via instructions to perform as described herein.
  • the processing logic 340 is in this manner configured to communicate via the communication interfaces 330 and 318 with the content management system 302 in order to manage access to stereoscopic content by a viewer for viewing on the display 344.
  • the remote user application may be operated by a user that provides user input using an input device.
  • Any suitable input device for interacting with the content access system 328 and provide the remote user application with input may be used. These include keyboard and mouse for computers and touchscreens with appropriate manipulateable interface elements for handheld devices but in this particular example the user input device is a remote control which interacts with the smart TV via an infrared port.
  • the remote user application interacts with the user by receiving input from the user input device and providing information over the display 344. It should be understood, as per the above discussion, that it is not necessary that the display of the content access system 328 be the same display that will be used for displaying the stereoscopic content but in this case it is.
  • a graphical user interface enable the functionality of the remote user application to be provided to a user.
  • Figures 6a and 6b provide an example of a graphical user interface 600 according to a non-limiting embodiment.
  • the graphical user interface 600 comprises a first pane 602 and a second pane 610 that do not, in this case, overlap.
  • the boundary 612 between the first and second panes 602 and 610 drawn out in this example but this does not need to be the case.
  • the instructions for controlling the processing logic to implement the remote user application are loaded from the flash memory into DRAM (or, alternatively SRAM) memory for execution.
  • the programing logic connected in this example to the display 344 by an LVDS link via a T-con board then produces the graphical user interface 600 n the display 344.
  • the first pane 602 displays a plurality of first visual elements 604, each of which represent a category of stereoscopic program, in this case categories of movies.
  • the visual elements in this case are textual icons indicating a category.
  • an input element associated thereto is operable by using a user input device.
  • the input element is the icon itself, which can be selected by navigating on it with arrows on the remote control (which shift between selected icons in the first pane 602 by pressing up or down and shift in and out of the first pane 602 by pressing left or right) and activated by pressing an "enter” key thus operating ("clicking") the input element.
  • a category pane may be open by default.
  • the processing logic receives from the content management system 302 a list of programs available to it (e.g. for rental or under a current subscription or both) in the digital content library 308.
  • the processing logic 340 sends a request to the content management system 302 for the list of programs available via the communication interfaces 330 and 318, and the content management system 302 accesses the stereoscopic content database 320, optionally applying user credentials obtained from the user database 322 to compile the list and sends it back via the same path to the content access system 328.
  • the second pane 610 displays a plurality of second visual elements 614 each being representative of a stereoscopic program in the category.
  • the second visual elements 614 are movie titles.
  • the second pane also comprises for each second visual element a second input element 615 being operable by a user using the input device to select the program.
  • the input elements 615 are not overlapping the visual elements 614 but are image icons, albeit visual as well, located above the title. Clicking in the same manner as described above on such an icon selects the corresponding movie and brings up the third pane 618 which replaces the second pane 610 while, in this example, the first pane 602 remains visible.
  • the third pane comprises a visual element displaying textual information about the particular selected stereoscopic program.
  • textual information may be requested by the content access system 328 from the content management system 302 upon selection of the film.
  • the third input element is a big "rent" button operable by clicking as described above.
  • Other further confirmation and payment screens may be additionally provided as desired.
  • a version visual element 622 indicating the availability of a plurality of versions of stereoscopic content, each of which corresponds to different configurations as described above.
  • the version visual element 622 indicates the presence of a child-safe mode.
  • the graphical user interface 600 further includes a version input element operable by the user using the input device 624 to select a version from amongst the plurality of available version. Since in this example there is only two version, the original and child-safe version a check box indicating a desire to view the stereoscopic content in the child-safe configuration is operable by selecting it with directional arrows on a remote control and clicking it with the "enter" button.
  • the graphical user interface 600 may also include a progress a control system, such as a progress bar and control buttons.
  • Control buttons may include buttons for selecting a version (e.g. original and child-safe) or toggle a version (child-safe on or off) directly on the progress and control system, which may allow switching between version (which may involve switching between streaming files) in realtime.
  • the version visual element 622 could be presented in a fourth pane such as a pop-up pane in response to actuation of the third input element 620.
  • the forth pane may include the version input element 624.
  • Other modes for presenting available version may be used, such a list of representations (e.g. textual names/descriptions) of selectable versions which can be browsed and selected using the user input device.
  • the graphical user interface 600 may provide input prompts for entering viewing parameters which will be sued as describe above to select a version. This can include text boxes which can be selected by "clicking" as described above, and into which text can be written by known textual entry methods using the remote control, e.g. the numeric keypad. Textual fields may include the age of a viewer, a viewing distance, dimensions/size of the display or a plurality of these.
  • the graphical user interface 600 can be split between the content access system 328 and another device as when a smartphone is used to control what is being displayed on a TV using GoogleTM Chromecast.
  • all graphical user interface 600 functions including those for content selection and playback control are provided on the smartphone display which is separate from the TV's display but in other embodiments the graphical user interface 600 could be split between two display.
  • the content access system 328 now has access to a list of content available in the digital content library 308 and has means of receiving an input for selection of a particular program to be viewed on the display 344.
  • the processing logic causes the transmission of a request for the particular stereoscopic content to the content management system 302. It does so by formulating the request and sending commands to the communication interface system 330 to transmit it to the address of the content management system 302 (and more particularly the communication interface 318 of the content management system 302).
  • the content management system 302 receives the request at the communication interface 318 and the processing logic 316 of the content management system 302 operating as described above ascertains the presence of multiple versions of the requested stereoscopic content in the digital content library 308 by consulting the stereoscopic content database 320.
  • the content access system 328 also causes the transmission using the communication interface system 330 to the content management system 302 of digital viewing parameter data indicative of at least one of the viewing parameters from the set of viewing parameters that will define the viewing environment at which the stereoscopic content will be viewed.
  • data indicative of viewing parameters were provided to the content management system 302 via a registration system 326 by a user operating a secondary device 324.
  • the digital viewing parameter data could also be provided directly by the content access system 328. As mentioned above, this could be additionally to the transmission of digital viewing parameter data at registration, (e.g. in the case where the digital viewing parameter data provided at registration is merely a preference to be used as default) e.g.
  • viewing parameters in a settings section of the remote user application implemented by the graphical user interface 600
  • signaling viewing parameters other than those provided at registration e.g. by selecting a particular version (e.g. selecting a child-safe version when the digital viewing parameter data provided at registration did not specify a child IOD) or by inputting when given the opportunity by the graphical user interface 600 a new digital viewing parameter data that is to override the default one.
  • the digital viewing parameter data are not provided at registration but rather are provided by the content access system 328 only.
  • the graphical user interface 600 offers a choice of a child-safe version.
  • the content access system 328 receives knowledge of the versions available for a selected stereoscopic content from the content management system 302 in order to present them to the user.
  • transmission of the request for content precedes the transmission by the content access system 328 of the digital viewing parameter data.
  • the digital viewing parameter data takes the form of the selection of a version.
  • the selection a version of stereoscopic content that is associated with certain viewing parameters acts as an indication of user viewing parameters corresponding at least in part to, or being closest to those of, the selected version and therefore an indication of the selection, when transmitted from the content access system 328 to the content management system 302 serves as digital viewing parameter data.
  • This kind of digital viewing parameter data is called version-association representation whereas when the digital viewing parameter data refers directly to the value (exact, or indication thereof such as a range or approximation) of an actual viewing parameter (e.g. dimension of display, VD, IOD or resolution) this is called a direct representation.
  • the digital viewing parameter data may include a configuration identifier indication a particular version to be selected from among a set of versions.
  • Selection of a version of the stereoscopic content may left up to the content access system 328, rather than the content management system 302 to determine.
  • the set of versions of the stereoscopic content may be received at the communication interface system 328 (e.g. being transmitted by the content management system 302 where it was derived by the processing logic 316 by consulting the stereoscopic content database 320) and the processing logic 340 may be configured to identify from among the set of versions a selected version of the stereoscopic content that has a configuration corresponding to the viewing parameters that best correspond to the user viewing parameters.
  • the indicator of the selected version is indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version.
  • the remote user application may provide to the user, via graphical user interface 600 on a viewing device such as the display 344 of a visual prompt requesting a user to enter at least one user viewing parameter and wherein in response to receiving the at least one user viewing parameter the processing logic 340 generates the digital viewing parameter data on the basis of the user viewing parameter inputted by the user.
  • the digital viewing parameter data may include the exact data (e.g. size of display) entered by the user or may include merely a representation of it (e.g. by providing the range into which the size entered by the user falls).
  • the processing logic 340 may generate the digital viewing parameter data without user input.
  • the digital viewing parameter data may include or be derived from a hardware or software identifier. Such an identifier can be hard- coded or hard-wired into the processing logic 340 thus eliminating the need for user input.
  • the processing logic causes the transmission of the digital viewing parameter data to the content management system 302 by generating the digital viewing parameter data and instructing the communication interface system to transmit it to the address of the content management system 302 (and more particularly to the communication interface system 318 of the content management system 302).
  • the content management system 302 selects a version of the requested stereoscopic content in a particular configuration on the basis of the digital viewing parameter data and causes its transmission to the content access system 328 as described herein.
  • the content access system 328 receives at the communication interface 330 the requested stereoscopic content in the particular configuration and causes it to be displayed on the display 344.
  • the communication interface system 330 may be in communication with the content storage system 304 for receiving stereoscopic content, for example in streaming form.
  • the requested stereoscopic content in the particular configuration may be received directly from the content storage system 304 in this embodiment.
  • the stereoscopic content in the particular configuration may alternatively be also received from the content management system 302.
  • the stereoscopic content may be in streaming form (e.g. for movie rentals) or in a file download form (e.g. for movie purchases).
  • the content transfer may be initiated by the content storage system 304 upon receiving instructions from the content management system 302 as described above, or it may be initiated by the content management system 302 by setting up a handshaking procedure between the content storage system 304 and content access system 328.
  • the content access system 328 receives from the content management system 302, in response to the transmission of a request for stereoscopic content and a digital viewing parameter data, an authorization token.
  • the processing logic 340 uses the authorization token to generate a request to the content storage server for the stereoscopic content the request comprising the authorization token and causes the communication interface system 330 to transmit the request to the content storage system 304 (and more particularly to the communication interface system 312 of the content storage system 304).
  • the stereoscopic content in the version selected by the content management system 302 is then transmitted from the content storage system 304 to the content access system 328.
  • Figure 19 shows a process/data flow according to another non-limiting embodiment.
  • the content management system 302, content storage system 304 and user end 306 are similar to those described above and shown in Figure 17 with the exception that the registration system 326 is absent because the service registration is performed directly using the content access system 328 instead of a secondary device 324.
  • the registration process is not shown in Figure 19, it is to be understood that the process is similar to that described in relation to the registration system 326 but the registration information is gathered on the content access system 328 instead of the secondary device 324 and is transmitted directly from the content access system 328 to the content management system 302 (via communication interface systems 330 and 318) instead of through a registration system.
  • a user may register for VOD services by entering registration information directly into his smart TV using the remote user application (e.g. VOD application) in his smart TV.
  • the remote user application may present to the user prompts similar to those contained in the form data 338 of the registration system 326 using the graphical user interface 600, and may receive the registration information from a user entering it using a user input device functioning as described above.
  • a user first starts the remote user application by, for example, opening the VOD application on a smart TV. From the remote user application, a log-in message is sent to the content management system 302.
  • the processing logic 340 of the content access system 328 generates the log-in message using login information stored in local storage 342 or inputted by the user and causes the communication interface system 330 to transmit it to the content management system 302. It may also send a request for a content list, e.g. as described in relation with the description of the graphical user interface 600, or the log-in message itself may serve to prompt the content management system 302 to send the content access system 328 a content list.
  • the content management system 302 identifies the user and the user credentials.
  • the processing logic 316 of the content management system 302 uses the login information to identify a corresponding user account in the user database 322 and corresponding user credential.
  • the content management system 302 comprises and stereoscopic content database 320 only temporarily, and does not store it long-term in the local storage 314. Nonetheless, it does obtain temporarily the stereoscopic content database 320 in order to provide the remote user application a list of stereoscopic content available.
  • the stereoscopic content database 320 is essentially the whole list of stereoscopic content available to the remote user application.
  • the content management system 302 queries the digital content library 308 based on those credentials.
  • the processing logic 316 of the content management system 302 generates a query message 506 for the control 310 of the content storage system 304 and instructs the communication interface system 318 to transmit the message to the content storage system 304 and more particularly to the communication interface system 312 of the content storage system 304.
  • the query message includes information on the user credentials and requests records of all the stereoscopic content in the digital content library 308 that satisfies the credentials.
  • the control 310 of the content storage system 304 compiles an stereoscopic content database 320 comprising all the stereoscopic content in the digital content library 308 satisfying the user credentials and returns it in message 508 to the content management system 302 via the communication interface systems 312 and 318
  • the processing logic may modify the information in the stereoscopic content database 320 to generate a list 510 to transmit to the remote user application or, if the stereoscopic content database 320 is already suitably formatted, it may transmit the stereoscopic content database 320 as the list directly.
  • a stereoscopic content database 320 comprising records of all stereoscopic content in the digital content library 308 could be stored by the content management system 302 and that the processing logic 316 can compile the list of stereoscopic content available for the remote user application by consulting the stereoscopic content database 320 using the user credentials to identify content that should be included in the list.
  • the remote user application presents a choice of stereoscopic content to a user, for example as described in the discussion relating to Figures 6a and 6b.
  • a user selection is received, also for example as described in the discussion relating to Figures 6a and 6B, and the selection, or an indication of the selected stereoscopic content is transmitted by the remote user application to the content management system 302.
  • the processing logic 340 of the content access system 328 generates a selection identification message 512 and instructs the communication interface system 330 of the content access system 328 to transmit it to the content management system 302 and more particularly to the communication interface system 318 of the content management system 302.
  • the viewing parameter data is requested in the form of a selection of a version.
  • the content management system 302 identifies available version of the selected stereoscopic content (for example the processing logic queries the stereoscopic content database 320 kept since it was received in 508 or the content management system 302 sends another query (not shown) to content storage system 304 to identify available versions).
  • Each version corresponds to difference configurations adapted to different viewing parameters.
  • the content management system 302 sends to the content access system 328 a list of the versions of the requested stereoscopic content that are available.
  • the remote user application obtains presents to the user via graphical user interface 600 a representation of the versions available and receives from the user via the user input device a selection of a version.
  • the selected version is considered indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version and may accordingly be consider digital viewing parameter data although even though it may be called a version-association representation of a viewing parameter.
  • the list of versions available may be, as represented in Figure 20b, an adult (or original) version and a child-safe version.
  • each film is offered in an original version and a child-safe version the child-safe version being a reconfigured version of the original to adapt to a viewing environment where the viewer interocular distance is a child interocular distance.
  • interocular distance is one of the parameters for which capture/synthesis parameters are typically adjusted most stereoscopic content is configured for, inter alia, a particular interocular distance. This is typically 65 mm or thereabouts. Indeed the typical interocular distance for adults is around 54-68mm. However, the typical interocular distance for children is much smaller, around 41-55 mm.
  • each stereoscopic film in the digital content library 308 comprises an original version and a child-safe version reformatted from the original version to account for a smaller IOD.
  • both versions are identified, the one as an adult version, which indicates as a viewing parameter an adult IOD, and the other as a child-safe version, which indicates as a viewing parameter a child IOD.
  • the selection of the version (e.g. the selection or lack of selection of the child-safe version by the user using the user input device) is considered to be indicative of a user viewing parameter insofar as it is indicative of whether the viewer is an adult or a child and therefore is indicative of the IOD of the viewer and more specifically of whether the IOD is an adult IOD or a child IOD.
  • This selection is placed in a message 516 sent from the content access system 328 to the content management system 302.
  • the processing logic 340 generates the message 516 containing an identification of a selected version of the stereoscopic content and instructs communication interface system 330 to transmit it to the content management system 302 or more particularly to the communications interface system 318 of the content management system 302.
  • the message 514 may simply request viewer parameter data, in response to which the content access system 328 may provide in message 516 direct-representation digital viewing parameter data.
  • each film is still offered in an original version and a child-safe version but instead of sending the list of versions and requesting a selection, the content management system 302 transmits in message 514 a request for direct-representation digital viewing parameter data, and more particularly an indication of an IOD.
  • the remote user application may prompt a user for user viewing parameter data as described herein and more specifically for IOD data, or may find this data within local storage 342 if it has been previously recorded (as it should be mentioned may be the case not only with IOD data but indeed with any variation of digital viewing parameter data).
  • the remote user application then generates the message 516 comprising digital viewing parameter data indicative of an interocular distance of a viewer.
  • the digital viewing parameter data may be indicative of the age of a viewer, the age of the viewer being indicative of and interocular distance.
  • the age indication may be a Boolean type of value (adult or child) it may also represent different age ranges characterized by different typical interocular distance.
  • the processing logic 316 of the content management system 302 selects the appropriate version of the selected stereoscopic content on the basis of the received digital viewing parameter data.
  • the next steps are similar to those described in the example provided with Figure 17.
  • the content management system 302 requests the content storage system 304 to transmit the selected stereoscopic content in the selected version to the remote user application.
  • processing logic 316 of the content management system 302 generates a message 520 identifying the selected content and the selected version thereof, for example by providing a location indicator for the selected version of the selected content, which may have been previously included in the stereoscopic content database 320 provided by the content storage system 304 in message 508.
  • Processing logic 316 then instructs communication interface 318 to transmit the message 520 to the content storage system 304, and more particularly to the communication interface 312 of the content storage system 304.
  • the content storage system 304 transmits the selected version of the selected content to the remote user application and more particularly control 310 causes the transfer of the selected version of the selected content from the digital content library 308 through the communication interface 3123 to the content access system 328 and more particularly to the communication interface system 330 of the content access system 328.
  • the different versions are universal, that is, they are shared by all programs in the digital content library 308. As such, identifying the different version available can be simplified to simply knowing what the universal versions are. This can be stored in the local storage 314 of the content management system 302 or the content storage system 304 can provide this information upon being queried.
  • the stereoscopic content is transmitted to the content access system 328 directly from the content storage system 304, it is to be appreciated that in order to afford greater control from the content management system 302, the stereoscopic content could be transferred to the user end 306 from the content management system 302. In such a case the content is transmitted from the content storage system 304 to the content management system 302 (via communication interfaces 312 and 318 prior to transmission by the content management system 302 (via communication interface 318) to the user end 306.
  • the functionality of the content management system 302 and the content storage system 304 may be combined in one single entity, a content management system 702 that contains the digital content library 308.
  • the digital content library 308 is shown here in its own storage medium, presumably a server storage bank and is separate from the local storage 314 which still contains the user database 322. Of course the two could be in the same physical storage media. Since the content management system 702 comprises the digital content library 308, the stereoscopic content database 320 has been omitted since the processing logic 716 has access to the contents of the digital content library 308 directly.
  • the content management system 702 may still comprise the stereoscopic content database 320 (not shown), for example in the local storage 314 as was the case with content management system 302 .
  • the content management system 702 communicates directly with the user end 306 and in particular with the content access system 328.
  • the content access system 328 remains relatively unchanged, with the exception that communications that were previously described as being between it and the content management system 302 and the content storage system 304 are now both between it and the content management system 702.
  • the communication interface 718 of the content management system 702 embodies the functions of both the communication interfaces system 318 and 312 except, of course the function of communicating between the communication interfaces system 318 and 312.
  • a stereoscopic content distribution system 700 shown in Figure 21 may include a reconfigurator 704 for doing the reconfiguration of stereoscopic content for example in the manner taught by the aforementioned copending application.
  • reconfigurator 704 is a real-time reconfigurator as taught in the aforementioned copending application and is used by the content management system 702 to reconfigure in rea-time the stereoscopic content contained in the digital content library 308.
  • the digital content library 308 needs only store one version of all stereoscopic content, e.g. an original version, and new versions are created on the fly in real-time in response to, and adapted for, the received digital viewing parameter data.
  • the content management system 702 may receive the stereoscopic content in a first (e.g. original) configuration. This may be, for example, received as studio files as described above or otherwise inputted into the digital content library 308. The content management system 702 may then receive digital viewing parameter data from the content access system 328 as described above. Using the digital viewing parameter data the processing logic 716 may determine a configuration suitable for viewing in the remote environment at the user end 306.
  • a first (e.g. original) configuration This may be, for example, received as studio files as described above or otherwise inputted into the digital content library 308.
  • the content management system 702 may then receive digital viewing parameter data from the content access system 328 as described above. Using the digital viewing parameter data the processing logic 716 may determine a configuration suitable for viewing in the remote environment at the user end 306.
  • the processing logic 716 then causes the performing of a reconfiguration operation by the reconfigurator 704 to generate a second stereoscopic configuration of stereoscopic content, the second reconfiguration corresponding to at least one parameter from a set of user viewing parameters defining a viewing environment at the user end, and of which the digital viewing parameter data was indicative of at least one viewing parameter.
  • the stereoscopic content in the second configuration is made to be transmitted to the content access system 328, in this case via the communication interface 718.
  • the stereoscopic content distribution system 700 may be a hybrid model whereby pre-reconfigured versions of stereoscopic data are stored in the digital content library 308 for the most common viewing environments and when uncommon digital viewing parameter data is received a special version of the stereoscopic content is reconfigured in real-time for the requestor.
  • the reconfigurator 704 may not be called upon to reconfigure in real- time but may simply be present to reconfigure stereoscopic content received at the content management system 702. Indeed, studio files being typically in an original version, it may be necessary to actually generate the reconfigured version of each program in order to be able to offer them.
  • the stereoscopic content distribution system 700 may include a reconfigurator 704, for example in the content management system 702, in order to generate reconfigured version of the stereoscopic content to make available to end users.
  • the reconfigurator may implement, for example the high-quality reconfiguration scheme provided in the aforementioned copending application.
  • Reconfigured versions of stereoscopic content may be subjected to all the modules and process steps described in relation to the content storage/provisioning system 210 illustrated in Figure 16 including quality checks, and multibitrate coding and encryption.
  • reconfigured versions of stereoscopic content may be subjected to an additional quality control step to verify the quality of the reconfiguration process itself and in particular to check for artefacts and infidelities that may be caused by the reconfiguration. This may include objective or subjective analyses.
  • the content access system 328 may be a hardware system such as the smart TV described above or other hardware system comprising a processing unit, and a network interface.
  • the processing unit may be a programmable processing unit configured by software instructions physically residing on software storage media which may be the local storage media and instructing the processing unit to perform implement a remote user application and to perform as configured.
  • the content access system 328 may also be a software system implementing a remote user application, wherein the communication interface system is a set of software instruction residing on software storage media for instruction a software- programmable device having a network interface to communicate over the network interface, and wherein the processing logic comprises software instructions residing on software storage media instructing a processing unit in the software-programmable device to perform the configuration defined by the software instructions.
  • the communication interface system is a set of software instruction residing on software storage media for instruction a software- programmable device having a network interface to communicate over the network interface
  • the processing logic comprises software instructions residing on software storage media instructing a processing unit in the software-programmable device to perform the configuration defined by the software instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Library & Information Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

A system for distributing stereoscopic video-on-demand content comprising a content management server having a digital stereoscopic content database comprising a set of records of stereoscopic content held in a digital content library; a digital content library storing the stereoscopic content; a remote user application in communication with the content management server and the content storage server at a remote viewing environment characterised by a set of user viewing parameters. The remote user application sends a request for a particular stereoscopic content and sends digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters. In response the content management server selects one of a plurality of possible versions of the stereoscopic content each having a respective stereoscopic configuration. The content management server causes the stereoscopic content to be transmitted in the selected version. Abstract not to be interpreted as limiting.

Description

RECONFIGURATION OF STEREOSCOPIC CONTENT AND DISTRIBUTION FOR STEREOSCOPIC CONTENT IN A CONFIGURATION SUITED FOR A REMOTE
VIEWING ENVIRONMENT
Technical Field
This invention relates generally to the field of video content distribution and more particularly to the field of stereoscopic content distribution.
Background
The field of video content distribution is a rapidly-expandmg one. Traditional physical media such as Blu-ray Discs™ are giving way to electronic forms of distribution. Services like Netflix™, iTunes™ or YouTube™ allow users to rent or buy movies directly online without having to go to the store and purchase a physical medium. Instead, these services allow users to download (either on-the-fly by streaming or by downloading a video file(s) ) video content in programs such as movies or TV shows. In the case of a rental, access to the program is typically limited by an amount of time or a number of viewings. For example, a streaming may only provide access to the rented content during the time of the rental, or a downloadable file may only be playable during that time.
3D content present unique challenges in a content distribution setting. In particular, 3D content is such that a user is presented with different images in the left and right eye so as to allow the user to perceive the content in three dimensions. Such content may be called stereoscopic.
[001] However stereoscopic content is generally defined by a configuration that is adapted to a particular viewing environment. In the field of video content distribution where the content may be viewed by different customers in different viewing environment, it has not been possible to provide high-quality 3D due to mismatch between ideal and real viewing environments. Summary
[002] In accordance with a broad embodiment, there is provided a method for managing access to viewable stereoscopic content in a digital content library by a remote user application for viewing at a remote viewing environment that is characterised by a set of user viewing parameters. The method comprises the steps of determining the presence in the digital content library of a first version of a stereoscopic content in a first stereoscopic configuration, the first stereoscopic configuration corresponding to a first set of viewing parameters. The method further comprises the step of determining the presence in the digital content library of a second version of the stereoscopic content in a second stereoscopic configuration, the second stereoscopic configuration corresponding to a second set of viewing parameters. The method further comprises the step of receiving digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters. The method further comprises the step of receiving from the remote user application a request for the stereoscopic content. The method further comprises the step of selecting on the basis of the digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application. The method further comprises the step of provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
[003] In accordance with another broad embodiment, there is provided a method for accessing viewable stereoscopic content from a digital content library by a user application for a viewing device being part of a viewing environment, the viewing environment characterised by a set of user viewing parameters. The method comprises the step of transmitting to a content management server a request for a particular stereoscopic content. The method further comprises the step of transmitting to the content management server digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter, the digital viewing parameter data being used for identifying a particular stereoscopic configuration corresponding to the viewing environment. The method further comprises the step of receiving the stereoscopic content in the particular stereoscopic configuration. The method further comprises the step of causing the stereoscopic content in the particular stereoscopic configuration to be displayed on a display associated with the viewing environment.
[004] In accordance with another broad embodiment, there is provided a method for providing stereoscopic video-on-demand content to a remote user operating a remote user application. The method comprises the step of at a content management server, providing the remote user a list of stereoscopic programs available in a digital content library for display on a viewing device to the remote user. The method further comprises the step of for a selected stereoscopic program in the list of stereoscopic programs, providing a regular version and a child-safe version, the regular version being an original configuration of the stereoscopic program and the child-safe version being a reconfigured version of the program reconfigured to adapt the program to a child interocular distance. The method further comprises the step of selecting on the basis of digital viewing parameter data received from the remote user application one of the regular version and the child safe version of the selected stereoscopic. The method further comprises the step of causing the selected version of the selected stereoscopic program to be transmitted to the remote user application for display on the viewing device.
[005] In accordance with another broad embodiment, there is provided a method for managing access to viewable stereoscopic content in a digital content library by a remote user application at a remote viewing environment characterised by a set of user viewing parameters. The method comprises the step of receiving a stereoscopic content in a first stereoscopic configuration, the first stereoscopic configuration corresponding to a first set of viewing parameters. The method further comprises the step of receiving from a remote user application a digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter. The method further comprises the step of determining on the basis of the digital viewing parameter data whether the first stereoscopic configuration is suitable for viewing in the remote viewing environment. The method further comprises the step of upon determining that the first stereoscopic configuration is not suitable for viewing in the remote viewing environment, performing a reconfiguration operation to generate a second stereoscopic configuration corresponding to the at least one viewing parameter from the set of user viewing parameter. The method further comprises the step of provoking the transmission of the stereoscopic content in the second stereoscopic configuration to the remote user application.
[006] In accordance with another broad aspect, there is provided a graphical user interface implemented with a viewing device for presenting to a user of the viewing device access to stereoscopic content in a digital content library offering for viewing at the viewing device. The graphical user interface comprises a first pane comprising a plurality of first visual elements, each of the first visual elements being representative of a category of stereoscopic program and for each first visual element, a first input element associated with the visual element, the first input element being operable by the user using an input device to select the category of stereoscopic program associated with the first visual element associated with the first input element. The graphical user interface further comprises a second pane comprising a plurality of second visual elements, each of the second visual elements being representative of a stereoscopic program and for each second visual element, a second input element associated with the visual element, the second input element being operable by the user using the input device to select the stereoscopic program associated with the second visual element associated with the second visual element. The graphical user interface further comprises a third pane comprising a visual element displaying textual information about a particular stereoscopic program and an third input element associated with the particular stereoscopic program, the third input element being operable to select for viewing the particular stereoscopic program. The graphical user interface further comprises a version visual element indicating the availability of a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters, the version visual element further providing for at least one of the plurality of versions information regarding the set of viewing parameters respective to corresponding stereoscopic configuration. The graphical user interface further comprises a version input element operable by the user using the input device to select a version from amongst the plurality of versions of the particular stereoscopic program.
[007] In accordance with another broad aspect, there is provided a content management system for managing access to viewable stereoscopic content in a digital content library by a remote user application at a remote viewing environment characterised by a set of user viewing parameters. The content management entity comprises a stereoscopic content database comprising a set of records of stereoscopic content in a digital content library, the digital stereoscopic content database comprising for at least one of the records of stereoscopic content the identification of a plurality of versions of the stereoscopic content, each of the plurality of versions being in a different stereoscopic configuration, each stereoscopic configuration corresponding to a different set of viewing parameters. The content management entity further comprises a communication interface system for communicating with a remote entity, the communication interface being suitable for receiving digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters. The content management entity further comprises processing logic configured for accessing the records of stereoscopic content in the stereoscopic content database, accessing the digital viewing parameter data received from the remote entity, selecting on the basis of the received digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application; and provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
[008] In accordance with another broad aspect, there is provided a method for permitting access by a remote user application to viewable stereoscopic content in a configuration adapted for a set of viewing parameters characterizing a remote viewing environment. The method comprises establishing communication with a remote user device. The method further comprises transmitting to the remote user device a registration form comprising queries prompting the inputting of registration information by a user at the user device, the queries including at least one query prompting the input of at least one viewing parameter. The method further comprises receiving from the remote user device the registration information, the registration information comprising digital viewing parameter data comprising the at least one viewing parameter. The method further comprises causing the association of the registration information with a unique user account at a content management system for selection by the content management system on the basis of the digital viewing parameter data of a version of stereoscopic from amongst a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters.
[009] In accordance with another broad aspect, there is provided a registration system for permitting access by a remote user application to viewable stereoscopic content in a configuration adapted for a set of viewing parameters characterizing a remote viewing environment. The registration the system comprises a communication interface system for establishing bidirectional communication with a remote user device. The system further comprises processing logic configured to cause the transmission using the communication interface system to the remote user device a registration form comprising queries prompting the inputting of registration information by a user at the user device, the queries including at least one query prompting the input of at least one viewing parameter. The processing logic is further configured to process registration information comprising digital viewing parameter data comprising the at least one viewing parameter received by the communication interface system from the remote user device to cause the association of the registration information with a unique user account at a content management system for selection by the content management system on the basis of the digital viewing parameter data of a version of stereoscopic content from amongst a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters.
[0010] In accordance with another broad aspect, there is provided a content access system for accessing viewable stereoscopic content from a digital content library for viewing in a viewing environment, the viewing environment characterised by a set of user viewing parameters. The system comprises a communication interface system for communicating with a content management system. The system further comprises processing logic configured to cause the transmission using the communication interface system to the content management system of a request for a particular stereoscopic content. The processing logic is further configured to cause the transmission using the communication interface system to the content management system of digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter, the digital viewing parameter data being used for identifying a particular stereoscopic configuration corresponding to the viewing environment. The processing logic is further configured to process a received stereoscopic content received at the communication interface system in the particular stereoscopic configuration in response to the request to cause the received stereoscopic content to be displayed on a display associated with the viewing environment.
[0011] In accordance with another broad aspect, there is provided a system for distributing stereoscopic video-on-demand content. The system comprises a content management server having a digital stereoscopic content database comprising a set of records of stereoscopic content held in a digital content library. The system further comprises a digital content library storing the stereoscopic content. The system further comprises a remote user application in communication with the content management server and the content storage server at a remote viewing environment characterised by a set of user viewing parameters. The remote user application is operative to send a request for a particular stereoscopic content from the content management server and send to the content management server digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters. In response to receiving the request for a particular stereoscopic content and the digital viewing parameter data, the content management server selects on the basis of the digital viewing parameter data one of a plurality of possible versions of the particular stereoscopic content each version having a respective stereoscopic configuration each corresponding to a respective set of viewing parameters. The content management server causes the particular stereoscopic content to be transmitted to the remote user application in the selected version.
Brief Description of the Drawings [0012] The invention will be better understood by way of the following detailed description of embodiments of the invention with reference to the appended drawings, in which:
[0013] Figure 1 shows a solution of view-environment violation problem e.g. for the broadcasting industry with stereoscopic content reformater;
[0014] Figure 2A shows the effect of image re-formatting on an X-Z plane;
[0015] Figure 2B show the effect of the image re-formatting illustrated in Figure 2A but on the Y-Z plane;
[0016] Figure 3 A shows a geometric model depicting on the X-Z plane the display of a point in 3D on two different displays;
[0017] Figure 3B shows a geometric model depicting on the X-Z plane the display of a different point in 3D on the two displays of Figure 3 A;
[0018] Figure 3C shows the geometric model of Figure 3 A on the Y-Z plane;
[0019] Figure 3D shows the geometric model of Figure 3B on the Y-Z plane;
[0020] Figure 4 shows the mam structure of the proposed re-formatting algorithm, according to an exemplary embodiment;
[0021] Figure 5 shows an image tube and its transforming in a view space;
[0022] Figure 6 shows a comfortable zone of the perceived depth in a view space;
[0023] Figure 7 the proposed comfortable zone adaptive depth transforming algorithm, according to an exemplary embodiment;
[0024] Figure 8 is a schematic diagram illustrating virtual stereoscopic content acquisition parameters for reformatting stereoscopic content for presentation on an intended stereoscopic display of a different size;
[0025] Figure 9 is a schematic diagram illustrating projection differences of a scene object onto a second plane;
[0026] Figure 10 illustrates a stereoscopic display displaying an object A in 3D;
[0027] Figure 11 illustrates a large stereoscopic display and a small stereoscopic display displaying a same image comprising an object B to a user at a same distance from the display;
[0028] Figure 12 illustrates three viewers in three different positions relative to a stereoscopic display viewing a same object;
[0029] Figure 13 shows a stereoscopic display showing two left-right vie-pairs of an object
D at two different position ("placements") on the display;
[0030] Figure 14 shows a single viewer viewing an object D on a stereoscopic display;
[0031] Figure 15 is a conceptual illustration of a stereoscopic viewing environment for stereoscopic content;
[0032] Figure 16 is a block diagram illustrating non-limiting embodiment of a stereoscopic content distribution system;
[0033] Figure 17 is a block diagram illustrating another view of a non-limiting embodiment of a stereoscopic content distribution system; [0034] Figure 18 is a conceptual illustration of a stereoscopic content database according to a non-limiting embodiment;
[0035] Figure 19 is a process/data flow according to a non-limiting embodiment;
[0036] Figure 20a is an illustration of one view of a graphical user interface according to a non-limiting embodiment;
[0037] Figure 20b is an illustration of another view of the graphical user interface of Figure 20a; and
[0038] Figure 21 is a block diagram illustrating another non-limiting embodiment of a stereoscopic content distribution system. Detailed Description
[0039] The images viewed in each eye differ by parallax providing the user a perception of depth. The configuration determines which viewpoint of a scene will be seen by each eye. The configuration of stereoscopic content is typically determined at capture by certain capture parameters. If only two images are captured in a stereoscopic scene, as is the case for typical content generated with stereoscopic camera pairs, transported over a two-image stereoscopic format and displayed on a display system providing two views to a user, one for each eye, such as an active shutter display or a polarized passive display, then the configuration of the content is determined by the parameters of the stereoscopic cameras. In more complex systems where multiple views are generated for the purpose of providing two to the user, the configuration of the content which determines which two viewpoints of a scene the eyes of the viewer will see can also be determined by capture parameters. However, it will be understood that the stereoscopic configuration of stereoscopic content can also be affected by factors other than pure capture parameters, for example, the content may be re-configured during post-processing. And of course, not all stereoscopic content is actually "captured" in the sense that they are caught on camera; much content nowadays is generated by computer and rendered in 3D in a particular stereoscopic configuration.
[0040] Thus the stereoscopic configuration of content is responsible for perception of depth by the user since it determines which viewpoints of a scene the user perceives and since the viewpoint perceived creates the parallax effect that translates into depth perception. However the actual depth perception created by a particular stereoscopic configuration depends upon the viewing environment. A same stereoscopic content in one particular stereoscopic configuration will appear differently in two different viewing environments. In particular the depth perceived by a user will be different, such that the content may look proportionally correct in one viewing environment may look stretched or compressed in the Z-direction (depth direction) in another viewing environment.
[0041] Viewing environments may be defined by certain viewing parameters as illustrated in Figure 15. The viewing parameters are any parameters that may change from one viewing environment to another and that may affect the three-dimensional perception of video content. In particular, we may define a viewing environment in terms of viewer distance (VD), the interocular distance of the viewer (IOD), the display resolution and the display dimensions, for example a diagonal size from corner-to-diagonally-opposed-corner, or a height and width. These parameters may typically change significantly in different viewing environments and can have a significant effect on the perceived depth of stereoscopic content.
[0042] Commonly, stereoscopic content is captured for one particular viewing environment, typically the cinema room. In one typical scenario, stereoscopic content is captured as a stereoscopic pair of images. Dual cameras are used for such capture to obtain a left-eye and right-eye view of the captured scene which is to be represented to the eventual viewer in such a way as the viewer's left eye sees the left-eye perspective and right eye sees the right-eye perspective. Capture parameters including camera focal distance, intercamera separation and camera angle of convergence, are selected with particular viewing environment in mind, to create a stereoscopic configuration of the captured content that provides an accurate perception of depth in that particular viewing environment For big productions, the target viewing environment will typically be a big-screen theater presentation with a centrally located viewer of typical IOD.
[0043] When such images are scaled and provided on a different format screen, the 3D effect may be distorted as a result of the disparity not representing the same relative depth as in the original viewing space. Moreover the resulting 3D image may exhibit stereoscopy that is uncomfortable for viewing under the new viewing parameters as it may require overconvergence or divergence of the viewer eyes under the new viewing parameters. This is not limited to merely made-for-theater movies played on the home screen, but is an effect that can occur whenever a stereoscopic video is captured for a given set of viewing parameters and viewed under a second set of viewing parameters. Moreover, 3D video captured without proper care for viewing parameters may also exhibit such problems when viewed on a 3D display.
[0044] In response to that, techniques to reconfigure stereoscopic content are developed. Stereoscopic reconfiguration is the generation of new viewpoints, or modification of existing viewpoints to create a new configuration for stereoscopic content in which the viewer sees a different viewpoint of a scene in at least one eye from the one seen in the original configuration. A form of stereoscopic reconfiguration may be called stereoscopic re-formatting. It should be understood that examples comprising reconfigurating or a reconfigurator may use reformatting or a reformater, for example of the type described herein.
[0045] Ultimately, one goal for stereoscopic reconfiguration is to allow the automatic real- time reconfiguration at the viewing-end of any received stereoscopic content to adapt it to the particular viewing environment in which it is being viewed. However, generating new viewpoints of a scene can be very challenging, particularly when the only existing information on the scene is contained in a stereoscopic pair of viewpoints provided by the original content. When stereoscopic content contains only two images, one for the left eye and one for the right eye, it can be very difficult to accurately re-create new views of the scene that are visually accurate-looking. Simpler reconfiguration schemes attempt to simulate new viewpoints by simple image-shifting or pixel-shifting techniques. One very effective reformatting technique has been invented by the Applicant and that scheme that generates highly accurate reconfigured stereoscopic image pairs from original stereoscopic image pairs. The technique is particularly efficient and requires low resources to generate high-quality images. This will now be described.
[0046] Stereoscopic contents re-formatting is very challenging in practice. Different from the traditional 2D contents, which can be rendered in different view spaces without leading to serious visual impacts, the stereoscopic contents are usually produced for specific view environments, and cannot be simply scaled to be rendered in different view spaces. Serious visual fatigue and visual discomfort may occur when stereoscopic contents are rendered in inappropriate view spaces.
[0047] Herein is described a novel image tube mathematical model, which dynamically describes the relationship between different elements of a view space, including the audiences, the geometric configuration of screen, and the input stereoscopic contents. With the proposed image tube model, the non-linearly changed screen parallax, which are equivalent to pixel-wise disparities when considering the view space information, can be precisely controlled according to the user-desired perceived depth map. This can greatly improve the visual quality of the reformatted stereoscopic contents. The image tube model can be considered an abstract dataset defining in virtual terms spatial relationships in the image, for example the spatial relationship of things in the image, such as the spatial relationships of pixels (particularly left-eye pixels and corresponding right-eye pixels -that is pixels illustrating a same part of the scene in the left-eye image and the right-eye image) in the image. The virtual terms of this spatial relationship may be for example, the eyes of the viewer and position of a perceived point and position of pixels on a screen and the spatial relationship may be defined in terms of lines intersecting eyes and pixels on the screen and/or intersection points of these lines and each other, the eyes and/or the screen, which in virtual terms may be a virtual screen (and virtual eyes) according to intended viewing parameters.
[0048] Based on the proposed image tube model, we provide an image-tube based stereoscopic re-formatting algorithm. First, the disparity map between the two views of a stereo image pair is computed, and the reliability of the obtained disparities are assessed with the occlusion mask between the two view. Unreliable disparities are refined to further improve the quality of later processing. Second, a set of image tubes are computed from the input stereo image pair and the current view space configurations, including view distance, screen size, etc. The image tube set is then transformed to another view space with the provided image tube transforming algorithm. Third, a new image tube set is computed from the transformed image tube set by spatial tube interpolation, thus the non-linearly changed screen parallax can be precisely computed from the new tube set. With the geometric information of the view space configuration, a new disparity map between the re-formatted stereo sub-images can be obtained thus the new stereo image pair can be synthesized. A post-processing containing occlusion processing and distortion control is then applied to the synthesized stereo image pair to obtain high-quality re-formatted stereoscopic contents.
[0049] With a prototype of the proposed stereoscopic re-formatting algorithm, we have proved that the proposed strategy effectively re-formats the stereo image pair produced for a specific view space to generate different stereo image pairs that are suitable for different view spaces. Evaluations show that the proposed image tube model can greatly improve the visual comfort in 3D experiences in different view spaces.
[0050] Stereoscopic contents become more and more popular in broadcasting industries nowadays. Although 3D movies have been available in markets for a long time, people mostly enjoy the 3D experience in theaters. Lacking of 3D contents, broadcasting industries did not take the stereoscopic broadcasting issues into account until very recent years. As the 3D programs become more and more popular, 3DTV has begun occupying a considerable market share, and the production of 3D contents has doubled even tripled each year, including movies, TV programs, games, and other 3D contents for mobile devices. However, broadcasting industries still lack of 3D programs. This is because of the particularity of the production and the rendering of 3D contents.
[0051] To provide audiences with the high-quality 3D scenes, the shooting conditions of a 3D scene are often strictly selected according to the view environment under which the 3D scene will be rendered. This is relatively easy for 3D movie production, since the environments of different theaters are similar, or at least, do not vary greatly. Therefore, a 3D movie shot according to an optimized estimation of the viewing parameters typical in a theater environments can be shown in most theaters, and provides audiences with satisfying 3D experience. But for the 3D contents made for broadcasting industries, this 3D production strategy will not work. In the receiver-end of a broadcasting system, the viewing parameters (or "view space", which may include the size of the display and distance of a viewer from the display; they may also include horizontal angle of a viewer relative to the display, height of the display relative to the viewer, and any other measure of position of the viewer and the display relative one another; they may also include more subtle consideration such as viewer eye spacing and viewer eye conditions such as far/near sightedness prescription) may be quite different. They may be a home theater, or a bed-room with a small 3DTV, a mobile 3D display device such as mobile DVD player, or even a head mounted display with separate displays (or portions of a display) for each eye, etc. The 3D contents made for a specific view environment may be rendered in a quite different view environment under different viewing parameters. Herein, we define this as view-environment violation. When view-environment violation occurs, audiences may suffer serious visual discomfort and visual fatigue, e.g., headache, dizziness, eye-strain, etc. Much research has been done to analyze the causative factors of visual discomfort and visual fatigue, and recent research shows that view-environment violation is an important factor that degrades the 3D experiences. To build a robust 3D broadcasting system, the view-environment violation problem must be solved.
[0052] Depth-image based render techniques promises to make the 3D contents compatible independently to view-environments, but the techniques are not currently available in multimedia markets due to 1) the technical obstacles in depth acquisition, and 2) the technical obstacles of view synthesis. In practice, most 3D contents are made for a specific view environment. A reasonable solution of the view-environment violation problem is to convert the 3D contents made for a view environment to a new 3D contents which are suitable for being rendered in a different view environment. As an example, Figure 1 shows this solution for broadcasting a 3D contents produced for a specific view environment. Note that in the receiver ends, view environments are different.
[0053] We put forward a novel approach at reformatting stereoscopic images to adapt them for comfortable viewing under viewing parameters when the images where not captured for those particular viewing parameters. We moreover provide a solution for modifying stereoscopic images in such a way as to adapt the depth effect according to viewer comfort, and this same adaptation may be used for other purposes such as displacing the images in the z-axis, scaling in the z-direction and scaling in 3D, for example. In general we put forth a powerful tool for modifying 3D images.
[0054] As can be seen in Figure 1, the 3D contents of real -world scenes are firstly obtained by 3D production 1105, which usually consists of stereo imaging systems and relevant post- processing. Note that the configurations of the stereo imaging system, including both the inner parameters such as the focal length of cameras and the external parameters such as the base-line between the cameras and angle of convergence, are carefully selected according to the desired view-environment. Then, the 3D contents are distributed via diverse channels, for example, by air, or by DVD, etc... using, in this example, existing distribution technology. In the receiver ends, a stereoscopic content reformater 1110 may be integrated into each rendering device. The 3D rendering device can be configured according to the actual view environments, thus the received 3D contents can be re-formatted to the new contents which are suitable for rendering in the current view environment. With the stereoscopic reformater, the received 3D contents can be well rendered in different view environments, and bring high-quality 3D experiences to different audiences.
[0055] It is very challenging to re-format stereoscopic contents companng to traditional 2D contents. The perceived visual quality of 2D contents are mostly related to image quality For example, the noise level, the sharpness, and the contrast. Usually, a high-quality image scaler, can obtain satisfying re-formatted 2D contents for divers rendering devices. To 3D contents, the perceived visual quality is not only related to the image quality of each sub-image in a stereo image pair, but also related to the stereoscopic distortion, and more important, related to the perceived depth. First, the image quality of each view of a stereo image pair affects the 3D visual quality. In general, the quality of a 2D image may be degraded by following artifacts: 1) noise, 2) block artifacts caused by codecs, and 3) unnatural blur. Noise existing in the two views can degrade the quality of 3D contents, however, the human vision system (HVS) has higher tolerance to noise in 3D contents than in the 2D case. Therefore, such noise affects the 3D visual quality less. Block artifacts caused by codecs may degrade the 3D visual quality, and eventually lead to visual discomfort such as eye-strain. The most important causative factors of visual discomfort in 3D experience is the unnatural blur. Image blur may greatly affect the HVS in stimulating accommodation and convergence thus lead to visual discomfort in some strong motion image regions. It has been argued that the unnatural blur is one of the most important factors to contribute the discomfort. However, this has been based on subjective evaluation, rather than theoretical analysis. Our research analyzed the relationship between unnatural image blur and the perceived depth, and we have found that unnatural blur can lead to serious perceived depth dislocation problem due to the discrete perceived depth plans. Therefore, it is an object of the proposed 3D re-formater is that it should keep the image quality of each view of the obtained stereo image pair as high as possible, especially, avoid blurry artifacts.
[0056] Depth is a unique feature of 3D contents comparing to 2D contents, and it is also a very important causative factors of visual discomfort in 3D experiences. We can define a depth comfort zone in terms of spatial characteristics, in terms of relative spatial characteristics (e.g. relative to the viewer and the screen) or in terms of viewer-perspective with consideration for the limit of perceptual depth and/or the depth of focus. In one particular example, we consider that within a view zone - 0-2 diopter, a viewer can obtain comfortable 3D experience and set the comfort zone so. According to this we can define the application rules of the comfortable zone of the perceptual depth, which suggest that the perceived depth should be always within the comfortable zone. Accordingly, it the proposed re-formator may ensure that all re-generated depth should be within the comfortable zone of the current view-environment.
[0057] Another serious problem that the proposed re-formator may meet is the geometric distortion, which may decrease the joy of 3D experience. Generally, geometric distortion in 3D contents are caused by stereoscopic imaging systems. Defects in imaging systems, including both camera configuration and calibration, may lead to six kinds of widely met VDF (Visual Discomfort and/or Fatigue) factors. They are 1) depth play curvature, 2) depth non-linearity, 3) shear distortion, 4) depth and size magnification, 5) keystone distortion, and 6) lens distortion. It has been shown that the most serious healthy problems in 3D experience, such as nausea and headache, are caused by these stereoscopic-imagmg related VDF factors. Since a 3D re-formator recreates a stereo image pair according to the current view environment, some of its behaviors are similar to stereo imaging. So some geometrical distortion caused by stereo imaging may also occur in re-formatting processing. The proposed 3D re-formator minimizes the geometrical distortion.
[0058] Another advantage of the proposed re-formator is that it is robust to occlusion and recovering. Since the depth may be greatly changed after re-formatting processing, image occlusion and recovering regions may also changed. This is similar to the case that a viewer will see different occlusion parts of two objects in different locations when the viewer moves toward or backward to the objects in real -world. Unnatural occlusion will decrease the depth perception and even lead to visual discomfort.
[0059] In addition to the requirements mentioned above, the proposed 3D re-formator is efficient and low-cost for the convenience of hardware implementation. Above all, we summarize the technical advantages of the proposed 3D reformater as follows:
[0060] 1. Avoid to visually degrade the image quality of each sub-image of a stereo image pair.
[0061] 2. Guarantee the perceived depth of the re-generated stereo image pairs are all within the comfortable zone of the current view-environment.
[0062] 3. Minimize the geometrical distortions caused by depth transmission.
[0063] 4. Be robust to the occlusion and recovering problems caused by depth transform.
[0064] 5. Be efficient and low-cost for hardware implementation.
[0065] Herein we put forward a novel stereoscopic algorithm based on image tube transforming. We mathematically model the view-environment violation problem, and according to the model, propose the mathematic model of the proposed stereoscopic re-formatting strategy. We present an overview of the design of the proposed algorithm. We define the conception of the proposed image tube model. The details of the proposed algorithm are then provided. To test the proposed re-formatting strategy and the proposed image tube model described we have implemented the proposed algorithm. With the implementation, we prove that the proposed mathematic model is applicable in real-world applications. The details of evaluations of the implementation are also provided herein.
[0066] One issue in stereoscopic re-formatting is that how to render the 3D contents produced for a specific view space in a different view space without bring serious discomfort and perceived depth distortions to audiences. To the 2D image contents, this is relatively easy. A high-performance image scaler can achieve satisfied results. However, it is very challenging to the 3D contents. We will geometrically model the re-formatting problem here. The geometric models discussed here are the base of the proposed image tube conception, and the proposed reformatting algorithm.
[0067] A traditional 2D image represents a real-world scene by pixels. Each pixel is a sample of a 3D real-world point which is projected to the imaging plane of a digital camera by an imaging system. Generally, a real-world point can be well described by the corresponding pixel in an image with three elements, namely, the coordinate of X-axis, denoted x , the coordinate of Y-axis, denoted ^ , and the intensities of the pixel, denoted i , which is usually a vector consists of the illuminance intensity and the chromatic intensities. Let P and P are the pixel of an image and the real-world point, respectively, we have
P( , 7, Z) → p(x, y, i),
(1) where X , Y , and Z are the coordinates of P in real-world coordinate system. [0068] In a 3D image pair, a real-world point P is represented by two corresponding pixels, one belongs to the left view, denoted P , and the other belongs to the right view, denoted P« . For a camera parallelly arranged stereoscopic imaging system, which is widely used in 3D productions, the difference between the locations of L and P« can be considered that only exists along the X-axis. Thus, the projection of a real-world point to a pixel pair of a stereo image pair can be described as
Figure imgf000015_0001
where ^ is the disparity between Pi and P« in pixel-wise, for the convenience of discussion, we call &x the disparity; L and 1r are the intensities of and P« , respectively. Theoretically, should be equal to 1r . However, in practice, they are usually not equal to each other because of the difference between the characteristics of the cameras and the calibrations.
[0069] The disparity ^ in (2) is the most important factor for depth perception in 3D experiences. It generates the screen parallax, denoted P , according to the parameters of a given view space, and P lead to the perception of the depth information via human vision system
(HVS). Let ^ and are the pixel-wise width and the hight of each view of a 3D image pair,
W H
and the 3D image pair is projected to a screen whose width and the height are s and s , respectively. The pixel amplification factors in the X and the Y direction can be easily obtained as ax = Ws/Wr
(3)
o j = HglHj . [0070] Therefore, the screen parallax is
P = av x Ax.
(4)
[0071] Figures 2 A and 2B sho When view a 2D image, each perceived point is located on the convergence plane, namely, the screen plane, therefore, no depth information can be perceived.
To a given 2D image, which is original suitable for rendering in a view space , when being rendered in a new view space, denoted ^ , re-formatting processing need not consider the depth issue since all the perceived points are on the convergence plane, as shown in Figure 2, where P and P' are two perceived spatial points in and ^v , respectively; ^v and v are the view distance of and , respectively.
[0072] As can be seen in Figure 2, the essential of 2D re-formatting processing is to determine a mapping rule which maps a perceived spatial point P on the screen 1 to a corresponding spatial point P' at the new location on screen 2. Since both of P and P' are located on the convergence planes, the perceived depth of P and P' are v and v , respectively. Without knowing ^v of and the desired ^v of a high-performance image scaler can complete the 2D re-formatting processing very well.
[0073] To the stereoscopic contents, it is much more complex to ref-format a stereo image pair than a 2D image. Not all the perceived spatial points are located on the convergence plane (only the points whose pixel disparity ^ is equal to zero that locate on the convergence plane). As can be seen in (2) and (4), the depth of a spatial point is related to both the pixel-wise disparity and the screen parallax. The pixel-wise disparity is related to the generation of the stereo image pair, namely, the parameters of the stereoscopic shooting system, including the baseline between the two cameras, the arrangement of the two cameras, the distance between the cameras and the objects of interest, the focal length of the camera, and the camera resolution, etc. Yet the screen parallax is related to both of the parameters of shooting systems and the parameters of the view space where the stereo image pair is rendered, as shown in (3). Further more, in addition to the screen parallax, the perceived depth of a spatial point is also greatly related to many view space parameters, including the view distance, and the human eye separation. We will discuss this in detail later.
[0074] As we have discussed above, since the perceived depth of each spatial point is related to both the shooting space and the view space, to correctly render stereoscopic contents in a given view space, the parameters of the shooting space must be carefully designed according to the parameters of the view space. Otherwise, the stereoscopic contents may lead to serious visual fatigue and visual discomfort in long-time 3D experiences. As a basic requirement of high- quality 3D content production, this issue has been researched and multiple stereoscopic shooting rules have been proposed in the past. For the briefness, we will not discuss this issue in detail in this proposal. When rendering a 3D image pair shot for the view space in a new view space
^v , the perceived depth may change seriously due to the changes of the view-space parameters.
How to render the stereo image pair in ^v without bringing visual discomfort is a major technical obstacle in stereoscopic content re-formatting. Note that in practice, for the given stereoscopic contents, the parameters of the their shooting space are usually unknown.
[0075] A way to render the stereoscopic contents shooting for in ^v is to 1) obtain the disparity map between ^ and ^R of each stereo image pair, and 2) apply some adaptations to the disparity map to avoid the perceived depth resulting in visual discomfort. However, this disparity-adaptation based strategy is not reliable for generating appropriate depth. As we will discussed later, the relationship between the disparity (or screen parallax) and the perceived depth is not linear. It depends on the parameters of the current view space, even human eye separation may significantly affect the perceived depth. Especially, due to the lack of object- oriented image content analysis, disparity-adaptation based algorithms linearly adjust the disparity maps. However, the linearly changed disparities will lead to non-linearly changed depth map. Serious perceived depth distortions may occur when rendering the adapted stereoscopic
. Ω
contents in v .
[0076] To overcome the problem we mentioned above, we propose a novel stereoscopic content re-formatting strategy in this report. The proposed strategy does not need to consider adaptations of disparity (or screen parallax) maps, but considers to directly adapt the perceived depth. Figure 3 shows the geometric models of the proposed strategy.
[0077] Figure 3 models the perception of a spatial point in different view spaces, note that in this report, the spatial point perception includes the perceptions of both the depth information
W x H along the Z-axis, and the plane position information along the X- and the Y- axises. Let s s , ^v , and ^ are the basic parameters screen size, view distance, and the human eye separation of
^0 , the proposed re-formatting strategy may involve finding a spatial point transition rule to move a spatial point , which can be comfortably perceived in ^° , to an appropriate position
P' in a new view space whose screen size and view distance are and ^v , respectively, without breaking the content continuities. Here, we assume that the eye-separation of an audience is always ^ in and . To the convenience of discussion later on, we model both of the negative convergence case and the positive convergence case in Figure 3 (a) and (b), respectively. Note that to conveniently describe the 3D real-world depth perception, we project the depth perception of a real-world point P into two planes, namely the X-Z plane, and the Y-Z plane, as shown in the top and the bottom row of Figure 3.
[0078] As an example, we model the proposed re-formatting strategy with the case of the positive convergence, as shown in Figure 3 (b). Let A and B be the locations of the left and the right eyes of an audience, respectively, ?L and R are two corresponding points in ^ and ^R projected from a spatial point P , respectively, and the screen parallax between L and PR is P , we can conveniently compute the perceived depth of the point P with following steps.
[0079] First, we discuss the positive convergence case in the X-Z plane (negative convergence case can be easily discussed in the similar way). As shown in the top row of Figure 3 (b), the homogeneous coordinates of A , B , ?L , and R are
b
A : (--,0,l)r
2
Figure imgf000017_0001
pR : (x + P,Dv,l)T , where x is the X -coordinate of point PL , and ^ is the transpose (see Figure 3 (b)).
[0080] From Figure 3 (b), we can see that P is the intersection of line ^ L and line ^PR The homogeneous equations of the two lines can be conveniently obtained by cross production A X PL and B PR , respectively. We then have
[0081]
Figure imgf000018_0001
The homogeneous coordinates of is then computed by s i e
P : ApL x BpR = {-Dv, x + ^,--b Dvf x (-Dv, x + P --b DvY
2 2 (7) b 2* + P
2 b - P ' b -P v' ' ' and Dv is the perceived depth of P . As can be seen, the perceived depth is depended on b - P
three factors: 1) the screen parallax ^ , 2) the view space ^v , and eye-separation It has no relationship with the point location in the image plane, i.e., the X - Y plane.
[0082] In the Y-Z plane, as shown in the bottom line of Figure 3 (b), the human eyes are located at the original point, and based on the parallel camera arrangement assumption, the coordinates of PL and PR in the Y-Z plane are the same, namely,
Figure imgf000018_0002
[0083] From (8) and (7), we can easily obtain the Y coordinate of P as -——y . Thus,
Ω
spatial location of a perceived 3D point in the original view space 0 is determined
Figure imgf000018_0003
[0084] For the negative convergence case, as shown in Figure 3 (a), the computation of the spatial location of a 3D point P in is very similar to the computation shown in (5)— (9). Once the spatial location of P is determined in ^0 , we can apply a set of transform to to determine a reasonable location of in a new view space ^ thus P can be comfortably perceived by the audiences in ^v . In later discussion, we will propose a content robust algorithm to spatially determine the locations of the 3D points in that are corresponding to the 3D points m 0 .
[0085] Below, we propose a novel geometric model which models the 3D re-formatting problem as a spatial point re-location problem. Here, we propose a novel stereoscopic-content reformatting algorithm based on the model described above. Figure 4 shows the main structure of the proposed re-formatting algorithm, according to one exemplary embodiment.
[0086] As shown in Figure 4, the proposed stereoscopic content re-formatting algorithm reformats a given stereoscopic video, which is shot for a view space , to a new stereoscopic sequence, which can be rendered in a new view space for comfortable 3D experiences, with four steps. First, the disparity map between the left and the right views of a stereo image pair is computed. To improve the reliability of the disparity map, occlusion detection and disparity quality assessment method are used in this step. Second, instead of directly adopt the obtained disparity map to re-format the 3D contents, we describe the relationships between 1) the screen pixels and their spatial depth, 2) the audiences' eyes and the perceived depth, and 3) the perceived depth and the changes of view space parameters, with a novel spatial item, namely, the image tubes. With the proposed image tube based representation, both of the static relationships between different view space parameters, such as human eye separations, view distance, etc., and the dynamic relationships between the changes of view space parameters, e.g., the change of view space, the change of screen size, etc., can be well represented by a single item. We will discuss the conception of the image tube in detail below. The third, an efficient linear transforming is applied to the image tubes obtained in the second step. With the image-tube transforming, the perceived depth of each stereo image pixel pair that can provide audiences with comfortable 3D experiences in ^v are firstly computed, and then a new disparity map which will achieve the new perceived depth map of ^v is reconstructed. With the new disparity map, the two views of the image pair in ^v can be easily synthesized to form a new stereo image pair. We describe the proposed image-tube transforming method in detail. At last, the obtained new image pair is sent to the post-processing to further improve its visual quality, including occlusion and recovering processing, distortion rectification, etc.
[0087] As we have mentioned previously, the perceived depth comes from screen parallax, and it plays a pivotal rule in stereoscopic re-formatting tasks. A way to express depth information is to directly compute the perceived depth from screen parallax, as shown in (10), d = -^—Dv, (10)
p i) _p where ^p is the perceived depth; ^ is the eye separation; P is the screen parallax; and ^ is the view distance, namely, the distance between the eye plane and the screen plane in a view space. The expression in (10) is convenient and direct to present depth information, however, it only can express the depth information. This is far away enough for re-formatting tasks. First, (10) only expresses the static depth information, yet in most re-formatting tasks, the perceived depth in different view spaces are not the same, i.e., the perceived depth is a dynamic factor in different view spaces. Second, (10) does not express the relationship between the perceived spatial points and different view spaces. A sample example is that (10) cannot express how to adapt ^ to obtain the similar ^p in a new view space. The third, (10) is not convenient to reconstruct a specific new depth in a new view space, since the relationship between P and ^p is non-linear.
[0088] Here, we propose a new conception specific to the re-formatting task, namely, the image tube, denoted where ^ is the index of a pair of corresponding pixels. An image tube can be considered as a passage that light travels between different points. It represents both the static parameters of a view space, e.g., the view distance, and the dynamic relationship between different parameters of the view space, e.g., how the perceived depth changes if the screen location changes. Thus, a stereoscopic image, including both its intensity information and its 3D spatial representations, can be conveniently expressed by a bench of image tubes. With the image tubes, potential changes of the perceptions of stereoscopic contents can be conveniently obtained from image tube transforming.
[0089] In this report, an image tube is defined as comprising a line family, which comprises three lines, namely, 1) the projective line between the left eye and the spatial point, 2) the projective line between the right eye and the spatial point, and 3) the line presenting the rendering plane. Namely,
Figure imgf000020_0001
me mathematical model shown in
Figure 3, and with the homogeneous coordinate expression, we express an image tube in the X-Z plane as
Figure imgf000020_0002
X X "7
where ' , r , and are the coordinates of two spatial points in X-Z plane (the two points have
X X "7
the same Z coordinate), note that 1 , r , and ^ are variables in image tube transforming; and ^ is the eye separation, which is regarded as a constant for the sake of simplicity but for which, it will be appreciated, we can account for variability as well. Figure 5 shows an example of the image tube of a point P in a view space. Note that in Figure 5, the solid lines represent the real image tube , and the dash lines represent a virtual tube ^ , which corresponds to after image tube transforming. From (11) and Figure 5, we can see that with the proposed image tube conception, the perceptions of stereoscopic contents in a given view space and any dynamic properties of 3D content perception can be easily presented.
[0090] First, we discuss the image-tube based expression of the 3D content perception in a given view space. Let ^° is a given view space with the view distance ^v , screen size ^ ^s . To a given stereo image pair, assume we have obtained the disparity map. Namely, to the ^ -th pixel pair ' and , which are the left and the right pixels in the two views, respectively, as shown in Figure 5, we know the screen parallax (see above). From (11), we can easily obtain the image tube of the pixel, i.e.,
Figure imgf000021_0001
where x is the X-coordinate of the ' in the screen. The perceived spatial point P determined by & and Pr can be conveniently expressed as
U x L' ,
(13) where x is the cross production. Eq.(13) also dynamically expresses the perceived point when or ^ is changed.
[0091] Second, we discuss the image-tube based expression of a given 3D spatial points family in different view space. In this case, we can determine the spatial screen locations of each pixel pair for a stereo image pair from a known spatial point P . Let P is a 3D spatial point with the location (^' ^) in the X-Z plane, from (11), we have x, = x, = x since P is a single
D„
point. For view space with the view distance v , we have the image tube as
Figure imgf000021_0002
[0092] As can be seen from (14), we can conveniently determine the screen location of each pixel of the pixel pair P' and Pr , which determines P , i.e.,
Figure imgf000022_0001
note that x is cross production. As can be seen, from (15), the disparity between ?l and r can be very easily computed.
[0093] The third, we discuss the image-tube transforming within the same view space, and between the different view spaces. With the proposed image tube, we can conveniently transform an image tube to another image tube either in the same view space, or between different view space. As shown in Figure 5, with (15), we can directly obtain a new corresponding pixel pair when the spatial point P moves to a new location P' in the same view space. Also, to a given spatial point P , the corresponding pixel pairs in different view spaces can be also easily obtained. In Figure 5, we present an example of the same spatial point P and its corresponding pixel pairs in three view spaces, given by screen A, B, and C. As can be seen, using (15), we can directly get the disparities of each pixel pair, see the disparities P ^ , P s , and P c in Figure 5.
[0094] With the proposed image tube, we can conveniently cope with the more complex 3D content perception problem. For example, both the location of spatial point P and the view space are changed. In such a complex case, we just to use (12) to initialize an image tube family in one view space. Then, with the desired rule, determine the desired location of all the spatial points in the new view space, according to the initial tube. Finally, with (14), we can directly obtain the new stereo image pair. We will discuss this below.
[0095] Herein we will describe the details of the system design shown in Figure 4. We briefly introduce some interest disparity estimation and refining strategies for the proposed reformatting algorithm. We present the details of the computation of an image tube from a stereo image pair, and the transforming of an image tube between different view spaces. The techniques of disparity map re-construction for new view spaces are given. Finally, we describe some postprocessing methods which are used to obtaining high-quality re-formatted stereoscopic contents.
[0096] Disparity is an important factor for perceiving depth in a view space. In the proposed re-formatting algorithm, disparity estimation is used for generating high-quality re-formatted stereoscopic contents. As we have described in Figure 4, the precisely estimated depth will generate the high-quality image tube, and eventually, after image-tube transforming and depth re-construction, we can get high-quality re-formatted contents.
[0097] Disparity estimation algorithms have been proposed. In our research, we have systematically investigated the state-of-the-art disparity estimation algorithms. We have concluded that most existing disparity estimation algorithms can be categorized into two classes, one is local stereoscopic optimization (LSO) based method and the other is the global stereoscopic optimization based method (GSO). LSO algorithms are relatively efficient and hardware friendly, but they may be less robust and may provide disparity maps of varying quality. The GSO based methods are more robust to video contents than the LSO, and may outperform the LSO (according to the performance rank listed in the Middlebury benchmark) however, the methods are usually iteration based may cause difficulties to implement by hardware.
[0098] The proposed re-formatting algorithm employs an efficient, hardware friendly, and robust disparity estimation method but any suitable disparity estimator may be used.
[0099] The quality of the estimated disparity maps may affect the quality of the re-formatted stereoscopic contents as errors in the disparity map may be propagated to the later processing, and finally result in visual impacts in the final outputs.
[00100] Occlusion parts in a stereo image pair are the image regions which are visible in one view but invisible in the other view. Therefore, the disparities of the corresponding pixel pairs existing in occlusion regions cannot be directly determined. Without occlusion-adaptive algorithms, the estimated disparities of the occlusion regions are determined by video contents and searching ranges instead of local-optimum criterion. Mistakenly estimated disparities may lead wrong perceived depth prediction, and eventually lead to visual impacts in the final outputs.
[00101] To improve the quality of the re-formatted stereoscopic contents, we propose an occlusion adaptive re-formatting strategy in this proposal. The main idea of the proposed strategy is to detect the occlusion regions between the two views first, then assess the reliability of the estimated disparities according to the obtained occlusion mask. The disparities belonging to the occlusion regions are regarded as non-reliable disparities. The non-liable disparities are refined by disparity refining processing.
[00102] To a given stereo image pair ^ and ^R , and the pixel-wise disparity map {^-(^)} between them, we can compute different image tubes in different view spaces. Let ^° is the view space for which ^ and ^R are shot, to re-format ^L and ^R thus they can be correctly rendered in a new view space ^v , we need to perform image tube transforming between an
[00103] First, we determine the important parameters of , including the view distance ^v ,
W x H R x C R the effective screen size s " , and the resolution of the input image pairs r v , where v is the number of rows and v is the number of the columns of the input images. Note that we define the screen size as the effective screen size. This is because that in many case, the image
R x C
resolution v v is not exactly matches the screen resolution in digital rendering devices. Black boundary filling, image scaling or shirking may be applied to render the input images in such a digital displayer. We address this issue in pre-processing stage. Many movers are shot for optimal viewing in a standard (or IMAX) theater. Generally, is known in practice however it can be derived from metatdata in a video stream or by user input or by other means. For the case that ^° is unknown, we can solve the problem by two ways. One is we estimated a set of reasonable parameters of according to existing 3D rendering environment. The other is we set the desired view space as ^° , thus we convert the problem of image tube transforming between different view space to the problem of image tube transforming within the same view space. Indeed it is to be appreciated that the solution described herein may be used for reformatting to convert between view spaces but also to transform an image within a same view space, for example to correct for uncomfortable filming, to tailor the 3D experience to individual preferences, to correct errors in a 2D-to-3D converted film, to adapt a film to limitations/increased capabilities of a particular display, or merely to adjust depth for flattening/deepening or otherwise altering an image or portions thereof.
[00104] Second, with (12) and (13), the image tube of Ω° , denoted is computed. Then, image tube transforming is applied to ^ according to the desired view space to obtain the intermediate image tube that can provide audiences with comfortable 3D content perception in Ων , denoted "*~(^) Note that k - 1,2, · · · , Ν ^ w^ei[e N js me number of the pixel pair of the input stereo image pair. In the present embodiment, after image tube transforming, the number of image tubes in both Ω° and ^v are the same, with each tube in Ω° corresponding only one tube in which as the same image pair index. The transforming rules are described as follows, according to the present embodiment.
[00105] The first step of image tube transforming is to perform spatial point translation to move each spatial point in Ω° , where k is the index of the pixel pair of the input stereo image pair, to the corresponding positions in . In avoiding serious distortion, we may adopt 2D linear scaling to complete the job. For the purposed of this example, we assume that an image is fully displayed within the effective screen size and that no cutting or extending (e.g., add black bars around the image boundaries) is applied to the image to be displayed. To a spatial point P
( X Y Z ) determined by "^0 ^ , we can easily obtain its spatial coordinates from (13). Let ^ " , ° ' is the coordinates of P , the input image resolution is W vy 1 j x Hi 1 , and the effective screen size in
W x H W' x H'
is s s , and the effective screen is we can obtain the X and the Y coordinates of in as
W
Y,
H.
(16)
[00106] Note that (16) keeps the image resolution not changed.
[00107] The second step of image tube transforming is depth stretching, which will change the perceived depth in Ω" . As we have mentioned in above, 3D contents are usually produced for specific view spaces. Therefore, the stereoscopic contents made for Ω° may generate serious visual discomfort in if the two view spaces are quite different. How to determine a reasonable perceived depth map for the 3D image pair made for Ω° but will be rendered in addressed by the proposed algorithm. A widely adopted way to solve the problem is to adapt the disparities. However, the perceived depth is not only related to the disparities but also related to view space parameters. When the view space changes, the perceived depth changes non-linearly. Therefore, directly adapt disparities usually generate relatively high depth distortions. In this report, we propose to linearly change the perceived depth in to build a reasonable depth map for that can bring audiences comfortable 3D experiences. In such a way, the perceived depth in ^ will have few distortions due to the linear changes, but the disparities will change non- linearly. However, with the proposed image tube, the non-linearly changed disparities can be conveniently obtained from (15).
[00108] Researches in literatures have proved that eye over-convergence caused by extreme screen parallax is one of the most important factors leading to visual fatigue and eye strain. It has been recommended that all perceived depth should be within a comfortable zone between - 0-2 diopter (D), as shown in Figure 6. However, it should be understood that a "comfortable zone" can be established according to other criteria as well.
[00109] As shown in Figure 6, the comfortable zone - 0·2β determines the limits of the perceived depth of a view space. In Figure 6, the minimum depth for comfortable 3D perception is expressed as the comfortable zone for foreground, denoted f , and the maximum depth for comfortable 3D perception is expressed as the comfortable zone for background, denoted * . Based on the comfortable zone analysis, we propose a linear transforming between the depth of and the depth of ^v to guarantee the perceived depth in denoted ^v^ are all within
[Zv Zvl Ω the comfortable zone f ' . Thus the visual comfort of 3D experiences in v can be maintained. Figure 7 shows the proposed comfortable zone adaptive depth transforming algorithm.
Ω \ZV Zvl
[00110] As can be seen in Figure 7, the comfortable zone of denoted ' , is firstly computed according to the parameters of ^v . At the same time, the minimax of the perceived depth of ^ is located, denoted ^f and ^b , representing the minimum depth (the foreground) and the maximum depth (the background) of "^0 ^ , respectively. Then, we check if the original
\ZV Z"l d° < Zv d° > Zv minimax depth are all within f ' h . This is done by checking if or if * 6 . If so, it means that the original depth map has the depth that exceeds the comfortable zone of , and depth transforming is required. Otherwise, namely, all the original depth, i.e., ^^^ , are
Ω d° < ZV within the comfortable zone of no depth transforming is required. For the case that f f , referencing from Figure 6, we linearly increase the depth of all the foreground points to avoid the d° > Zv
audiences' eyes over converged. For the case that * * , we need to linearly decrease the depth of all the background points to avoid the negative convergence of the audiences' eyes. For some of the spatial points belonging to the object of interest, to which the two shooting cameras are converged, the pixel-wise disparities are zero, thus the screen parallax of the points are zero, too. Therefore, the new depth of ^ in ^v can be afdo(k) foreground
zero - disparity
ahdB (k) background,
(17) where / and b are the depth scaling factors for the foreground and the background points, and
Figure imgf000026_0001
af ≥l.O h≤l .O , D'■ ■ . - Ω and from Figure 6, we can see that f , and 6 ; and v is the view distance of ^
[00111] The third step of the image tube transforming algorithm is to new tube computation in which obtain a new image tube according to the parameters of ^v and the transformed depth {dv (k)} 06†^ame mcjex j m may ¾e different from k , which is the tube index in i io , since the proposed algorithm also takes the case that the image resolution is also changed into account. When the image resolution is changed, the number of image tubes in different view spaces are different. In this report, we compute with image tube interpolation. Let the intermediate image tube computed from (16) and (18) be ^(^)} me proposed image tube interpolation strategy is to obtain {TvO)} from {T(£)} t th t th number of the tubes in is equ aall ttoo tthhee number of the tubes in 5 and not equal in this case to the number of the tubes in { vO")} due to the image resolution changes.
[00112] From Figure 5, we can see that the image tube in v is determined by the pixel
Ω D'
locations in the screen of , and the view distance v . Note that we consider that the eye separation is a constant in this example; however variable eye separation could be accounted for as well. Therefore, from the screen size in and the desired image resolution, we can obtain the screen location of each tube "^( ) Let A and ^ j are the X- and the Y- screen coordinates of a pixel of the left view, and from the obtained , we can determine the screen pixel location of each intermediate tube with (15), denoted Xk and ^k . Thus, for each screen pixel we can determine a neighborhood of p{k) which is centered by Pv U) As shown in Figure 5, this is equivalent to determine a neighborhood of tube J(k) , denoted Nvt/) centered by ^ U) ^ m^ dv(j) js unknown. Let ; is the local index of which are centered by (fi , we have
NW(7) = {T( )}, (19) where * _ 1=2,..., ^ M -g ^e numDer 0f me neighboring tubes of "^^ . To the convenience of computation and considering the projective relationship between spatial point P and its projected pixel location in the screen to the left eye, we can set = 4 . Thus, to a tube Tv ( j) ^ -ts dv (j) can ke 0^tajnecj applying a bilinear interpolation algorithm to ^v ^ , namely dy(j) = M{dM), (20) where is the bilinear interpolation function, and z is the local index of the image tubes centered by "^J' . Once ^vU) is obtained, image tube can be easily determined by the projective line between the left eye and the screen location of the pixel in the left view, as shown in Figure 5.
[00113] Above, we transform the image tube set of to the desired view space ^v , and obtained a new set
Figure imgf000027_0001
w|m me depth ¾ ')} 0 a perceiVed spatial point in , this is equivalent to obtain its corresponding spatial point P' in , as shown in Figure 5.
[00114] With this exemplary image tube formulation, the disparity reconstruction is relatively simple. First, the screen location of each pixel of the left view in has been determined by the given parameters of and the desired image resolution, denoted ' Χ' . Second, the screen location of each pixel of the right view in ^v , denoted ^r ^Xr ' ^v ^ , can be directly determined by TvC w|m Since the human eyes are generally located on the X-axis of and PR will have the same Y coordinates. (Note that the prior computation of a depth map is not necessarily required to reconstruct disparity.)
[00115] The screen parallax of ^U) can De directly computed by
Figure imgf000027_0002
[00116] From ^ U) ^ me pixel-wise disparity of "^v^ , namely ^*'^') can be computed from the pixel size of ^ . The right view in ^ thus obtained by
(22)
[00117] A stereo image pair L and R shot for view space ° can be re-formatted as a new stereo image pair L and with the proposed algorithm described previously. However, the direct output of the algorithm is preferably processed to remove visual artifacts. The artifacts may be mainly caused by two factors. One is the occlusion between the left and the right views, and the other is the distortions caused by image tube transforming.
[00118] The occlusion parts between the two views of a stereo image pair are a challenge for real-world applications. Since the pixels of the occlusion parts only exist in one of the two views, the disparities of the pixels cannot be directly estimated from disparity estimation using traditional disparity estimation techniques. However, without the well-estimated disparities, the visual quality of the occlusion parts in the re-formatted image pairs will be lower. Therefore, it is a benefit for the post-processing in the proposed algorithm to properly address occlusion region refining.
[00119] Although many occlusion detection algorithms have been proposed in literatures, they might not be suitable for solving the problem of occlusion in stereoscopic re-formatting applications. This is because that different from the occlusion problem in disparity estimation (see above), where the occlusion regions for sure appear in one view, stereoscopic re-formatting may make the regions which are invisible in both of the views visible in the re-formatted views. This is mainly caused by the relatively big changes in view distance. Therefore, occlusion may lead to gaps in final results, especially, to the case that the view distance changes greatly. In the proposed algorithm, solve the problem by adopting region-analysis based interpolation algorithms.
[00120] Another problem of stereoscopic re-formatting that has been overcome is that of distortions. As we have discussed, as the parameters of the view space are changed, including screen size, view distance, and image resolution, the re-formatted stereoscopic contents will be scaled in all the X, the Y, and the Z directions. Although image scale in the X and the Y directions are similar to a common 2D image scaler, the scaling factors used in the Z direction may be quite different from the factors used in the X and the Y directions, Z-scaling factors are depended on 1) view distance, and 2) pixel properties. First, for the relatively big view distance, the comfortable is also relatively big, Z-scaling factors may be relative small; yet if the view distance is small, and the comfortable zone is also small, Z-scaling factors may be relatively big. Second, the Z-scaling factors are different for the foreground and the background pixels. As we have mentioned above, to maintain the comfortable zone, we need to increase the depth of some foreground points, i.e., adopting the Z-scaling factor that is bigger than 1.0 for the foreground points, and decrease the depth of some background points, i.e., adopting the Z-scaling factor that is smaller than 1.0 for the background points. We have achieved a balance between the different Z-scaling factors to avoid serious distortions in object size and Z-scaling required for comfort.
[00121] Our solution is based on the surprising finding that although different Z-scaling factors may lead to distortions, the problem may not seriously degrade the re-formatted stereoscopic contents. First, to a reasonable view space, audience usually will not watch 3D contents in a very short view distance and with a relatively big screen. Generally, audiences will be closer to the screen when the screen size is small, e.g., some mobile device, and be far away from the screen when the screen size is relatively big, e.g., the home theater system. For example, assume the screen size is relatively big in ^0 , as can be seen, for the small screen in
^v , scaling factors in X-Y plane is relatively big, and since small view distance is usually adopted to the small screen, the Z-scaling factors will also be relatively big; and for the big-size screen in , the X-Y scaling factors are relative small, and since big view distance is usually adopted to big screens, the Z-scaling factors will also be small. It is similar to the case that a small screen is used in ^0. This will greatly maintain the naturalness of the stereoscopic contents. Second, different scaling factors in different direction may result in some unnaturalness. However, it has been shown that in 3D experiences, the tolerance to unnaturalness of the human vision system significantly increased companng to the 2D experiences. Therefore, in most cases where the view space are not greatly changed, the distortions caused by scaling will not result in serious visual impacts. Thus we are able to reformat stereoscopic pairs to introduce z-scaling required for comfort that stays within a threshold of distortion that is acceptable to the human brain.
[00122] The third problem that the post-processing should solve is the sharpness of the reformatted stereoscopic contents. Research has shown that unnatural blur is a causative factors leading to visual fatigue and visual discomfort.
[00123] We keep the sharpness of the input stereoscopic contents as much as possible. However, to some noisy input, noise removal algorithms may be adopted in the pre-processing, and this may lead to blur in image details. For such cases we either adopt image detail preserving noise removal algorithms in the pre-processing, or apply image sharpening processing in the post-processing, to keep the sharpness of the 3D contents.
[00124] Herein, we propose an exemplary content-robust stereoscopic re-formatting algorithm. Based on the proposed image tube model, in this proposal. First, the disparity map and the occlusion mask between the sub-images of a given stereo image pair are computed. The reliability of the estimated disparity map is then assessed with the occlusion mask. Second, the unreliable disparities are refined to further improve the quality of the estimated disparities. We then propose a novel image tube mathematical model for solving the ill-posed stereoscopic content re-formattmg problem. The proposed image-tube dynamically describes the relationship between the audience, the view space, and the perceived depth of a given stereo image pair. Thus the non-linear changes of screen parallax caused by any changes of view space can be precisely controlled according to the user-desired changes of the perceived depth. The image tube of each corresponding pixel pair is computed from the geometric properties of the current view space, including the view distance, screen size, and the natural characteristics of audiences. Thus, a stereo image pair is converted to an image tube set in the current view space. Then, the obtained image tube set is transformed to a new tube set according to the parameters of a new view space. The transformed image tube set may have different number of tubes from the original tube set depending on the configuration of the new view space. The fourth, the non-linearly changed screen parallax of each corresponding pixel pair is computed from each transformed image tube, thus the pixel-wise disparities can also be obtained from the new view space configuration. With the new disparity map, the new stereoscopic image pair can be synthesized. Finally, the quality of the synthesized image pair is refined by a set of post-processing, which contains occlusion processing, distortion refining, and sharpness enhancement if it is necessary.
[00125] The subjective evaluations of the prototype of the proposed re-formatting design show that the proposed image tube model can effectively re-format the stereo image pair containing extreme contents, including extremely huge disparities, multiple objects with different spatial positions, etc. Subjective evaluations show that the proposed model greatly increases the visual comfort of the audiences in 3D experiences in different view spaces.
[00126] As will be readily appreciated, the algorithm proposed can be implemented in software code. Additionally, this algorithm has specifically been designed so as to be readily implementable in a hardware embodiment with minimal use of costly resources. It is designed to be executable quickly enough to be executed in real time in practical implementations.
[00127] As will be appreciated, more than two views can be created using the above model. Indeed, the steps of image tube transformation, disparity and reconstruction of views may be applied to generate additional views, for example for the purpose of a multi-view display. As such the above algorithm may be used in the generation of multi-views from a stereoscopic image pair.
[00128] Moreover, it will further be appreciated that the above algorithm is already suited for adaptation to a view-and-disparity-map format or a view-and-depth-map format or for a 2-views- and-(l or 2)-depth-map. Indeed, using known methods for converting between depth map and disparity map, we may use such formats to replace the disparity estimation step by obtaining directly the disparity map of the image, allowing us to compute image tubes therefrom. Thus the above algorithm can take advantage of such formats to reduce the computational burden while the remainder of the algorithm, as described above, allows the generation of stereoscopic image pairs (or any other number of different views) tailored to desired viewing parameters or with a desired depth effect.
[00129] It is to be appreciated that the image tube transformation may be made to respect several different kinds of constraints. For example, the image tube transformation may set hard limits on the maximum and/or minimum depth of objects in the image Moreover, the image tube transformation may be used to ensure a certain proportionality between the resizing in the X-Y plane and the resizing in the depth direction. As the image undergoes a linear scaling to fit the screen, an appropriate modification of the depth can be selected based on the X-Y scaling undergone. In particular, knowing the image tube in the original viewspace; for any particular X- Y scaling factor and starting depth, the proper scaling effect in the Z-direction can be ascertained using known computational methods or by referring to a lookup table comprising pre-computed values. Thus we can ensure that upon reformatting objects in the image maintain a depth proportional to their size in the X-Y plane.
[00130] Maintaining proportionality and respecting the comfort zone is a matter of balance. As it will be appreciated, a proportionally reformed object may violate the comfort zone in a new viewspace. As such, in an optional embodiment, the algorithm provides proportional image reformatting with a safeguard for view environment violation whereby when the transformed image tubes would images to points going beyond the comfort zone, the image is further scaled in the Z-direction before proceeding to re-construction and view synthesis. In one embodiment, the image tubes indicating points violating the view-environment may simply be transformed to bring the points back within the comfort zone. But this, however, may lead to an awkward flattening of the image about the edge of the comfort zone. Alternatively the whole set of image tubes may be scaled by depth together. This may be done linearly in the Z-direction or, alternatively, the scaling may be non-linear (e.g. logarithmic) such that the image tubes in violation are subject to the greatest change in depth while those that represent points that are closest to the plane of convergence are the least modified. Advantageously, since the object of interest in a video is usually featured at or near the plane of convergence, this object will be the least affected by the scaling, while remaining proportional. It is to be understood that Z-scaling to avoid view environment violation may be done even in cases where Z-scaling for proportionality isn't performed.
[00131] As it will be appreciated, the reformater allows for reformatting according to different criteria, e.g. proportionality, comfort zone adherence, and (as will be seen later) child safety. As discussed above, scaling can be performed to reconcile the different criteria where reformatting according to one criterion would cause a conflict of another criterion, however such scaling may cause distortion. In a non-limiting embodiment, a reformater is implemented in a viewing system allowing for a selection of those criterions (e.g. proportionality, comfort and child safety) that are desired. A user may be provided via a graphical user interface implemented by a processor in the viewing system, a button interface or elsehow a choice of which of the criteria the user wants the reformater to base reformatting on. An additional option can be no reformatting, which may be selected separately or merely by virtue of not selecting a criterion. For example, in an example where the viewing system is a television, the user may access a menu using a remote control, the menu providing a set of visual indicators of a reformatting criteria and a selection indicator, e.g. in the form of a radio button. By navigating visually through the options by selecting with a remote control selection input one of the cirteria, the user may select one criterion and by providing an input, e.g. pressing a "select" button on the remote control, the user may indicate that this is the criterion according to which the reformater is to reformat. In response the processor sets the parameters according to which the reformater is to operate and causes the reformater to operate in the manner described herein according to the constraints.
[00132] In a variant of the above embodiment, the user may select a hierarchical order for the criteria, each causing a different transformation of the image tubes in inverse order, that is, the most important one being performed last and the least important one first. Thus if the user orders the criteria as comfort being least important, proportionality being second most important and child safety being most important, the reformater will first transform the image tubes according to the view space, then transform them to fit the comfort zone, then transform them to maintain proportionality and finally transform them to ensure child safety. In this embodiment, prefereably no scaling is perform to reconcile the different criteria. As such, the last transformation is sure to be applied throughout the image, but the others may have had their effects modified by subsequent transformations.
[00133] This manipulation of image tubes lends itself to other useful inventive features. For example, it has been observed that when watching 3D on a screen, object the human brain tends to have trouble perceiving depth for objects that are adjacent the edge of the screen. Even though an object may pop out towards the viewer the presence of a discontinuity at the edge of the screen, next to which the rest of the world is at a different depth causes the brain to badly perceive the 3D effects. In addition, if an object has high disparity but is near the edge of the screen, some portions of the image of the object will be visible to one eye but not to the other because that other eye's view of that portion of the object would be off the screen This results in a strange 2D perception by the edge of the screen. In short strong 3D effects near the edges of a display tend to be problematic. This is not a big problem in, for example, an IMAX setting because the sheer size of the display means tha the edges are far away from a viewer's main focus (usually the middle of the display). But when video intended for such a large display is viewed on a smaller display, the problem of poor 3D at the edges becomes more pronounced. Generating video content that avoids this problem is difficult and requires an elaborate safety zone around the frame of the capture to ensure high-disparity (e.g. nearby) objects don't touch the frame.
[00134] In one embodiment, the reformatting system is used to correct for such near-frame problems after capture. This may be implemented in real-time in a display system or in a post- production setting. It may allow for more liberal capturing (e.g. filming) and for more satisfying transition from very large screens to more modest ones. In one example, the image tubes are analyzed to identify the presence of points near the edge that would have a large depth in the new viewspace. If such points are found, the set of image tubes may be transformed to shift in the Z-direction to bring the points near the edges closer to the plane of convergence. As in the above embodiment, the scaling may be linear or non-linear. In this case, the scaling may be applied (e.g. non-linearly) across the frame such that points closest to the edges are scaled most and points further from the edge towards the center (where the object of interest likely is) are scaled less. If there are different depths around the edge of the frame, the scaling can vary across frame accordingly.
[00135] It will be appreciated from the foregoing discussion that image tube transformation may be exploited to implement a shift in the Z-direction. In particular reformatting may be done, not only to accommodate a new viewspace Ων, but also to accommodate other modifications of the image as may be, for example, indicated by a user at an input. In a particular embodiment, the reformater is implemented on a viewing system comprising a display and a user input though which a user may input reformatting parameters. Reformatting parameters may include a shift in the Z-direction to implement a so-called reformater Z-shift. In one example, the viewing system is a television and the input is provided by way of a graphical user interface through which the user may provide input using a remote control. More details of how such a graphical user interface may work can be derived from the example provided below relating to child safety mode. It will be appreciated that this may also be implemented elsewhere than on viewing systems such as on post-production equipment. In particular, the user may input an indication of a desire to shift in the Z-direction, for example by first selecting a Z-shift option in a menu and using arrow buttons to indicate a shift inwards or outwards from the screen. At the image tube transformation stage of the reformater, the reformater transforms every image tube to impart the change of depth required. The subsequent image synthesis reconstructs the images and the result is a 3D scene that moves inwards and outwards of the display according to user input. Alternatively, the user may input a Z-shift value using a numerical interface.
[00136] In addition to a Z-shift, the unique flexibility offered by image tube technology enable the development of even more special techniques. Zooming is problematic in 3D for reasons that can be understood in light of the above description. While in 2D zooming merely involves scaling the image in the X and Y direction (in the case of a zoom-in, this may involve abandoning or cropping out parts of the image that would, once scaled, fall outside of the edge of the view area/display, in the case of a zoom out, it may result in an image smaller than the view area/display and thus a black or empty border around it). However, as we know such manipulation causes non-linear shift in the resulting depths in a 3D image pair. Merely resizing a stereoscopic pair of images is therefore not satisfying.
[00137] In a non-limiting embodiment, the reformater serves to implement a 3D zoom. Very advantageously the so-called reformater 3D zoom allows for proportional 3D zooming. In one example where 3D zooming is performed in response to user input, the reformater 3D zoom is implemented on a viewing system comprising a user input. When a user input indicates a zoom is desired in the Z-direction, the reformater follows the same steps as described in the main embodiment with some modifications. In particular, the image tubes are transformed to respect the new viewing parameters (viewspace) but are more over transformed to reflect the zoom in the X-Y plane. Using the proportional Z-direction scaling described above, the image tubes are modified to reflect a proportional change in depth. During image synthesis and reconstruction, the scaling of the image to use with the disparity map is altered by the amount as the scaling in the X-Y plane performed on the image tubes to reflect the transformation resulting from the zoom. As a result, if the zooming action is a zoom-in, some of the original image will fall outside of the frame of the display and consequently not be displayed. The excluded portions of the image can be removed from the originating images during the X-Y scaling in image reconstruction, such that although all the image tubes were transformed according to the zoom, only those that pertain to points that will be visible in the new images are actually used in synthesis according to the above method. Likewise in a zoom-out scenario, some the image may shrink to less than the size of the display such that an empty (black) border surrounds the image. In this example the zooming operation is performed on the totality of the image tubes therefore as the image is zoomed in or out, the whole visual information is available if it can fit in the display. In alternate embodiments, X-Y zooming of the originating images may be used in advance in order to compute those image tubes which will remain and which are worth transforming.
[00138] It is intended that comfort-zone based reformatting as described above may further be applied to ensure that the zoomed image suits the viewing parameters it is viewed under. In such a case, the end-result of zooming in may cause objects to move outwards towards the viewer up to the limit of what may comfortably be seen and then squish together at that depth. IT will be appreciated that this and other interesting effects may be applied by transformation of the image tubes.
[00139] Typically 2D TVs have zoom functionality for adapting to different input image sizes. In one example a 3D TV performs similar zoom functionality but using the reformater 3D zoom when the input feed is 3D.
[00140] It will be appreciated that a reformater 3D zoom may be implemented elsewhere than on a viewing system, for example in postproduction equipment. Likewise it will be appreciated that the reformater 3D zoom may also be operated non in direct response user input. 3D zooming may be used, for example automatically, to correct for detected problems around the edges of a display (described above) or in other non-user-input directed manners.
[00141] The reformater 3D zoom and reformater Z-shift provide powerful 3D manipulation tools with minimal computational burden. Since these tools rely on the lightweight reformater design provided above, they are, like the reformater, implemented in hardware for real-time application in a display system chip. They are, of course also be implemented in software for a variety of applications including software video processors, video editing tools and post- production equipment. The combination of reformater 3D zoom and reformater Z-shift enables a user to blow up and shrink down and move into and out of a 3D image at will. This is combined with X and Y direction translation allows an unprecedented full range of motion to the user.
[00142] X and Y direction translation is performed by translational module that causes stereoscopic image pairs to shift together in the X and Y direction by altering the display timing of their pixel lines according to known methods.
[00143] In a particular embodiment the viewing system is a tablet comprising a 3D viewing display displaying 3D imagery to the user. The user interacts with the viewing system by a user interface, in this case a touch screen interface. By providing specific inputs the user is able to zoom into and out of the 3D image, Z-shift in and out of it, and move it up, down, left and right. In this particular example, the well-known pinch zoom is used to indicate that zoom in or out of the 3D image is requested and the image is zoomed by the reformater 3D zoom accordingly. Dragging a single finger over the touch schreen indicates a desire to cause a shift in the image in the dragging direction and the image is moved accordingly while a two finger drag indicates a request to Z-shift into (dragging in one direction) and out of (in the opposite direction) the 3D image and the image is Z-shifted by the reformater Z-shift accordingly. Alternatively other inputs may be used to indicate a request for Z-shifting such as twirling a single finger in one direction (e.g. clockwise) to shift inwards and twirling it in the other (e.g. counter clockwise) to shift outwards. Known multi-touch technology may be used to implement this input interface.
[00144] In an alternate embodiment, the viewing system is a 3D television and the user interface is implemented using a combination of a remote control and a GUI. The remote control may comprise a 3D zoom in button, a 3D zoom out button, a Z-shift in button, a Z-shift out button, a move left button, a move right button, a move up button, a move down button or any subset thereof. The buttons are used by the user to input a request to zoom/shift/translate the image accordingly. A processor in the viewing system receives the request and causes the reformater 3D zoom, reformater Z-shift or translational module to perform the requested transformation. The processor then processes the resulting images for display and displays them on the display of the viewing system.
[00145] The reformatting technique will now be described from a different perspective. Human three dimensional depth perception employs triangulation provided by binocular vision. Objects within a distance from a viewer exhibit parallax as viewed by each eye. Distant objects exhibit least parallax and objects exhibiting imperceptible parallax are perceived to be infinitely distant.
[00146] As touched upon above, stereoscopic content capture for applications such as remote inspection, remote guidance and entertainment, typically employs a pair of spatially rotated cameras spaced apart to provide an observer with views of a scene from two points of view. Left and right stereoscopic image pairs are acquired and presented to the observer as stereoscopic content in the form of overlapped image fields of view which provide comfortable stereoscopic observation of un-occluded objects within a limited region. The parallax between the corresponding images on the stereoscopic display causes the observers gaze to converge at points behind or in front of the plane of the stereoscopic display where the observer perceives the three dimensional image.
[00147] As discussed, stereoscopic content presentation or playback typically employs a stereoscopic display having a size and resolution. For any particular application, comfortable stereoscopic perception depends on the display characteristics. Due to a limited resolution of the display, objects exhibiting a parallax less than can be represented on the display due to pixel size are perceived as very far, or at infinity. Conversely, objects exhibiting excessive parallax may suffer from extreme disparity and be perceived as double.
[00148] When the observer's eyes are focused on the stereoscopic display, depth perception depends on a variety of viewing parameters. Viewing parameters affect the perception of depth for a given 3D program. These may include the size of the display and distance of a viewer from the display. They may also include horizontal angle of a viewer relative to the display, height of the display relative to the viewer, and any other measure of position of the viewer and the display relative one another. They may also include more subtle consideration such as viewer eye spacing and viewer eye conditions such as far/near sightedness prescription.
[00149] When the observer's eyes are focused on the stereoscopic display, the depth of field of the observer's eyes covers comfortably only a finite distance in front and behind the display. The region of comfortable stereoscopic observation which can be displayed on a stereoscopic display may not cover the entire region over which the left and right stereoscopic pair image overlap. This has been described above. In order for an observer to comfortably fuse the left and right images to perceive the stereoscopic effect, the parallax of objects in the scene must not exceed a maximum parallax. That is to say, the disparity (the distance between equivalent points in the left and right images presented to a viewer) must not exceed a certain maximum in either direction (overconvergence or divergence) for comfort. In terms of disparity, the maximum/minimum amounts depends upon the viewing parameters, as a disparity of X pixels will represent a different distance on different screens and a disparity of X cm will demand a different angle of convergence of the eyes at different distances. Stereoscopic content presentation of a display having a different size than the size for which the content was intended introduces a scaling factor to parallax which in turn caused a change in the perceived depth of objects in the scene. When the apparent location of a three dimensional image is around either the far or the near edge of the depth of field of the eye, the observer may only fuse the stereoscopic image pair with effort straining the eyes. For objects outside the depth of field of the observer's eyes, image fusion is impossible leading to double vision. Undue eye strain is experienced when objects have excessive parallax.
[00150] Figure 10 illustrates a stereoscopic display displaying an object A in 3D. (Note that the projection plane represents here the convergence plane which is the plane occupied by the screen in a typical (theater projection screen, LCD or plasma) screen-based stereoscopic display.) The stereoscopic display shows two images of the object A, one visible only to the left eye and one visible only to the right eye. These two images are not in the same location such that a viewer looking at the stereoscopic display sees the object A in two different places with his left and right eyes. Figure 10 shows the eyes of a viewer (at constant eye spacing c) according to three different positions of the viewer: a mid-range distance (Distance A) from the screen, a far- back distance (Distance B) from the screen, and an up close distance (Distance C) from the screen. From each position, the viewer's left and right eyes see the object A at a different angle. The dotted lines joining the left and right eyes to the left and right perspective images of object A represent the angle of view of the object for each eye. The depth of the object A is perceived at the point where these two lines meet.
[00151] As can be seen, the perceived depth of object A varies with the distance of the user from the display. Object A appears further out from the display as the user positions himself further from the display. Thus, the stereoscopic image being displayed appears stretched or compressed in the z (depth) direction as a viewer positions himself further or closer to the screen. [00152] Figure 11 illustrates a large stereoscopic display and a small stereoscopic display displaying a same image comprising an object B to a user at a same distance from the display. On the smaller display, the left and right perspective views of object B are closer together, resulting in the appearance of object B being closer to the display. Images having been captured for viewing parameters other than those under which they are viewed may appear more flat in the Z (depth) direction. On the larger display, the left and right perspective views of object B are further apart because the overall image is larger, which leads to a perception that object B is further away from the display. Images having been captured for viewing parameters other than those under which they are viewed may appear to have greater depth variations in different circumstances. Thus the size of the screen may affect the perceived 3D effect. Moreover, the perceived position of the object, if too close, may be outside of the comfortable range of comfortable for a viewer's eyes, leading to discomfort or double vision.
[00153] Other viewing parameters may affect the 3D effect of a viewed image. For example, variations in eye spacing (interocular separation) of viewers will similarly lead to different angles at which each eye sees an object and thus a different intersection of these angles and thus a different depth of the object. Furthermore, while we described the effect of the position of the viewer on the 3D effect only in terms of distance from the screen, it will be appreciated that the position in space in general of a viewer will affect the perception in 3D of objects displayed on a stereoscopic display. Figure 12 illustrates three viewers in three different positions relative to a stereoscopic display viewing a same object. Each perceives the object at a different point in space.
[00154] In each of the above examples, the perceived position of an object was shown as varying as a function of the display or viewer. However, the perceived position of an object also depends on the manner in which the object was stereoscopically captured. Parameters at capture affecting the parallax, 3D effects, or generally the depth perception are capture parameters. During capture of stereoscopic content, camera pairs are positioned in such a way as to capture left and right eye perspectives of images as they would appear to a viewer viewing the display on which it will be displayed at a particular position. Camera positioning and orientation are examples of capture parameters. In particular, the choice of spacing (separation) between the cameras of the camera-pair and the angle of convergence (the angle by which the cameras are aimed towards one another -if they are parallel, they are said to have an infinite angle of convergence) affect the stereoscopy (3D effect) of the resulting image at a particular viewing parameter. The capture parameters may also include focus and the position and/or orientation of the cameras other than the spacing and angle of convergence. For example, if the cameras have been imperfectly aligned and have a certain vertical offset or an uneven and/or vertical angle relative one another, these may affect the depth perception for the viewer (e.g. in a negative way).
[00155] It has been discovered that a comfortable stereoscopic viewing region for stereoscopic content displayed on a stereoscopic display of a certain size is to a great extent influenced by camera separation and camera angle employed in capturing the stereoscopic content.
[00156] Figures 14 and 15 illustrate some of the effects that can result from varying the parameters.
[00157] Figure 13 shows a stereoscopic display showing two left-right vie-pairs of an object D at two different position ("placements") on the display. The two placements may be the results of different capture parameters. Filming a scene with first capture parameters would result in object C having placement 1 on the display. In this placement a viewer at the position labeled "viewer 1" would perceived object C at the position illustrated. If the viewer where to move to the location labeled "viewer 2" in the Figure, we know from our earlier discussion that object C would appear in a different place in space. However, where the image originally captured with a second different set of capture parameters that resulted in object C being located at placement 2, shown, the perceived position of object C would be the exact same for this viewer as was for the first viewer with the first set of capture parameters.
[00158] The two viewer position may represent different viewing parameters. Using the above-described algorithm and exploiting the ability it provides to transform the geometry of the image, it will be appreciated that it is possible to reformat the stereoscopic image pair so as to maintain the position of object C even when the viewing parameters change, as for example from the viewing parameters of viewer 1 and the viewing parameters of viewer 2. Indeed, in the image tube transformation step, the image tubes may be transformed as described above to provide a placement of object C at placement 2 where it would originally have been showed (without reformatting) at placement 1, for example.
[00159] While two placements of object C have been shown on a same display at the same time, it is to be understood that the screen does not necessarily show both placements simultaneously. In fact, if the film is captured using a single stereoscopic camera pair with a particular capture parameters, this would result in only one image which would including object C at only one placement. However, as will be described further below, where views from multiple capture parameters are available, these may be displayed simultaneously in a manner such that each viewer sees only one such view at a time.
[00160] Figure 14 shows a single viewer viewing an object D on a stereoscopic display. Two placements of object D are shown, according to two different capture parameters. Using first capture parameters to capture an image comprising object D results in the first placement of object D on the display, which results in object D being perceived by the viewer as being at a first location, close to the display. Using second capture parameters to capture the image results in a second placement of object D on the display, which results in object D being perceived by the viewer as being at a second location, further from the display. Thus in this example, the choice of capture parameters affects the perceived depth of the object D.
[00161] These examples are simplified for the purpose of illustrating that capture parameters affect the perceived location in 3D space of the various objects in a stereoscopic image. [00162] In general, the capture parameters are computed to provide the right depth perception for given viewing parameters. Oftentimes, movie programs will be filmed using camera positioning tuned specifically for a central viewer watching it on a movie theater screen.
[00163] Stereoscopic system camera positioning for the acquisition of stereoscopic content is dictated to a great extent by the display on which the stereoscopic content is intended to be displayed in order to accommodate a range of viewer interocular distances (the distance between the eyes). The extent of the region of space which can be displayed in three dimensions without causing undue eye strain to the observer can be controlled by the acquisition system's geometrical (and optical) parameters. Therefore stereoscopic (video) content is captured for output/playback on an intended stereoscopic display having a particular size. The cameras of the stereoscopic content acquisition system are spaced an initial distance apart and oriented towards the object of interest, also known as toe-in or angle of convergence.
[00164] In general viewing a program with viewing parameters other than those for which the capture parameters where optimized may lead to a degraded depth perception. Viewers may experience substantial viewing discomfort, distortion and/or loss of stereoscopic perception.
[00165] For example, for viewers watching from non-optimal areas of the theater, the 3D effect may be skewed such that objects appear unnaturally flattened or stretched in the depth direction or otherwise distorted. Spheres may appear deformed like a rugby ball or objects may appear as flat sheets with varying depths. Such may be the effect of viewer positioning in a sub- optimal place. In other examples, if the as-captured stereoscopic (video) content is presented/played back on another stereoscopic display having a different size, the gradation of the depth perception may be quite pronounced. This occurs, for example, when watching a program filmed for a large screen on a screen having a drastically different size, and/or a viewer having a drastically different position than the big-screen viewer. This occurs for example when production movies are adapted from the big-screen version to Blu-ray™ version that will be played on small screens.
[00166] It has been discovered that the observing stereoscopic content at viewing parameters other than those for which the capture parameters were established, can be reduced through reformatting the stereoscopic content. It is in this optic that the above-described reformatting system has been developed. Reformatting can be performed, for example, by applying a geometric transformation to images captured at first (original/real) capture parameters to generate transformed images approximating or simulating the images that would have been generated at second (other/simulated) capture parameters. Such a transformation can be used to change stereoscopic images that were captured with capture parameters intended for viewing parameters other than those at which the stereoscopic images will actually be viewed, in such a way as to recreate images optimized for the real viewing parameters. This may simulate images that were captured with capture parameters optimized for the real viewing parameters. For example, a movie filmed for the big screen may be reformatted to be optimized for viewing on a screen typical of home 3D TV's by a viewer at a typical at-home viewing distance. [00167] Reformatting can also be performed to correct mistakes or imperfection in the original capture parameters. For example, if it is determined that the camera positioning (e.g. angle of convergence or spacing) was not ideal for the intended viewing parameters for a particular video, the ideal capture parameters may be found, as discussed below, and the images of the video may be reformatted to transform the images of the video such that they are as if they were captured at the ideal capture parameters. Alternatively, the above described image tube- based solution can be used to modify the image in the view space it will be viewed in to correct the effects of improper capture of the images. This is done by selecting a transformation of the image tubes such that the resulting reformatted image respects the comfortable viewing zone of the viewer. Additional intelligence can readily be built into the system so as to modify the location of objects in the 3D viewing space, e.g. to respect certain three dimensional proportions. Indeed, since the 3D location of a point in the view space can be readily derived from the image tubes, and since the image tubes, as shown above can be transformed to adopt virtually any other 3D location, it follows that an image can be reformatted not only to respect a comfortable viewing zone but also for many other purposes as will be described below.
[00168] Thus this gives rise to a new method of capturing stereoscopic video content. At present, capturing video content in 3D requires very precise computation of the ideal capture parameters. Only one set of viewing parameters can be optimized for, and considerable effort must go into ensuring that the stereoscopic camera pairs are at the optimal position for each scene. The camera placement computations often take into account the scene itself, e.g. to set where the neutral plane (where things are perceived, at the optimal viewing parameters, to be exactly where the display screen lies) lies and to select a point of focus and tune the cameras appropriately. Thus filming in 3D requires complicated calculations and recalculations at every scene and limits the ability to change scenes and move the cameras at will.
[00169] In order to streamline production, reduce cost and complication and to reduce the cost of camera calibration errors, and generally to reduce the burden of filming stereoscopically, a program may be filmed with deliberately non-optimized capture parameters. For example, the camera placement (e.g. angle of convergence and/or spacing) may be set at a fixed position for several scenes even though an ideal system may be otherwise optimized. The video so captured is then reformatted such that the capture parameters (now simulated) are optimized for each scene. Optimal capture parameters for each scene may be determined in any suitable manner. For example an expert may compute the optimal camera positioning using a cameral placement algorithm or other known techniques on site, while being relieved of the necessity to actually set the cameras accordingly. Alternatively, a visual analysis of known objects in the scene may be undertaken to identify distortion or other effects of improper capture parameters. Alternatively still, a test pattern may be implemented into the scene (in a manner ideally invisible or unrecognizable to the viewer) to be used for identifying distortion. For example, test patterns of a single frame may be implemented within a video and, provided that the frame rate is high enough, this would be generally invisible to the viewer. Alternatively still, a test pattern may be included at the beginning or end of every scene, to be used to compute actual and/or optimal capture parameters for the scene and to thereafter be removed from the video during editing. Test patterns will be discussed further below.
[00170] Where the ideal capture parameters vary (e.g. linearly) during a scene, they may be computed at several discrete points in the scene and a function of change may be computed such that for each frame (or for each discrete sets of any number of frames, if it is desired to change the capture parameters in larger steps than single frames) the optimal capture parameters may be found from the computed function and the frames (or set thereof) may be reformatted accordingly.
[00171] Alternatively instead of deriving a function from two or more discrete points at which the ideal capture parameters are computed, as above, the ideal capture parameters may be computed for each individual frame.
[00172] The above enables dynamically changing capture parameters for scenes where it is desirable, (for example where the depth of the focus changes) without physically changing the camera placements to the extent needed to achieve the capture parameters desired.
[00173] Thus a program is captured using capture parameters that are not necessarily ideal, but may be for example, simpler, easier and/or more cost effective or accessible to implement. Ideal capture parameters are then determined. The program then undergoes a reformatting step to reformat the program according to the determined ideal capture parameters. This determining ideal capture parameters and reformatting may be done for the entire program, scene-by-scene, for discrete portions of scenes or even in a dynamic manner, changing (e.g. linearly) even within scenes.
[00174] The identifying ideal capture parameters and reformatting the program may be done several times for a same program to optimize the program for different viewing parameters. Thus in a single production, the IMAX™ version, movie big-screen version and Blu-ray (or VOD, etc .. ) version of a program may be produced all at once. It is not necessary for the original capture parameters to be non-ideal for each version produced. For example, the capture parameters may be optimized during filming for the big-screen, and be reformatted during production to optimize the program for IMAX and typical Blu-ray™-connected displays. A film may also be produced using this method in several formats, for a variety of typical viewing parameters. In particular, a film may be offered in Blu-ray™ or VOD (or through other distribution means) in a selection of optimized viewing parameters such that the viewer/purchaser may select his screen size and/or layout. In the case of Blu-ray™, there may be several optimization of a film on a same Blu-ray™.
[00175] It is to be understood that the term filming as used herein may refer to actual filming with cameras or generating/rendering CGI animation or games. In a similar manner, the term "camera" as used herein (even when qualified as "original" or "actual" or "real") may be virtual camera positioned computed for the purposed of rendering a 3D graphics scene.
[00176] Other than the camera spacing and angle of convergence, other capture parameters that are imperfect for the viewing parameters. For example, where the vertical alignment or vertical angle is found to be not quite right, or where a single cameras angle is off, the error can be corrected using the reformatting techniques described herein without having to expensively re-film erroneous content.
[00177] In an early solution to the problem, it was been discovered that input stereoscopic image pairs of an input stereoscopic video stream formatted or acquired for presentation on a first stereoscopic display having a first size, can be reformatted for comfortable presentation on a second stereoscopic display having a second size by orthogonally projecting at least one of the stereoscopic image pairs onto a virtual plane passing through a stereoscopic image convergence axis perpendicular to a virtual central line of sight of a virtual camera positioned to provide comfortable stereoscopic content presentation on the second stereoscopic display. This camera placement-based solution was improved upon with the above image-tube based algorithm.
[00178] It was discovered that applying a geometric transformation to at least one image of a stereoscopic pair reflecting a change in a virtual camera position reduces discomfort when viewing stereoscopic content on a stereoscopic display for which the stereoscopic content was not formatted or acquired.
[00179] In accordance with one aspect of this solution is provided a method for reformatting an input stereoscopic content for proper presentation of three dimensional content on a second stereoscopic display having a different format than that of a first stereoscopic display for which the input stereoscopic content was generated, each stereoscopic display having a corresponding predetermined stereoscopic camera pointing angle and a predetermined intercamera separation, the method comprising: projecting each first stereoscopic channel image from a first virtual image plane perpendicular to a line of sight of a corresponding first camera onto a second virtual image plane perpendicular to a line of sight of a corresponding second camera to form a corresponding second stereoscopic channel image and re-pixelating each said second stereoscopic channel image.
[00180] In accordance with another aspect of this solution there is provided a method for reformatting an input stereoscopic content for proper presentation of three dimensional content on a second stereoscopic display having a different format than that of a first stereoscopic display for which the input stereoscopic content was generated, each stereoscopic display having a corresponding predetermined stereoscopic camera pointing angle and a predetermined intercamera separation, the method comprising: projecting each first stereoscopic channel image from a first virtual image plane perpendicular to a line of sight of a corresponding first camera located at a focus point of said first camera along said line of sight of said first camera onto a second virtual plane perpendicular to a corresponding second camera line of sight, said second virtual plane being located at a focus point of said second camera along said corresponding central line of sight to form corresponding second stereoscopic channel image and re-pixelating each said second stereoscopic channel image.
[00181] With reference to Figure 14, input stereoscopic content, including left and right stereoscopic image pairs, is provided along with original camera separation and original camera pointing angle parameters employed in capturing, generating or initial formatting of the stereoscopic content for an original stereoscopic display of an original size. Such original camera and original display parameters can be predetermined in advance, or provided in metadata with the input stereoscopic content. In the case of stereoscopic video content, original camera parameters can be specified once for the entire stereoscopic video program or on a scene- by-scene basis whereas original display parameters can typically be provided once for example at the beginning of the program.
[00182] In accordance with another implementation of this solution, original camera parameters are derived from the input stereoscopic content. Deriving original camera parameters from stereoscopic content is being described elsewhere, and is beyond the scope of the present description. A preliminary step of determining the original camera parameters from the input stereoscopic content subjects stereoscopic image pairs to a preprocessing step, for example to determining geometric aspects of the imaged scene. Without limiting the invention, the determination of the original camera parameters can be performed once for the entire stereoscopic program, once per scene, for each stereoscopic image pair or can include averaging over multiple stereoscopic image pairs. The stereoscopic program can dictate the type and extent of such preprocessing, for example a static stereoscopic video scene would require less preprocessing whereas an action stereoscopic video scene would require more.
[00183] The angle between the central lines of sight of the original cameras show in Figure 14 is referred to as the angle of convergence and is a primary stereoscopic content acquisition parameters; another primary stereoscopic content acquisition parameter is the distance between the cameras. The point in plan view where the central lines of sight converge is referred to as the point of convergence. More generically, employing a dual camera stereoscopic content acquisition system convergence is achieved over a line, for cameras having long focal length lenses over a substantially straight vertical line of convergence.
[00184] As another preprocessing step, comparing a display size of a stereoscopic display on which the input stereoscopic content is intended to be presented with the original display size, can be employed to trigger the reformatting of the input stereoscopic content in order to minimize stereoscopic viewing discomfort as presented herein:
[00185] The original camera parameters and original stereoscopic display specification defines for each channel a first virtual image plane (Loami, Lcam2) illustrated in Figure 8, perpendicular to the line of sight of a corresponding camera, the first virtual image plane passing through a vertical convergence axis line (Dconv) defined by the original camera separation and the original camera pointing angle(s) (6cami, ecam2) where the central lines of sight of the cameras intersect.
[00186] Similarly with reference to Figure 8, the display parameters of the stereoscopic display on which the stereoscopic content is intended to be presented require a second camera separation and a second camera pointing angle for comfortable viewing of the stereoscopic video content.
[00187] In accordance with an embodiment of this solution, and referring to Figure 9, each side image of the original stereoscopic image pair is projected onto a second virtual plane perpendicular to the corresponding (same side) second camera line of sight passing through the convergence axis. For clarity, the above described image processing does not use information from the other stereoscopic channel to provide content reformatting for a given stereoscopic channel.
[00188] The corresponding image projections create a new second stereoscopic image pair. The projection may be orthogonal, which projection imparts a twist and shift to the original image in reformatting the corresponding new image. The twists and shifts are complementary between the reformatted right and left images. Although proposed image processing may not be in every case fully faithful to the imaged scene because all image pixels are modified proportionally, experimental results confirm that such orthogonal projection provides a desirable reduction in viewing discomfort for original cameras calibrated to focus at convergence. Particularly small camera pointing angle differences can be accommodated with minimal processing providing a substantial reduction in eye strain.
[00189] In accordance with another implementation of the proposed solution, the original cameras either through improper calibration or by intent are not focused at convergence. In this case each original stereoscopic image is orthogonally projected onto a second virtual plane perpendicular to the corresponding second camera's central line of sight and located along the line of sight at the point of focus of the original camera. Accordingly an original directorial intent is more faithfully reproduced while providing a substantial reduction in eye strain.
[00190] Examples where the points of focus of the left and right cameras are not the same as the point of convergence include the cameras non being ideally set up or done purposefully to achieve some a visual effect, for example poorly focussed eyes, e.g. to simulate being tired, euphoric or disoriented. In such a case the second virtual planes of the left and right images may not intersect the point of convergence of the two cameras. Projecting an original image from the first virtual plane of the original image to the new second virtual plane which is located proportionally spaced from the (new virtual) point of convergence to mimic the same (error) effect as in the original input stereoscopic content. For clarity the actual point of focus of the cameras represents a secondary parameter which may or may not be taken into account, depending on whether the system is assumed to be ideal or not.
[00191] The invention is not limited to the above assumption that cameras are placed side by side along a horizontal line and oriented towards one another at a slight angle of convergence to mimic the eyes of the viewer. Other possible secondary parameters include camera positions in a three dimensional world relative one another. It is possible that due to error or intent the original cameras were not perfectly placed, including one camera being recessed or brought forward. The pointing angles of the two cameras would not be perfectly equal, etc. So long as information regarding such secondary parameters is provided (or can be inferred/determined), the second virtual plane of the left and right images can be defined based on the central line of sight and the focus of each camera. In the absence of one or two secondary parameters assumptions can be made. For example, if it is known that the left camera was recessed a bit while the actual focus point of the left camera is not known, the location and orientation of the second virtual plane can be derived from the vector of the line of sight assuming that the focus is at the point of convergence where it crosses the right camera's central line of sight. As another example, if the two camera's central lines of sight do not actually cross (askew) because the cameras are a bit misaligned vertically, the point of convergence could be set as the middle of the shortest line linking the two central lines of sight.
[00192] With such secondary camera parameters, either provided, for example in metadata, or determined from preprocessing the input stereoscopic left and right images, by applying an appropriate image transformation, in accordance with the proposed solution a projection, to reformat the input images in the stereoscopic pair according to the new parameters, the original stereoscopic content can be reformatted not only to adapt it for presentation on a different sized stereoscopic display on which the stereoscopic content is intended to be displayed, but rather/also to correct imperfections in the original content capture. For certainty, the original cameras need not be imperfectly set up, any virtual secondary parameters can be employed to achieve desired effects.
[00193] In practice, the original left and right stereoscopic images provided as input undergo a transformation to simulate new camera parameters for the intended presentation to reduce viewing discomfort by reducing eye strain. The images are reformatted (transformed) to project them on the second virtual plane in which they would have been had the original images been captured by the virtual cameras having the second (virtual) camera parameters. Any suitable projection technique may be used to reformat the original input images. The goal is to achieve as accurately as possible what the images would have looked like had the images been taken with the second virtual cameras. Given that objects in the imaged scene are three dimensional, there are regions of space which would be occluded from one camera's viewpoint but visible in the other. Consider for example a vertical cylinder having rainbow colors drawn all around it. Seen from one camera's point of view at one camera angle some colors can be seen which cannot be seen from another camera point of view at another camera angle. In this sense, each original input stereoscopic image has some information which remains after the transformation however which should be absent (occluded) in the transformed stereoscopic image when compared to a corresponding ideal virtual image. Likewise each original input image lacks some information (which is occluded in the original input image) that would be present in an ideal virtual image. For small camera angle differences and small positioning differences between the original and virtual cameras, which usually lead to the type of eye strain being mitigated herein, there is typically very little excess/missing image data (occlusions).
[00194] With reference to Figure 9, an object in the real scene which lies between the two planes illustrates aspects of the proposed solution. Where the line connecting the object and the first original camera intersects the first original plane, is where the object is located in the original image. Where the line connecting the second virtual camera and the object intersect the second virtual plane is where the object should be located in an ideal second (simulated) image. Since orthogonal projection is used, the actual location of the object on the new second image is orthogonally above the point on the first original image where the object is located in the first image. There is a small difference between the ideal and the transformed locations which is small for objects near the first plane, desirably close to the focal plane of the camera. The transformation may yield some less-than-ideal results for background objects and foreground objects away from the focal plane of the camera, desirably these objects are out of focus and likely to attract less of the observer's attention.
[00195] In accordance with the proposed solution, a reasonable approximation of an image taken with the virtual camera parameters is to use a projection as the transformation step as described hereinabove. The application of an orthogonal projection to each original input image onto the corresponding virtual image plane can be done using any suitable projection technique. For example, the pixels of each original image can be mapped onto the corresponding second virtual plane by providing each pixel of the original image a new position in an area of the second virtual plane orthogonally above the corresponding original image. In so doing, the transformed pixels may be stretched or compressed by the mapping (depending on whether the second stereoscopic display is larger or smaller than the original stereoscopic display). For certainty, the invention is not limited to the manner of implementing orthogonal, projections other suitable techniques may be used.
[00196] As a result of the orthogonal projection transformation, the original image is morphed to appear stretched out and shifted (with respect to the central line of sight of the virtual camera, which may not be in the center of the new virtual image). An area in the second virtual plane of the second virtual camera corresponds to an area where the virtual camera would have captured the required image. This is a rectangular area centered around the central line of sight of the second virtual camera, referred to hereinafter as the virtual image area. The mapping on the second virtual plane is located orthogonally above the original image in the original image virtual plane but may not cover entirely the virtual image area. Thus by having projected the image orthogonally above the original image plane, it has become shifted with respect to the second virtual camera, and has also deformed by the difference in angle of the planes which causes the pixels of the original image to have a certain slant or stretch in the second virtual plane.
[00197] With the virtual image area partially covered with the projected pixel mapping, in accordance with one implementation, the projected pixels can be shifted to cover the entire virtual image area (assuming it is larger in the second virtual plane). In accordance with a preferred implementation, however the projected pixels are left as projected and re-pixelisation is performed by applying a pixel grid over the virtual image and assigning values to pixels according to the value of the projected pixel mapping at grid coordinates. In one example, each grid pixel is assigned at least one value from: brightness, chroma, at least one pixel component (RGB, YUV, etc.) in the projected mapping that is in the center of that pixel. In another example, an average (or other weighted value) in the projected mapping is assigned for the area of the pixel to the pixel.
[00198] The resulting image has black bands on either one or two edges because the virtual image area was not fully filled by the projection mapping. While the left eye and right eye of the viewer will not see whole images, because of left and right image overlap providing a substantial region over which the second left and right images can be fused to perceive the stereoscopic effect, it will appear that whole images are being displayed. This is because the black band(s) of the left reformatted image will overlap with a filled-out portion of the right reformatted image and vice versa. Thus the reformatted images will appear to the viewer as whole images. There will however be no stereoscopic effect in the black band(s) region(s) since in any such area the human viewer will only see one image which lacks parallax. This lack of stereoscopic effect is limited to the sides of the reformatted stereoscopic image and not in the main area of focus.
[00199] Notably, orthogonal projections yield a shift and twist of the image to simulate a different position and angle of the camera, and are computationally light. Without limiting the invention, non-orthogonal projections can be employed for example by a direct calculation or two-step projection employing a first orthogonal projection on an intermediary plane and then employing a second orthogonal projection from that plane to the intended final plane. Alternatively a two-step (orthogonal or not) projection can be employed using an intermediary surface of projection which is not flat having a shape intended to impart a certain non-linear transform to the input image. Thus an intermediary plane could be used in a two-step projection approach to distort the output image or to shift it. In an extreme case employing a non-flat intermediary projection surface can provide a different transform effect for each pixel by using a sufficiently detailed intermediary surface. That is, pixels could be treated differently by using a distortion surface which causes certain areas of the input image to be projected differently than others due to a varying slope of the intermediary surface.
[00200] In accordance with another embodiment of the proposed solution, pixel depths are taken into account and differentiated modifications are applied for different image areas or individual pixels in each original image. Both images of each original stereoscopic pair are processed to infer imaged object depth for each pixel or other data such as a depth map (a mapping of pixel depths, sometimes -but rarely- provided with a stereoscopic video feed) is employed.
[00201] While extensive reference has been made in the above to the reformatting of input stereoscopic content for reducing eye strain when displayed on a different stereoscopic display than that for which the stereoscopic content was acquired, the invention invention is not limited thereto. The proposed solution can be employed to account for changes in the observer's position with respect to the stereoscopic display.
[00202] An advantage of the use of projection to generate the transformed image is that this is a computationally-light technique, which may be generated in real-time. Thus while a video (e.g. film) may be produced in a reformatted format for several different viewing parameters as described above, with real time processing it may also be generated and distributed (received) in a single format and reformatted in situ, even in real-time. This real-time reformatting may be used in movie theaters (e.g. of non-conventional dimension) or may be used at the home to account for the large variety of viewing parameters in homes.
[00203] We have described a reformater solution as an image tube-based reformater or a camera placement-based reformater. The presence of a reformater in a viewing system opens up several new possibilities.
[00204] In a particular embodiment, the reformater is used to implement a child safety viewing mode on a video viewing system. It is believed that forcing the eyes to focus beyond the usual range of convergence angles may be bad for vision. In young children who are still growing and whose anatomy is still developing in particular, it is feared that such strained focusing may lead to abnormal development of the eyes and eye muscles and cause vision problems on the long term As discussed herein, most 3D content is captured for a particular set of viewing parameters including an interocular distance. Commonly, it is assumed that the viewer will be an adult with an average interocular distance. However, if viewed by a user with a different, e.g. smaller, interocular distance, as would be typically the case in children, the depth perception will vary on account of the difference in this particular viewing parameter, which may cause the video to feature parallax beyond the normal range. The development risk to children has led manufacturers of 3D equipment of all kinds (televisions, handheld game consoles, etc... ) to recommend limits to exposure for children and even avoidance of 3D altogether for children below a certain age.
[00205] In accordance with this embodiment, the video viewing system comprises a display and at least one user input for enabling or disabling the child safety viewing mode. In one example, the commercial video viewing system is a television and the input is controlled by the user via a graphical user interface controllable by a remote control, whereby a user can access a menu in which a visual presentation (e.g. icon) of the child safety mode option is presented to the user. The user may activate child safety mode by selecting with buttons on the remote control the child safety mode icon and pressing a selection button. Instead of the menu, child safety mode could be accessible via a button on the remote control. In an alternative embodiment, the vide viewing system may be a gaming console with a child safety toggle button or a 3D-enabled mobile device (phone or tablet) with a graphical user interface including a setting menu including a display submenu accessible as visual presentation (e.g. icon and/or text heading) in the setting menu by a user pointing device such as a touch screen. The display submenu may comprise a further 3D submenu similarly accessible, and the 3D submenu may comprise a visual presentation (e.g. an icon) representing the child safety mode option, which presents the user the possibility of toggling child safety mode by interacting with the corresponding visual presentation using the pointing device, e.g. by pressing on it on a touchscreen. [00206] In a first example, activation of the child safety mode causes a change in the viewing parameters used to compute the reformatting performed. In the image tube-based implementation, the new viewing parameters used to reformat the image comprise a smaller interocular distance more typical of young children. In the camera placement implementation, the camera placement algorithm used to "place" the virtual cameras, is a camera placement algorithm that takes interocular distance as an input and it is provided the smaller interocular distance more typical of young children.
[00207] A refinement of the above is possible where the input device allows for additional information to be received from the user. In a simpler implementation, there is not just one possible "smaller interocular distance" but a range of interocular distances, for example corresponding to different children ages. Thus a user may be presented an option of selecting one of a plurality of age groups for the intended viewer. There may be for example a "6 year or less" option, a "6-9 years" option, a "9-12" years option, and a "teen" option. These options may be represented in a menu, e.g. a pop-up menu brought forth on the screen in response to the activation of the child safety mode. Selection of an option is input by a user using an input device, e.g. a pointing device like a touchscreen on a mobile device or a remote control with arrow buttons allowing selection of an option and a select button to confirm the selection. (In this example, these options are presented upon activation of the child safety mode as described above. It will be appreciated that in an alternative embodiment, activation of a child safety mode may be performed by virtue of selecting an option appropriate for children, like an age category in the above-described menu. In this case, there needn't be a specific "child safety mode" button, but rather the menu above may represent the whole age gamut including "adult". Child safety mode may be activated by user input constituting of the activation of an option suitable for children.)
[00208] If stereoscopic images are reformatted for a interocular distance that is smaller than one's own, all other things being equal, the 3D image may tend to be flattened towards the plane of convergence (the plane of the screen in typical one-screen displays). While this leads to an inferior 3D experience, it is not considered harmful, and in any event is far safer than when the interocular distance is set too high. In the latter case, the 3D effects may appear exaggerated, leading to parallax well beyond the comfortable viewing zone and even eye divergence. For this reason, upon selection of the child safety mode (or upon selecting an interocular distance selector, as will be described below), a user may be presented a visual messages instructing the user as to how to best select the interocular distance-related option. Such instructions may include instructions on how to measure interocular distance (for the below example) and/or other visual indication displaying a message instructing the user as to select an option. In particular the message may warn the user that when in doubt, it is safer to select the option corresponding to the younger age/closer interocular distance.
[00209] In another example, instead of displaying age categories as options, the options may be displayed directly as ranges of interocular distance. In such a case, the instructions on how to measure such distance will be particularly useful. While the above refers to a child safety mode, it will be appreciated that this indeed represents a user safety mode, as it can apply not just to children but to anybody that does not have the interocular distance used by the content creator in the calibration of the capture system.
[00210] As a further elaboration, the system may allow for the user to input information corresponding to the interocular distance, either by inputing the actual interocular distance of the viewer through a numerical input interface or by providing other information such as a date of birth through which interocular distance may be estimated.
[00211] It is to be appreciated that the child safety mode may moreover impose additional restrictions on the reformatting. Indeed, in addition to informing the viewing parmater used in reformatting, the child safety mode may, for example, cause the reduction of depth by, e.g., implementing a smaller comfort zone. It may also cause a reduction of the image size, e.g. by using the reformater 3D zoom, to compensate for kids sitting too close to the TV. This may also be done in 2D, by simply reducing the image size, e.g. by downsampling or downscaling.
[00212] In another embodiment, the reformater is combined with a viewing parameter acquisition module (or viewspace module) that identifies viewing parameters and provides them to the reformater to derive Ων. In one example the viewspace module stores information relative to the size of the screen and relative position (e.g. distance) of the viewer from the screen. In one example the viewspace module is implemented by a processor in a viewing system (in this example a 3D TV) that implements a graphical user interface allowing a user to input viewing parameter information for the viewspace module. In particular, the viewspace module may be hard-wired with information on the size of the display but the graphical user interface may prompt the user, upon a first activation of the television to input the distance of the couch to the TV. The prompt may be implemented by a visual message on the display including a schematic diagram of the measure to take and a text box may allow the user to enter on the remote control the number of, e.g., inches between couch and TV.
[00213] In an alternate embodiment, the viewspace module may use alternate means to identify viewing parameters. In particular, televisions today occasionally come with front-facing cameras for VOIP applications. In the field of digital cameras known etechniques exist for identifying where in the frame faces are located for the purposes of focusing the camera. In one example, the viewspace module is connected to a front-facing camera and uses known face- identifying techniques for determining where users are located relative to the screen Distance (depth) of the users can be ascertained using previously user-mput distance-to-screen information, using range-finding techniques or by assuming a certain interocular distance (either a common or average interocular distance or one corresponding to a previous user input, e.g. as described above) and computing a distance based on the difference between the observed and assumed interocular distance. Alternate means of ascertaining the position of a viewer include using 3D glasses, in systems where such are used. By outfitting glasses with a detectable indicia detectable either by the front-facing camera or by a dedicated piece of hardware a position may be ascertained, e.g. by visually identifying the indicia and comparing it's perceived size to a known size. Any other means, such as GPS tagging or local triangulation may be used.
[00214] In the case where multiple viewers are present several choices or reformatting are possible. Regardless of reformatting criteria, the 3D image will look slightly different to each viewer since each have different viewing parameters. In such as case, it may be desired to simply reformat according to viewing parameters whereby a viewer is sitting in front of the display at a particular (e.g. previously inputed) distance. This will result in a central ideal viewing position at the central location and progressively distorted views as one distances oneself therefrom. Alternately, one viewer's position may be selected as the target viewing parameters for reformatting such that at least one viewer has an ideal view. Alternatively still, the viewspace module may locate a postion geometrically centered between all the viewers as location of the target viewer in the viewing parameters.
[00215] In an additional improvement, the reformater may have as a criterion (on top of the ones discussed above, or alone) the respect of the comfort zone for all viewers. That is, while only one viewing location may be the ideal viewing location which depending on the viewing parameters fed to the reformater by the viewspace module, the reformater may be provided additional sets of viewspace parameters corresponding to all the viewers not in the ideal location criterion under the criterion that under no circumstance should the 3D image extend beyond the confort zone of any viewer. In such a case, an image tube-based implementation performs a first transformation of image tube to the new viewspace, transforms the image tubes according to the other criteria desired (e.g. to ensure proportionality form the ideal location's perspective and to ensure that the comfort zone for that viewer is not violated) and then subsequently performs a further transformation for every other viewspace corresponding to all other viewers to make sure that the image tubes will not lead to a violation of the comfort zone for any viewers. If a violation is found, the image tubes are further transformed, e.g. to reduce depth effect, to prevent it and the resulting transform is once again verified for all viewspace parameters. It should be noted that it should always be possible to satisfy all comfort zones as there is no lower limit to the 3D effect (it may be decreased asymptotically to 2D which is free from depth perception comfort issues). However, it is also possible to set limits to prevent over distortion for the sake of non-central viewers.
[00216] In another way of dealing with multiple viewers, the reformater provides multiple reformatted stereoscopic image pairs targeted at different viewers, e.g. by reformatting an input stereoscopic image pair using several different instances (in sequence or in parallel) of the method described herein. One of the particularly useful consequences of the foregoing, is that the present reformater makes it possible to provide a multi-view output formatted to provide images for users at different 3D positions from the screen, including depth, not just angles. aS such a multiview generator implemented by the image tube-based implementation of the reformater may provide a better viewing experience on a autostereoscopic or multiview display than has so far been possible. As such, it is an object of the present invention to provide a mutliview generator comprising the reformater described herein, wherein the reformater generates a plurality of image view including at least a plurality of stereoscopic image pairs, each of said plurality of image pair being optimized for the viewing parameters (e.g. for the viewing location) of a different viewer, said image views being for display on a multiview display. Taking into account the angle at which each of the different viewers are located from the screen, the plurality of image views are then displayed on the screen in such manner as to present to each of the different viewers the respectively optimized image pair using known autostereoscopic/multiview display methods. Alternatively where a depth or disparity map is available, the reformater may use one (or more) image and the disparity map to generate image tubes and therefrom generate stereoscopic image pairs for the users according to the method above, although disparity estimation will not, in this case, be required.
[00217] Viewing equipment in the home may be equipped with reformatting technology to enable an ideal 3D viewing experience in the home. The camera placement-based solution like the image tube-based solution may be implemented in hardwear for real-time running.
[00218] Such viewing equipment may include televisions, VOD terminals, Blu-ray™ players, gaming console. They may be pre-programmed to reformat according to particular viewing parameters and may receive the original capture parameters alongside the 3D program, for example in metadata provided therefor. Alternatively, they may provide customization tools to customize the reformatting to particular viewing parameters or to a particular program.
[00219] For example, a viewing equipment may identify on its own the position of the viewer and may already know, or be inputted the size of the display. Any method for locating the user may be used such as facial recognition combined with range finding, methods used in autofocussing of digital cameras, detecting sensors, transponders or reflectors in 3D eyewear, etc... If multiple viewers are present, the viewing equipment may identify an average viewing parameter for the multiple viewers, or may provide multiple instances of reformatted program if it is possible to display different views to different viewers (e.g. with shutter glasses at offset timing or by providing different views in an autostereoscopic display's different rays of visibility).
[00220] Alternatively a user may input its viewing parameters, such as angle and distance to the display and/or display size. The program may then (e.g. in real time) be reformatted for the particular viewing parameters.
[00221] At test pattern may also be used to determine viewing parameters. A test pattern that has a clearly known shape (e.g. a sphere, a cube at an oblique angle or an "x, y, z," unit vector combination also at an angle) may be displayed to the user, who can then tell if the image is distorted or not based on whether the shape appears correctly. The user may then use an input tool such as a remote control to vary different aspects of the image until the shape appears correctly, the viewing equipment then determines the corresponding viewing parameters either using a lookup table or by revers -determining which capture parameters would have led to the distortion created by the viewer. Alternatively, the viewer may directly change the viewing parameters or the viewing equipment may provide the viewer with a feed that constantly changes the test pattern and the viewer may tell the viewing equipment using a user input tool such as a remote when the test pattern looks right.
[00222] Alternatively still, video programs may be provided with a test pattern (e.g. the known image) at the beginning of the program, the viewer may then provide inputs to the distortion experienced and the reformater may apply changes in simulated capture parameter accordingly until the image appears undistorted (or less distorted) to the user. Thus the change in capture parameters can be directly determined for a particular video.
[00223] Basically, in a non-limiting embodiment the reformater may be adjustable subjectively. That is the input of the viewing parameters may be done, not by precise manual entry or parameter values (e.g. distance-from-screen, inerocular distance, etc..) nor by visual acquisition by the viewspace module but by a user-subjective evaluation. In particular, the a reformater may be implemented on a viewing system comprising a display and a user interface for allowing a viewing user to interact with the viewing system, the user interface comprising adjustment mechanisms for allowing the user to adjust reformatting parameters. The reformatting parameters may be a Z-direction stretch, or may reflect viewing parameters (e.g. distance from screen, interocular distance, position relative to screen or any function thereof). In one example, the viewspace module causes the display by a processor of a graphical user interface of a visualization of instructions instructin gtrhe user to adjust the image using buttons on a remote control. In this particular example, the user uses the left and right arrow buttons to stretch or compress in the Z-direction the image until it looks subjectively right to the user. In a particular use case, the user may pause a video when a human head or another easily recognizable feature is in view and adjust the reformatting as described until the geometry, and particularly the ratio of the Z to the X and Y directions looks right. In one example, a video stream may begin with one or more well-known shapes (e.g. spheres and wireframe boxes) for that exact purpose.
[00224] In a variant, the user may have multiple control inputs for varying Z-scaling at different depths. In a variant of the above example, the user can uses the side arrows to compress or expand depth at near the display depth (convergence plane) and uses the up and down arrows to compress and expand depth at near-viewer depths, the reformater plotting and applying a depth-variance function satisfying both inputs.
[00225] The present reformater allows for the universal reformatting of images for all in a manner tailored for all viewing system such that video shot on a cell phone stereoscopic camera may be enjoyed on a large screen home theater system and movies shot for image or big screen theaters may be enjoyed at home on a TV or tablet.
[00226] It will be appreciated that the reformater can be used in a post-production setting to adjust a video stream in any of the manner mentioned above. In particular, the reformater is useful for fixing camera calibration errors, for correcting for moving/changing camera parameters rather than to recalibrate cameras, for reformatting for different viewing parameters, e.g. target screen size, etc...
[00227] The skilled person will appreciate that the stereoscopic reformatting scheme described above can be implemented in programming as described to create a suitable stereoscopic reformater in software or hardware. Specifically, the functionality described hereabove can be implemented, for example, in software code to create a software program capable of reformatting 3D content according to the above scheme. A such software may be particularly useful for so-called "offline" reformatting, that is to reformat content in non-realtime for storage for later display under corresponding viewing parameters, as will be described in more details below. This algorithm, however, has been developed specifically to enable rapid real-time performance and implementation in hardware (e.g. FPGA or ASIC). A skilled person will also appreciate that the above-described scheme and algorithm can be programmed in an appropriate hardware-defining language to program a chip to perform the functionality described herein. The description above has been provided in a form intended to enable a person skilled in the art to understand and replicate the invention.
[00228] A reformater programmed according to the foregoing description may comprise the modules described and illustrated, although certain modules may be merged (e.g. the disparity map estimation and refining, depending on the refining method used) as will be understood by a skilled programmer. An example of an implementation that a skilled programmer may create would be one wherein an externally -built (e.g. purchased) disparity estimation module is used (optionally, a refining module is added to it externally), a transformation engine is programmed to comprise the functionality illustrated in the "image tube generation and transforming" and "disparity re-construction and view synthesis" portions of Figure 4, and external post-processing software is used on the input of the transformation engine.
[00229] The need to employ precious processing and storage resources means that not all viewing devices may necessarily be able to implement this reconfiguration scheme immediately. Moreover even if all stereoscopic viewing devices are made with reconfiguration capability, there will still remain legacy devices consuming stereoscopic content that do not have reconfiguration capability.
[00230] Reconfiguration is one method of obtaining multiple configurations of stereoscopic content. Depth-image based rendering (DIBR) techniques also exist that allow generation of different configuration on the basis of a depth-image which provides the depth or disparity of pixels from a particular viewpoint Combined with at least one view-image DIBR techniques are used to generate two or more views to provide a particular stereoscopic configuration for a viewer. Thus if a depth-image is available, DIBR techniques can provide another method of generating multiple configurations of stereoscopic content.
[00231] There are other ways of obtaining multiple configurations of stereoscopic content. For example, the content may be generated the content into different configuration directly at capture. Some capture schemes proposed employ more than two cameras to capture content. In such a case, it is possible to obtain multiple viewpoints of a scene which can be used to create multiple configurations each characterized by different viewpoints being presented to the viewer's eyes. It should be noted that two different configurations may share a viewpoint. For example in two different configurations, the viewpoint of a scene presented to the left eye of a viewer may be the same but the viewpoint of the seen presented to the right eye of the viewer may be different such as to provide different depth perception. Or the viewpoint of a scene presented to the left eye of a viewer in one configuration may actually be the same as the viewpoint of the scene presented to the viewer in another configuration provided that the other viewpoint shown is appropriately selected to provide depth perception in a viewing environment.
[00232] Stereoscopic content made from computer-generated and rendered into images can be also rendered into multiple viewpoints to create at generation different configurations of the content.
[00233] In general, however, current methods of 3D film production produce only one configuration for the stereoscopic content, which is adapted to a cinema-room-and-adult-viewer viewing environment. Since reconfiguration is highly technically challenging and resource intensive some of the best techniques may not be available for real-time application on most viewing devices. However, it has been found that very high-quality reconfigurations can be generated with offline (non-real-time) algorithms running on sufficiently powerful machines, optionally with human input. It is also envisaged that stereoscopic content could be generated in multiple configurations as discussed above. However, there is currently no suitable means of distributing such content such that the best configuration of stereoscopic content is provided to remote users.
[00234] Figure 16 illustrates a stereoscopic content distribution system 200 according to a non-limiting embodiment. In this example the content distribution system provides video-on- demand (VOD) service which allows a customer 215 having a smart TV 220 to rent movies that are then streamed directly to their TV 220. Only one customer is shown in this illustration but it will be understood that the system may be used for distribution to a large number of customers. The system 200 has several parts that ensure the quality and non-reproducibility of the video as may be required by the video content providers such as film studios that produce stereoscopic films.
[00235] At its core, the system has two main portions: the content management system 205 and the content storage system 210. In this example, the content management system 205 and the stereoscopic content distribution system 200 are separate entities, however as will be described further below, the two could be embodied by a same entity such as a single server.
[00236] The content storage system 210 is responsible for storing the digital content and in this example, for streaming it directly to the customer 215 via a content distribution network 225. For this reason, the content storage system 210 is also considered a content provisioning system, although it will be simply referred to as content storage system 210 for simplicity. In this illustration the content distribution network 225 is shown as separate from the content storage system 210 but it could also be considered to comprise the content storage system 210. [00237] The content management system 205 is responsible for managing access to the stereoscopic content in the content storage system 210 by the customer 215 and by any other customers. It will now be described in more details.
[00238] In this example stereoscopic content are video programs and more specifically stereoscopic films that are provided by film studios in master source files 230. The master source files 230 provide the stereoscopic content in multiple stereoscopic configurations. In this example, each stereoscopic configuration for a particular film is treated like a separate film and provided separately in different files. In this example, the stereoscopic content is received in multiple configurations each having two views, a left- and a right-eye view which are each treated like a monoscopic film, meaning that each view has its own source file, however the two together are considered a single master source file 230 for the purpose of this description. Other manners of representing the stereoscopic content are possible. As shown, the master source files 230 may be provided by various means, such as electronically by FTP or physically in a hard disk or tape.
[00239] At the receiving end of the content storage system 210, a data ingest system 235 receives the master source files 230 and performs a variety of initial functions. The data ingest system 235 are first quality-checked and prequalified for ingestion. If a frame-compatible stereoscopic format is used, like Side-by-Side (SBS) or Quincunx-Side-by-Side (such as SENSIO™ HiFi 3D), the stereoscopic content is converted to that format at this stage. Master source files 230 are then converted to mezzanine files which in turn are quality checked. Any metadata is created or received and is appropriately formatted at this stage.
[00240] The mezzanine files are then passed to a demuxer system 240 in which the mezzanine files are compressed and transcoded to H.264. At this stage multiple bitrates may be applied to allow for adaptive streaming bitrates. This may lead to the creation of multiple encoded files. All associated files and metadata are then packaged to form complete deliverable.
[00241] The results are then passed on to a cypher encryption server 245, shown here as part of the content storage system 210 but which could be located remotely and operated by a separate data protection company. There the video content is encrypted and packaged. An encryptor generates an asset key that protects the whole asset. One or more content keys are used to encrypt video data. Each content key is placed into an entitlement control message (ECM). The asset key is registered with a cypher key management server 250 and used later to create permissions.
[00242] Encrypted files and metadata are stored on a media streaming system 255, which may be a separate server. The media streaming system is not only responsible for storing the encrypted media but also for streaming it via content distribution network 225 to the customer 215. The content is accessed through an application programing interface and delivered to approved licensors.
[00243] Access to the stereoscopic content on the media streaming system 255 is controlled by the content management system 205. An asset management system 260 is responsible for content management and user (account) management. The content management employs content provisioning system API to query catalog contents (based on customer credentials) and access catalog contents. The User management includes user registration and verification.
[00244] An application system 265 communicates directly with a remote user application on TV 220 via a network such as the internet and provides a storefront and acts as a media portal for ordering, account management, etc...
[00245] A reporting and analytics system 270 generates financial and usage reports that are made available on a periodic basis (e.g. monthly). These include default data on movie rentals including the quantity of movies that have been rented and the quantities of each configuration that has been provided to users.
[00246] A stereoscopic content distribution system 300 will now be described with focus on the distribution of different configuration versions of stereoscopic content with reference to Figure 17. Certain elements of the stereoscopic content distribution system 200 of Figure 16, such as elements associated with data encryption, are absent in the stereoscopic content distribution system of Figure 17; these may be absent from this implementation or, alternatively, may be present but simply not shown.
[00247] In this embodiment the stereoscopic content distribution system 300 comprises a content management system 302, a content storage system 304, a registration system 326, which work together to provide access to stereoscopic content at a user end 306. In this non-limiting embodiment, the content management system 302, content storage system 304 and registration system 326 are separate entities and more particularly are servers, but it will be appreciated that the functions of some or all of these could entities could be combined and performed by a single entity or server or could be further divided into more entities or servers as will become clear in the following description.
[00248] Broadly speaking, the content management system 302 is responsible for managing access to viewable stereoscopic content in a digital library 308 by a remote user. The content storage system 304 is responsible for storing the stereoscopic content and therefore comprises the digital content library 308. In this example, the content storage system 304 is also responsible for providing the stereoscopic content to the user end 306 and therefore also acts as a content distribution server.
[00249] In this example, the content storage system 304 is a server which comprises a large storage system containing the digital content library 308. The digital content library 308 is a content library that contains stereoscopic content in the form of one or more stereoscopic programs such as films or TV shows. For this example, it will be assumed that the digital content library 308 comprises a multitude of stereoscopic films. The digital content library 308 may comprise also non-stereoscopic programs such as 2D films, but for the purpose of this example, it will be assumed that only stereoscopic films are contained in the digital content library 308. For at least one of the stereoscopic programs in the digital content library 308, the digital content library 308 comprises a plurality of versions of the stereoscopic program. For the purpose of simplifying the description of the invention, it will be assumed that every program in the digital content library 308 is available in a plurality of versions, however this does not need to be the case. Each version of a program corresponds to a different stereoscopic configuration of the program. Each stereoscopic configuration corresponds to different viewing parameters or, by extension, to different viewing environments defined by those viewing parameters. However, a version may be re-configured from an existing version/configuration not for an entire viewing environment, taking into account every viewing parameter defining it, but rather only to account for a difference in one viewing parameter (for example IOD, as discussed below). Thus we say that each version is in a stereoscopic configuration corresponding to a respective set of viewing parameter, which set may be a plurality of viewing parameters (up to every viewing parameter defining a viewing environment) or simply one viewing parameter such as the interocular distance (IOD).
[00250] In this example, each version of a stereoscopic program is stored as a separate file in the same manner as different stereoscopic programs would normally be stored in a content library It will be appreciated that this can be done in other manners. In particular, as will be discussed below, reconfiguration of stereoscopic programs can be performed on-demand in realtime on the basis of a single version stored in the digital content library 308.
[00251] The functioning of the content storage system 304 is controlled by a control entity 310, in this case a processor programmed with the functions of the server.
[00252] The content storage system 304 is provided with an appropriate communication interface system 312 which directs communications with the content management system 302 and the transmission of stereoscopic content to the user end 306. The communication line from the content storage system 304 to the user end 306 is shown as unidirectional to reflect the fact that in this example the user end 306 directs requests and other data to the content management system 302 and merely receives content from the content storage system 304, but of course some feedback, e.g. for streaming control or for audio selections and other such controls may be provided from the user end 306. Optionally even this feedback may be transmitted to the content management system 302 and forwarded by the content management system 302 to the content storage system 304. The communication interface system 312 of the content storage system 304 comprises appropriate communication links to the content management system 302 and the user end 306. In this example, both are accessed via the internet and the communication interface 312 comprises a very high-bandwidth connection, however the content management system 302 can be separate but local in which case the content storage system 304 and content management system 302 can communicate through a different link also part of the communication interface system 312.
[00253] While the content storage system 304 is treated herein like a single entity, the skilled addressee will understand that the content storage system 304 may be distributed, e.g. like in a server bank. It is moreover not uncommon for a content distribution network to comprise several server locations at different places each offering the same service to users in respective geographical regions near them. Thus the singleness of the content storage system 304 and indeed of the content management system 302 and the registration system 326 shown here are not intended to be limiting.
[00254] The content management system 302 will now be described. The content management system 302 comprises a storage system which in this example is local storage 314 but could also be implemented by other storage means like cloud-based solutions, processing logic 316 and a communication interface system 318. In this embodiment the content management system 302 is a server.
[00255] The local storage 314 is used for storing a stereoscopic content database 320 and a user database 322. Both are stored in the same local storage here, but each could have its respective storage system.
[00256] An exemplary embodiment of the stereoscopic content database 320 is shown in Figure 18. The stereoscopic content database 320 comprises a set of records 321a, 321b, 321c... each corresponding to different stereoscopic content, in this example to different films. Each record comprises information to be provided to the user end 306 to allow for a selection of a program at the user end 306. This may include a title, a brief description and a thumbnail. In additional, each record comprises an identification the different versions of the stereoscopic content available. More particularly each record comprises an identification of a plurality of versions of the stereoscopic content, each of the plurality of versions being in a different stereoscopic configuration, each stereoscopic configuration corresponding to a different set of viewing parameters. In this example, each version is identified and associated to its respective set of viewing parameters.
[00257] It will be appreciated that stereoscopic content database 320 can be part of a larger content database comprising non-stereoscopic content but for the purpose of this non-limiting example, only stereoscopic content are offered.
[00258] The content management system 302 builds populates the stereoscopic content database 320 via communications with the content storage system 304. In particular it may receive from the content storage system 304 a list of stereoscopic programs contained in the digital content library 308. To this end the communication interface system 318 of the content management system 302 may be in communication with the content storage system 304 (via its communication interface system 312) from which it receives the information with which to build or populate the stereoscopic content database 320, including for each program an identifier of the stereoscopic content. This information may then be placed by the processing logic 316 of the content management system 302 into the records 321a, 321b, 321c, ... of the stereoscopic content database 320. In particular the identifier may comprise an address where the respective stereoscopic content may be accessed, such as an indication of a corresponding location in the digital content library 308. Where different versions of stereoscopic content are stored in the digital content library 308, the content management system 302 may similarly receive an identifier of each version, which may also comprise an address of the respective version of the stereoscopic content such as an indication of a corresponding location in the digital content library 308. Accordingly, each of the records 321a, 321b, 321c, ... may be provided with a location identifier for each of the versions of the respective stereoscopic content.
[00259] In this example, the stereoscopic content database 320 is stored in local storage 314. It may be stored there in a persistent manner, being periodically updated by pushed updates from the content storage system 304 or via queries to the content storage system 304. In such a case, records 321a, 321b, 321c... may contain optional credentials data (shown in brackets) which provide information on the user credentials required to access the stereoscopic content. This credential data may be used to determine which content to present to the user end 306, or which content requests from the user end 306 to honor. Alternatively, however, the stereoscopic content database 320 may be present in the content management system 302 only temporarily, as will be described in the example related to Figure 19.
[00260] In the present example, the digital content library 308 comprises a plurality of versions for each film, corresponding to different types of viewing environments. As mentioned early, a viewing environment can be defined by a large number of viewing parameters, however some viewing parameters can guessed or can be inferred or approximated from others. For example, the IOD is typically considered constant across all viewers, often being estimated to be 65mm. The display resolution may be considered to be a typical resolution such as 1920x1080. Thus the IOD and display resolution may simply be guessed. Alternatively, the display resolution may be inferred from other parameters such as the display dimensions. If the display dimensions indicate a very large display (say, 70" and above) then the resolution may be assumed to be ultra-high-def (4K) resolution while if the display dimensions indicate a typical television-size or laptop/desktop monitor size (say 11 "-69") the resolution may be assumed to be 1920-1080 while for smaller sizes more typical of tablets or smartphones, the resolution may be assumed to be 1280x720. Likewise, certain other viewing parameters may be inferred from other parameters. In particular, it is possible to obtain a reasonable estimate of the viewing distance from the display dimensions. Typically television owners will set up a couch at a certain distance from the TV, which distance tends to be related to the size of the television, bigger TVs being more typical of bigger homes. Thus for very large displays (>70") we may estimate a viewer distance (VD) common in larger rooms, while for more typical TV-sized displays (26"-70") we may estimate another VD more common in typical living rooms, while for laptop/desktop monitor-sized displays (11 "-26") we may estimate a VD typical of a viewer sitting at a desk in front of the display, while for smaller displays (<H") more typical of handheld devices we may estimate a VD representative of the typical distance at which a user holds a handheld device from his eyes.
[00261] Another example of a viewing environment for which the digital content library 308 may have a version of the film is a head-mounted display (HMD) such as the Occulus Rift™. A viewing environment may include a type of display that is an HMD or even a particular HMD, which may include particular display characteristics such as a distance from the eyes of the screen, a position of a screen (or respective left- and right-eye display portions for the HMD), a lenticular effect or type of lens, a distortion required on the 3D image (or, more particularly, individual left- and right-eye subimages), a resolution, a screen (or individual display portion) size, and other parameters. The digital content library 308, may contain reconfigured versions of stereoscopic content for the general HMD display viewing parameter or, preferably, for individual HMD displays or display-types (e.g. Occulus Rift™, Samsung Gear VR™, a particular type of kit with a particular size smartphone, a generic single-screen-split-display type, etc . ).
[00262] Head-mounted display include a display, which may be unitary or split into two parts that may consist of individual display screens, mounted about the head, in front of the eyes, generally in close proximity thereto, that each display for each eye a respective image. In the case of a unitary display, a single display may be used on which the image is split down the middle such that the left side of the display presents an image for the left eye and the right side presents an image for the right eye. An example of such a head-mounted display is the Samsung Gear VR™ which allows a smartphone to be mounted in front of the head of a wearer and used as a such unitary HMD display. Respective left-eye and right-eye displays or display portions may overlap in a head-mounted display, with appropriate screening or multiplexing occurring before each eye to ensure each sees only the image intended for it, for example using polarization or page-flipping techniques used in TV screens or other known display techniques.
[00263] A configuration of stereoscopic content for display on a head-mounted display may include several modifications of the content. The content may be reformatted, such as with the algorithm described herein or using any other appropriate reformatting algorithm to correct the depth-to-width (or depth to another proportion) ratio and/or to avoid over-convergence, divergence or other discomfort-causing effects. To this end, the image tube model may be used. In this example, the viewing parameters may include non-overlapping viewing areas for each eye. Accordingly, a constraint is placed on the reformatted image in that it should not include pixel placements where corresponding left- or right-eye pixels would be located beyond the area of their respective left- or right-eye display area. Using the example of Figure 4, this may be done by placing as a constraint in the target view parameters the determination the dimension and/or location (e.g. relative to the other display) of the display portion (e.g. the right-eye portion) for which a new image will be synthesized. The constraint in one example is a threshold for a value in the image tube defining parallax such that the transformed image tubes cannot define a relationship between the eye and a point that would place the point beyond a certain angle or distance. Alternatively, a feedback loop may be implemented at the view synthesis step whereby if any pixels are found to be placed outside of the area of a respective display portion, an indication of this, and optionally an indication of the offending pixel(s) itself and the amount and direction by which it is off-display is sent back to the image tube transforming module so that it may recalculate a new set of image tubes in an attempt to avoid the same problem or apply correction to the next image it processes. [00264] To display a 3D image on a head-mounted display, a distortion effect may be applied to the image. For example, the Occulus Rift™ applies a barrel distortion to the two subimages (left- and right-eye views of the 3D image/scene) which is then corrected by the pincushion effect created by lenses in the headset. The end result of the distortion and lenses is a spherical- mapped image for each. The distortion may be applied by hardware within or connected to the HMD. For example, a "kit" to transform a smartphone into a HMD may include a crude cardboard frame, lenses and software to cause the smartphone to decode 3D content (e.g. in SBS format) and to apply distortion and display left and right images in respective display areas. However, in certain instances, a device may not include the software or hardware required to apply a desired distortion to a 3D image/stream. As described, a distortion may be desired to cooperate with a lens to create a particular effect (e.g. barrel distortion described above, but other distortions may also be applied for other lenses and/or effects). Alternatively a distortion may be desired simply to create an effect desired for any type of display, although with a HMD, it may be preferable to apply distortions to account for the close proximity of the screen and/or higher visibility of pixels. The application of such a distortion may be performed, using any suitable distortion-creating scheme, such as well-known barrel-distortion techniques, at the reconfiguration stage. As such, a version of 3D content in the digital content library 308 that is for a head-mounted display viewing environment may include an image distortion pre-applied to the image.
[00265] Viewing parameters for a version of content stored in the digital content library 308 may also include a software (or general scheme) used to display content. Certain video players that can be run on HMDs may present a "virtual cinema room" whereby stereoscopic content is presented to the viewer alongside virtual context content. The virtual context content may be, for example, a visual reproduction of a cinema room, with the stereoscopic (e.g. film) content displayed in on a virtual screen in the cinema room. In such a case the viewing parameters may also include virtual parameters including the desired perceived distance (e.g. convergence plane) and size of the stereoscopic display. The viewing parameter may be defined in such terms, or simply by a type of software decoder known to run certain parameters or by a particular known scheme (e.g. "IMAX room" or "virtual living room" or "virtual hotel room") that corresponds to a particular set of viewing parameters. These viewing parameters may be provided as described herein by a user or by the software. Thus the digital content library 308 may include versions of content corresponding to different viewing parameters which may include HMD physical parameters as well as virtual parameters.
[00266] In the present example, the films in the digital content library 308 are pre- reconfigured from an original version to include versions for the typical viewing parameters of handheld-sized displays (<11"), monitor-sized displays (11 "-26"), TV -sized displays (26"-70") and very large TVs (>70"). It is to be understood that other versions corresponding to other configurations are possible, including for entirely different viewing parameters or simply to provide a different gradation of display size (e.g. more/fewer/different size ranges). [00267] Optionally, the stereoscopic content distribution system 300 may include a mean for providing a user with an estimated error or a correction factor. By knowing exactly for which viewing parameters each version of stereoscopic content has been optimize, and knowing at least a portion of the viewing parameters of a user, the content management system 302 may provide an estimate error in terms of depth fidelity or a suggested correction factor in terms of other viewing parameters. For example, the content management system 302 may compute exactly how well or poorly the reformatted version of the stereoscopic will be faithful in depth if the viewing parameter (e.g. display size) at the user end differs from the one the version was ideally created for (e.g. if the 26"-70" range version was actually created for a 50" TV and the viewing environment includes 26" TV). It may provide this in terms of a depth-fidelity score. Alternatively, or additionally, the content management system 302 may compute an ideal other viewing parameter (e.g. viewing distance) with which to view the content in the particular viewing environment (e.g. on the particular viewing display) of the user or in the intended ideal viewing environment (e.g. on the 50" display the version was configured for) and may provide this information to the viewer for ideal viewing.
[00268] The user database 322 comprises information on all users/customers of the stereoscopic content distribution system 300. For each user this information may include unique user account including a unique user identifier, credential information which may include payment information (such as a credit remaining and or information on the progress of a present payment) and/or subscription information (such as a level of subscription to unlimited access or a number of content accesses remaining under the current subscription).
[00269] Some of the user information may be present only temporarily in the content management system 302. For example the content management system 302 may optionally communicate with a payment authority to authenticate a payment from a remote user to rent or purchase stereoscopic data using either payment information (e.g. credit card details) stored as part of the user information in the user database or received directly from the user end 306. In such a case, payment information (such as a confirmation of a payment) may be received from the payment authority and used by the processing logic 316 to authenticate a transaction and not stored persistently in the local storage 314.
[00270] The user database 322 may also comprise for each user digital viewing parameter data indicative of at least one viewing parameter from a set of user viewing parameter defining a remote viewing environment in which stereoscopic content provided by the stereoscopic content distribution system 300 is to be viewed. This digital viewing parameter data is typically provided from the user end 306 as described herein. In this example, the digital viewing parameter data is provided by the user end by a secondary (separate) device 324 via a registration system 326 and is stored as part of the user information 322 in the user database 322. But as will be described further herein, for example in the discussion pertaining to the embodiment illustrated in Figure 19, the digital viewing parameter data does not need to be persistently stored in the user database 322 and can be obtained on a per-content-request basis. Alternatively still, the user database 322 may contain a user preference, which may serve as default unless a digital viewing parameter data indicative of another set of viewing parameters is received upon request of stereoscopic content.
[00271] The content management system 302 also comprises a communication interface system 318. The communication interface system 318 is adapted for all external communications of the content management system 302. In this example, the communication interface system 318 is for communicating with the user end 306, the content storage system 304, and a registration system 326. To this, end, the communication interface system may comprise separate modules for communicating with each of these entities or it may do so over a single point of entry/exit, such as a high bandwidth connection to the internet.
[00272] The content management system 302 further comprises processing logic 316. The processing logic 316 can be dedicated hardware with hard- or firm-coded instructions such as an FPGA, but is more likely a general-purpose processor controlled by software instructions tangibly stored in a computer-readable storage medium such as local storage 314 instructing the general-purpose processor to behave as described herein. Communication between the processing logic 316, the local storage 315 and the communication interface system 318 can be done by any suitable manner but in the present example they are linked by internal buses. This is similarly the case in the content storage system 304 and the content access system 328.
[00273] Given the increasing distribution of processing and storage resources it is also conceivable that the content management system 302 be implemented by a distributed network of processing resources accessing a distributed network of storage resources (e.g. a cloud) instead of the local storage 314.
[00274] In accordance with a non-limiting embodiment, the content management system 302 may implement a method for managing access to viewable stereoscopic content in a digital content library by a remote user application (described in more details further below) for viewing at a remote viewing environment characterized by a set of user viewing parameters. In order to do so, the processing logic 316 is configured to access the records of stereoscopic content in the stereoscopic content database 320 and determines the presence in the digital content library 308 of a plurality of versions of the stereoscopic content each having a respective stereoscopic configuration corresponding to respective sets of viewing parameters.
[00275] The communication interface system 318, being adapted to communicate with the remote user application receives from the user application digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters.
[00276] In this example, the digital viewing parameter data are received from the user end 306 via a registration system 326, which will be described in more detail below. In this example, the digital viewing parameter data is indicative of a display size provided during registration.
[00277] The process of registration will now be described before returning to the description of the content management system 302. In this example, as is not uncommon with Video-on- Demand (VOD) systems, registration for the stereoscopic content distribution system 300 (or more particularly, the VOD services) is done via a secondary device 324 separate from the device used to access the content. Many VOD services like 3DGO!™ allow access to video content directly on a smart TV. However, in order to register for the service, a secondary device 324, separate from the TV, may be used to provide registration information. Some VOD services, such as Netflix™ are multi-platform services which allow viewing on several devices. In such a case, the secondary device may actually also be used as a content access device implementing a remote user application / content access system 328, but for the purpose of this example, in order to better illustrate the registration system 326, the secondary device will simply be considered a separate device from the one implementing the remote user application. Also for the purpose of describing the registration system 326, it will be assumed that the content access system 328 implementing the remote user application is in a smart TV, although as will be clear in the description of the content access system 328, this does not need to be the case and should not be construed as limiting.
[00278] In this example, registration to VOD service is done via the world wide web (web). To this end, a user wishing to access the stereoscopic content in the digital content library 308 on his smart TV must first register online using a web browser. To allow registration a registration system 326 is provided. In this example, the registration system 326 is a web server. The skilled addressee will understand all the variations possible in this regard. The registration system 326 is in communication with the content management system 302 and the secondary device 324 via a communication interface system 332 typical of a web server. The registration system 326 further comprises processing logic 334 in the form of a processor acting upon instructions stored tangibly in a computer-readable storage medium, such as the storage 336, that instruct the processing logic to perform the tasks described herein thereby making is to configured. The registration system 326 further comprises website and form data 338 in the storage 336, which may be local storage as shown or otherwise distributed according to any suitable storage system.
[00279] The communication interface 332 being suitable for a website host is capable of bidirectional communication with the secondary device 324 which is a remote user device. In a first step to registration, the registration system 324 established bidirectional communication with the secondary device 324. This may be done in the classic way when a user at the secondary device 324 directs a web browser running on the secondary device 324 to an address designating the registration system 324.
[00280] Once communication is established, the processing logic 334 causes the transmission of the registration website to the secondary device 324. To this end, the processing logic accesses the storage 336 to extract the website data which includes form data 338 and causes its transmission to the secondary device 324 using the communication interface system 332. The form data 338 comprises queries prompting the inputting of registration information by the user of the secondary device 324.
[00281] The registration information entered at the secondary device 324 may include unique or semi-unique identifiers such as a name, address. To this end, the queries may include text boxes for entering such data. The registration information may also include hardware information identifying the hardware on which the stereoscopic content will be viewed or on which the content access system 328 will run, or a software platform on which it will run. For such input a text box or roll-down menus may be used.
[00282] In this example, the registration information include digital viewing parameter data indicative of at least one viewing parameter characterizing the remote viewing environment at which the stereoscopic content provided by the stereoscopic content distribution system 300 will be viewed. Any form of digital viewing parameter data may be provided but in this example, as discussed above, the digital viewing parameter data is indicative of a dimension of the display on which stereoscopic content will be viewed. In particular, the registration form contains a query prompting a user of the secondary device 324 to enter in a text box, or to select from a roll-down menu, a diagonal length of the display.
[00283] Upon entering the registration information by the user of the secondary device 324, the registration information, which comprises the digital viewing parameter data, is sent to the registration system 326 where it is received by the communication interface system 332 and processed by the processing logic 334 which causes the registration information to be associated with a unique user account in the user database 322 at the content management system 302, where the unique user account also comprises the digital viewing parameter data, or at least information indicative of the user parameter(s) indicated in the digital viewing parameter data, such that the content management system 302 can select on the basis of the digital viewing parameter data the version of the stereoscopic content when such stereoscopic content is requested from the user end 306 on the basis of the user viewing parameters.
[00284] To this end the registration system 326 is in communication with the content management system 302 via the communication interface system 332. In this example, the processing logic 334 causes the association of the registration information with a unique user account at the content management system 302 by transmitting the registration information comprising the digital viewing parameter data together to the content management system 302 for association thereat by the content management system 302. Thus the user account may be generated by the content management system 302 and associated to the registration information by the content management system 302 upon prompting by the registration system 326 by being provided with fresh registration information. In another embodiment, however, the processing logic 334 is configured to generate a unique user account and associating it to the registration information comprising the digital viewing parameter data and providing the unique user account details to the content management system 302 via communication interface systems 332 and 318 for storage in the user database 322 at the content management system 302. Creation of a unique user account may include generating an entry as described above for the user database 322.
[00285] The registration information may optionally include payment information, such as credit card details. In this particular example, the registration form includes prompts for entering payment information including text boxes for a credit card number, name and CCD, roll down menus for the expiry month and year, and radio buttons to select a type of card. In this example, the payment information is not sent to the content management system 302, but rather sent directly from the registration system 326 to a payment authority server which processes the information. The payment information may be used for actual payment, or for verifying the uniqueness/authenticity of a registration request or both. If the user is registering for a fixed-fee service (e.g. with a monthly fee), the payment authority processes the payment and provides a confirmation to the registration system 326 and/or the content management system 302. The registration system 326 may forward the confirmation alongside the registration information or user account to the content management system 302. In the case of ad-hoc payments, such as for a rental service, the payment information may for example merely serve as a unique identifier to which to tie the user account, or it may be kept on file by the payment authority for future purchases/rentals. In such a case the payment authority may link the payment information to a unique payment information identifier and return this to the content management system 302 or registration system 326 which then treats it similarly to the aforementioned confirmation.
[00286] Alternatively, payment information may be simply stored and associated to the unique user account, similarly to the digital viewing parameter data, to be communicated to the payment authority to process a payment as needed by the content management system 302. In such a case, the payment information is provided to the content management system 302 by the registration system in the same manner as the rest of the registration information.
[00287] As will be discussed in more details further below, for example in the discussion pertaining to Figure 19, the digital viewing parameter data may include information indicative of a version of stereoscopic content.
[00288] Once the registration is complete and a unique user account has been created and is associated with digital viewing parameter data, the digital viewing parameter data entered at registration will not necessarily dictate the version employed every time the stereoscopic content is obtained over the stereoscopic content distribution system 300. For example, the digital viewing parameter data entered at registration may be indicative of a user preference, which serves by default for the content management system 302 to select a version but can be overridden by providing from the remote user application along with a program selection another digital viewing parameter data in the form of a actual viewing parameter(s) or a version selection.
[00289] It should be noted that the user of the secondary device 324 is not necessarily the user of the content access system 328 or indeed the viewer associated with user viewing parameters provided in the digital viewing parameter data. The person registering for VOD services is not necessarily the one that will be watching the stereoscopic content every time or ever. The user viewing parameters are associated to him insofar as he is entering them into the registration process but calling them associated to him should not imply that they necessarily apply to him. He could be registering for the content access for himself or for his family or for someone else.
[00290] It should also be noted that the form needs not be presented in a single instance (single web page) but may be provided in multiple segments or portions, for example in a page- by -page sequence with each page comprising a different portion of the form.
[00291] Optionally, the content management system 302 receives a request from the remote user application for a particular stereoscopic content. In this example, the remote user application is implemented on a content access system 328 which may control access to, e.g., VOD content on a device such as a TV, a computer, a HMD, or a handheld mobile device. The remote user application first logs into the system. This may be done by sending a log-in message providing user identification. In response to the log-in or to a separate request therefor, the processing logic 316 of the content management system 302 accesses the records of the stereoscopic content in the stereoscopic content database 320 and transmits the list of available stereoscopic content to the remote user application. In order to do so, the processing logic may simply provide the complete list of available stereoscopic content (if, for example, all the films are available for rental from anyone) or it may first access the user database 322 to verify user credentials and provide only the list of films accessible to the user account associated with the remote user application. The skilled addressee will understand that such communication may be performed by an application system like that of Figure 16 and may be done in several steps. For example the list of available title may be presented to a user via the remote user application in successive panels each comprising a different category of programs and may therefore be provided portion- by -portion as the user browses the catalogue.
[00292] In any event, in this example the processing logic 316 of the content management system 302 accesses the stereoscopic content database 320 and sends using the communication interface system 318 to the remote user application with a selection of programs from which to select. At the user 306, a selection of a title is made and transmitted to the content management system 302. In particular, a communication interface system 330 in the content access system 328 is made to transmit an indication of a selected program to the content management system 302, which is received at the communication interface system 318 and processed by the processing logic 316 which uses this indication (e.g. and identifier) to identify the selected program.
[00293] Having identified the selected program, the processing logic then accesses the user database 322 to obtain the digital viewing parameter data and selects on the basis of the digital viewing parameter data a version of the selected stereoscopic content to be transmitted to the remote user application. In particular, the processing logic 316 selects a version of the stereoscopic content from among the plurality of versions of the stereoscopic content listed in the corresponding record in the stereoscopic content database 320 by identifying a set of viewing parameters from among the respective sets of viewing parameters provided in the corresponding record that best corresponds to the user viewing parameters identified in the digital viewing parameter data. Selecting a version may simply entail looking up which version corresponds to the exact user viewing parameter provided, if the user is only permitted to provide viewing parameters corresponding to existing versions of the stereoscopic content. In this particular example, the user provided an exact display size (in terms of diagonal length) during registration but the digital content library 308 only contains four versions of each stereoscopic program corresponding to different display size range. Finding the best version therefore entails finding the display size range in which the user's digital viewing parameter data falls. Other methods could include finding the size closest to the one in the digital viewing parameter data if each version corresponds to a specific size. If multiple viewing parameters are present, a multidimensional closest approximation can be found or a hierarchical layered approach may be used (e.g. find the closest IOD, then the closes display size, then the closets VD).
[00294] Once the processing logic 316 has selected a version of the stereoscopic content to provide to the user end 306, the processing logic 316 provokes the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application. In this example, the processing logic 316 does this by sending an instruction to the content storage system 304 via communication interface systems 318 and 312 to transmit the selected stereoscopic content in the selected version from the digital content library 308 to the user end 306 and particularly to the remote user application. The instruction includes the necessary information for the content storage system 304 to know where to send the data, for example it may contain an address for the remote user application or the information to facilitate a handshaking between the content storage system 304 and the content access system 328.
[00295] In another embodiment, the processing logic 316 may provoke the transmission of the stereoscopic content in the selected version from the digital content library 308 to the user end 306 and more particularly the remote user application by providing the remote user application with a token with which to access the selected stereoscopic content in the selected version. The token may be a decryption key, whereby the remote user application can request and access any content from the content storage system 304 but will be only able to decrypt the content and version corresponding to the received decryption key, or the token may be an authorization token attesting to authorization from the content management system 302 to access the selected stereoscopic content in the selected version (and perhaps identifying it). Upon receiving the authorization token, the remote user application transmits it to the content storage system 304 via communication interface systems 330 and 312 whereupon when the control 310 of the content storage system 304 receives the authorization token, it determines that the remote user application is indeed authorized to access the selected stereoscopic content in the selected version and transmits it.
[00296] In this particular example, the transmission of the stereoscopic content in the selected version is done from the content storage system 304 to the remote user application by streaming as is common in Video-on-Demand (V OD) systems. However, it will be appreciated, that in the case of a purchase, the transmission could be a file download from the remote user application. This could even be a file download in the case of a rental provided that the file and/or remote user application is protected by suitable mechanisms to prevent viewing of the content outside of the rental period/parameters. [00297] As briefly described in the discussion pertaining to the registration system 326, hardware information identifying the hardware on which the stereoscopic content will be viewed or on which the content access system 328 will run, or a software information identifying a software platform on which it will run can be provided to the content management system 302. In certain embodiment, such information can be used as digital viewing parameter data. Indeed, hardware information may be indicative of at least one viewing parameter. For example, if the hardware information includes a model number for a television or other device, it is possible to ascertain the size of an associated display. To this end, the content management system 302 (or registration system 326) may have access to a lookup table (stored locally or remotely, e.g. over the internet) through which a model number may be translated to a display size for use in selecting a version of stereoscopic content as described above. In such a case, the digital viewing parameter data received from the remote user application may simply be the hardware identifier, which may have been provided to the remote user application by its user, but which in the case of a remote user application linked to the display device on which the stereoscopic content will be viewed (such as a VOD app running on a smart TV) may simply be automatically sent to the content management system 302 without requiring input from a user. In yet another embodiment the digital viewing parameter data could be a software identifier indicative of a software platform insofar as a software platform may be indicative of at least one user viewing parameter such as a display size. This may be the case where different TVs are running different versions of a remote user application or where the OS upon which the platform is running is indicative of an approximate screen size (e.g. android = tablet/phone size, windows = monitor size and television OS = TV size). Here too the digital viewing parameter data may be provided automatically by the remote user application without input from a user.
[00298] In the above example, the content storage system 304 transmits the requested stereoscopic content in the selected version directly to the user end 306. It should be understood that according to a different architecture, transmission of the stereoscopic content could pass through the content management system 302. In particular, in provoking the transmission of the stereoscopic content in the selected version from the digital content library 308 to the remote user application, the processing logic 316 could simply request the content from the content storage system 304, wherein the control 310 of the content storage system 304 is configured to unquestionably honor any such request from the content management system 302 and wherein the processing logic 316 of the content management system 302, upon receipt of the content from the content storage system 304 (via communication interface systems 318 and 312) transfers the selected stereoscopic content in the selected format to the user end 306 and more particularly to the remote user application via communication interface system 318. In yet another embodiment, the stereoscopic content contained in the digital content library 308 is only contained in a single version (typically the original version) and is reconfigured at the content management system 302 by a reconfiguration module. An example of such real-time reconfiguration will be provided further below. [00299] The user end 306, as shown in Figure 17 comprises a content access system 328 and, in this example a secondary device 324 for registration that is conceptually considered at the user end 306 but is not necessarily physically located at or near the control access system or viewing environment.
[00300] The content access system 328 comprises a communication interface system 330, processing logic 340 and local storage 342. Once again it will be appreciated that although the storage in this example is local, in other embodiments it could be physically located elsewhere, such as on a storage cloud.
[00301] The content access system 328 implements a remote user application. In this example, the content access system 328 is located on a smart TV, whereby the communication interface system 330 comprises the smart TV's WiFi interface and wired network interface. The processing logic 340 can be dedicated hardware with hard- or firm-coded instructions such as an FPGA, but is more likely a general-purpose processor controlled by software instructions tangibly stored in a computer-readable storage medium such as local storage 342 instructing the general-purpose processor to behave as described herein. In this particular example, the processing logic is contained within a system-on-a-chip (SOC) contained in the smart TV. The processing logic is linked to the communication interface system 330 by an appropriate logic path such as a bus. The local storage 342 is flash memory accessible by the processing logic 340 via a flash interface (not shown).
[00302] In this particular example, the content access system 328 includes a display 344, which is the display of the smart TV. It is shown in the figure as optional since the content access system 328 which controls access to the stereoscopic content offered by the stereoscopic content distribution system 300 is not necessarily on the same device as will display the stereoscopic content. Indeed, VOD applications may run on devices other than TVs, computers and handheld communication devices. Set-top boxes, Blu-ray™ players and Apple TV-type devices may be used to access the stereoscopic content in the digital content library 308 despite not having a display. Moreover even the presence of a display does not necessarily mean that the stereoscopic content is to be viewed on the device embodying the content access system 328. Indeed, new technologies such as Google™ Chromecast allows control of content streaming to a TV (or other such device) using another connectivity device such as a phone or computer, which itself also has a display capable of displaying video. In the case of Google™ Chromecast or similar devices, content access may be controlled using a first device (say, a mobile phone) embodying the content access system 328 while the stereoscopic content is streamed to a second device (say a television), yet the stereoscopic content may still be said to be transmitted to the content access system 328 and is received by the content access system 328 insofar as some information on the content such as video progress information is sent to the content access system 328 and that the transmission of content is effectively under control of the content access system 328.
[00303] The display 344, which may or may not be a part of the content access system 328, is a part of the viewing environment as illustrated in Figure 16.
[00304] As mentioned above, the content access system 328 implements a remote user application. In this example the remote user application takes the form of a smart TV app stored as program instructions on a computer-readable storage medium, here a flash memory, configuring the processing logic 340, here a SOC's processor via instructions to perform as described herein. The processing logic 340 is in this manner configured to communicate via the communication interfaces 330 and 318 with the content management system 302 in order to manage access to stereoscopic content by a viewer for viewing on the display 344.
[00305] The remote user application may be operated by a user that provides user input using an input device. Any suitable input device for interacting with the content access system 328 and provide the remote user application with input may be used. These include keyboard and mouse for computers and touchscreens with appropriate manipulateable interface elements for handheld devices but in this particular example the user input device is a remote control which interacts with the smart TV via an infrared port.
[00306] In this non-limiting embodiment, the remote user application interacts with the user by receiving input from the user input device and providing information over the display 344. It should be understood, as per the above discussion, that it is not necessary that the display of the content access system 328 be the same display that will be used for displaying the stereoscopic content but in this case it is. A graphical user interface enable the functionality of the remote user application to be provided to a user.
[00307] Figures 6a and 6b provide an example of a graphical user interface 600 according to a non-limiting embodiment. As shown, the graphical user interface 600 comprises a first pane 602 and a second pane 610 that do not, in this case, overlap. The boundary 612 between the first and second panes 602 and 610 drawn out in this example but this does not need to be the case. When a user of the smart TV starts the remote user application by taking the appropriate action such as selecting it in a list of smart TV apps, the instructions for controlling the processing logic to implement the remote user application are loaded from the flash memory into DRAM (or, alternatively SRAM) memory for execution. The programing logic, connected in this example to the display 344 by an LVDS link via a T-con board then produces the graphical user interface 600 n the display 344.
[00308] The first pane 602 displays a plurality of first visual elements 604, each of which represent a category of stereoscopic program, in this case categories of movies. The visual elements in this case are textual icons indicating a category. For each visual element, an input element associated thereto is operable by using a user input device. In this case, the input element is the icon itself, which can be selected by navigating on it with arrows on the remote control (which shift between selected icons in the first pane 602 by pressing up or down and shift in and out of the first pane 602 by pressing left or right) and activated by pressing an "enter" key thus operating ("clicking") the input element.
[00309] Operating a first input element causes the selection of that category and the display in the second pane 610 of a plurality of second visual elements, each being representative of a particular stereoscopic program in the category, in this case movies. It should be noted that it is not necessary in all cases for a category icon to be clicked to have the category contents displayed in the second pane, a category pane may be open by default. In order to populate the graphical user interface 600, the processing logic receives from the content management system 302 a list of programs available to it (e.g. for rental or under a current subscription or both) in the digital content library 308. To this end, the processing logic 340 sends a request to the content management system 302 for the list of programs available via the communication interfaces 330 and 318, and the content management system 302 accesses the stereoscopic content database 320, optionally applying user credentials obtained from the user database 322 to compile the list and sends it back via the same path to the content access system 328.
[00310] As shown in Figure 20a, the second pane 610 displays a plurality of second visual elements 614 each being representative of a stereoscopic program in the category. Here the second visual elements 614 are movie titles. The second pane also comprises for each second visual element a second input element 615 being operable by a user using the input device to select the program. In this case the input elements 615 are not overlapping the visual elements 614 but are image icons, albeit visual as well, located above the title. Clicking in the same manner as described above on such an icon selects the corresponding movie and brings up the third pane 618 which replaces the second pane 610 while, in this example, the first pane 602 remains visible.
[00311] The third pane comprises a visual element displaying textual information about the particular selected stereoscopic program. In this case there are several visual elements including a larger version of the icon image in the second pane 610, and a textual title, description and price. This information may be requested by the content access system 328 from the content management system 302 upon selection of the film. There is also a third input element associated with the particular stereoscopic program, the selected film, operable to select the program for stereoscopic viewing. In this example the third input element is a big "rent" button operable by clicking as described above. Other further confirmation and payment screens may be additionally provided as desired. In this particular example, there is also a version visual element 622 indicating the availability of a plurality of versions of stereoscopic content, each of which corresponds to different configurations as described above. In this case the version visual element 622 indicates the presence of a child-safe mode. The graphical user interface 600 further includes a version input element operable by the user using the input device 624 to select a version from amongst the plurality of available version. Since in this example there is only two version, the original and child-safe version a check box indicating a desire to view the stereoscopic content in the child-safe configuration is operable by selecting it with directional arrows on a remote control and clicking it with the "enter" button.
[00312] During the playing of the stereoscopic content on the display 344, the graphical user interface 600 may also include a progress a control system, such as a progress bar and control buttons. Control buttons may include buttons for selecting a version (e.g. original and child-safe) or toggle a version (child-safe on or off) directly on the progress and control system, which may allow switching between version (which may involve switching between streaming files) in realtime.
[00313] Instead of being presented in the third pane 618, the version visual element 622 could be presented in a fourth pane such as a pop-up pane in response to actuation of the third input element 620. The forth pane may include the version input element 624. Other modes for presenting available version may be used, such a list of representations (e.g. textual names/descriptions) of selectable versions which can be browsed and selected using the user input device. Alternatively, the graphical user interface 600 may provide input prompts for entering viewing parameters which will be sued as describe above to select a version. This can include text boxes which can be selected by "clicking" as described above, and into which text can be written by known textual entry methods using the remote control, e.g. the numeric keypad. Textual fields may include the age of a viewer, a viewing distance, dimensions/size of the display or a plurality of these.
[00314] It should be understood that the graphical user interface 600 can be split between the content access system 328 and another device as when a smartphone is used to control what is being displayed on a TV using Google™ Chromecast. In this case, all graphical user interface 600 functions including those for content selection and playback control are provided on the smartphone display which is separate from the TV's display but in other embodiments the graphical user interface 600 could be split between two display.
[00315] Returning to the embodiment of Figure 17, the content access system 328 now has access to a list of content available in the digital content library 308 and has means of receiving an input for selection of a particular program to be viewed on the display 344. In response to the selection of a particular stereoscopic program by a user, the processing logic causes the transmission of a request for the particular stereoscopic content to the content management system 302. It does so by formulating the request and sending commands to the communication interface system 330 to transmit it to the address of the content management system 302 (and more particularly the communication interface 318 of the content management system 302). The content management system 302 receives the request at the communication interface 318 and the processing logic 316 of the content management system 302 operating as described above ascertains the presence of multiple versions of the requested stereoscopic content in the digital content library 308 by consulting the stereoscopic content database 320.
[00316] Optionally, the content access system 328 also causes the transmission using the communication interface system 330 to the content management system 302 of digital viewing parameter data indicative of at least one of the viewing parameters from the set of viewing parameters that will define the viewing environment at which the stereoscopic content will be viewed. In the present example, data indicative of viewing parameters were provided to the content management system 302 via a registration system 326 by a user operating a secondary device 324. But the digital viewing parameter data could also be provided directly by the content access system 328. As mentioned above, this could be additionally to the transmission of digital viewing parameter data at registration, (e.g. in the case where the digital viewing parameter data provided at registration is merely a preference to be used as default) e.g. by changing viewing parameters in a settings section of the remote user application (implemented by the graphical user interface 600) or by signaling viewing parameters other than those provided at registration, e.g. by selecting a particular version (e.g. selecting a child-safe version when the digital viewing parameter data provided at registration did not specify a child IOD) or by inputting when given the opportunity by the graphical user interface 600 a new digital viewing parameter data that is to override the default one.
[00317] In one alternative embodiment, however, the digital viewing parameter data are not provided at registration but rather are provided by the content access system 328 only. In the description of the graphical user interface 600, an example was given whereby the graphical user interface 600 offers a choice of a child-safe version. In that example, the content access system 328 receives knowledge of the versions available for a selected stereoscopic content from the content management system 302 in order to present them to the user. In this example, transmission of the request for content precedes the transmission by the content access system 328 of the digital viewing parameter data.
[00318] The reader will also note that in this case the digital viewing parameter data takes the form of the selection of a version. Indeed the selection a version of stereoscopic content that is associated with certain viewing parameters acts as an indication of user viewing parameters corresponding at least in part to, or being closest to those of, the selected version and therefore an indication of the selection, when transmitted from the content access system 328 to the content management system 302 serves as digital viewing parameter data. This kind of digital viewing parameter data is called version-association representation whereas when the digital viewing parameter data refers directly to the value (exact, or indication thereof such as a range or approximation) of an actual viewing parameter (e.g. dimension of display, VD, IOD or resolution) this is called a direct representation. Thus the digital viewing parameter data may include a configuration identifier indication a particular version to be selected from among a set of versions.
[00319] Selection of a version of the stereoscopic content may left up to the content access system 328, rather than the content management system 302 to determine. For example, Indeed, the set of versions of the stereoscopic content may be received at the communication interface system 328 (e.g. being transmitted by the content management system 302 where it was derived by the processing logic 316 by consulting the stereoscopic content database 320) and the processing logic 340 may be configured to identify from among the set of versions a selected version of the stereoscopic content that has a configuration corresponding to the viewing parameters that best correspond to the user viewing parameters. In this case the indicator of the selected version is indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version.
[00320] To derive direct-representation digital viewing parameter data, the remote user application may provide to the user, via graphical user interface 600 on a viewing device such as the display 344 of a visual prompt requesting a user to enter at least one user viewing parameter and wherein in response to receiving the at least one user viewing parameter the processing logic 340 generates the digital viewing parameter data on the basis of the user viewing parameter inputted by the user. The digital viewing parameter data may include the exact data (e.g. size of display) entered by the user or may include merely a representation of it (e.g. by providing the range into which the size entered by the user falls).
[00321] Alternatively still, the processing logic 340 may generate the digital viewing parameter data without user input. As mentioned above, the digital viewing parameter data may include or be derived from a hardware or software identifier. Such an identifier can be hard- coded or hard-wired into the processing logic 340 thus eliminating the need for user input.
[00322] The processing logic causes the transmission of the digital viewing parameter data to the content management system 302 by generating the digital viewing parameter data and instructing the communication interface system to transmit it to the address of the content management system 302 (and more particularly to the communication interface system 318 of the content management system 302).
[00323] In response to the transmissions of the request for stereoscopic content and of the digital viewing parameter data (not necessarily in that order), the content management system 302 selects a version of the requested stereoscopic content in a particular configuration on the basis of the digital viewing parameter data and causes its transmission to the content access system 328 as described herein. The content access system 328 receives at the communication interface 330 the requested stereoscopic content in the particular configuration and causes it to be displayed on the display 344.
[00324] As show in Figure 17 and discussed above, the communication interface system 330 may be in communication with the content storage system 304 for receiving stereoscopic content, for example in streaming form. The requested stereoscopic content in the particular configuration may be received directly from the content storage system 304 in this embodiment. (In another embodiment, discussed further below, the stereoscopic content in the particular configuration may alternatively be also received from the content management system 302.) [00325] Where the stereoscopic content is received directly from the content storage system 304, it may be in streaming form (e.g. for movie rentals) or in a file download form (e.g. for movie purchases). In either case, the content transfer may be initiated by the content storage system 304 upon receiving instructions from the content management system 302 as described above, or it may be initiated by the content management system 302 by setting up a handshaking procedure between the content storage system 304 and content access system 328. But in one embodiment alluded to above in the discussion pertaining to the content management system 302, the content access system 328 receives from the content management system 302, in response to the transmission of a request for stereoscopic content and a digital viewing parameter data, an authorization token. Using the authorization token, the processing logic 340 generates a request to the content storage server for the stereoscopic content the request comprising the authorization token and causes the communication interface system 330 to transmit the request to the content storage system 304 (and more particularly to the communication interface system 312 of the content storage system 304). The stereoscopic content in the version selected by the content management system 302 is then transmitted from the content storage system 304 to the content access system 328.
[00326] Figure 19 shows a process/data flow according to another non-limiting embodiment. In this embodiment the content management system 302, content storage system 304 and user end 306 are similar to those described above and shown in Figure 17 with the exception that the registration system 326 is absent because the service registration is performed directly using the content access system 328 instead of a secondary device 324. While the registration process is not shown in Figure 19, it is to be understood that the process is similar to that described in relation to the registration system 326 but the registration information is gathered on the content access system 328 instead of the secondary device 324 and is transmitted directly from the content access system 328 to the content management system 302 (via communication interface systems 330 and 318) instead of through a registration system. For example, using the analogy of the example of Figure 17, a user may register for VOD services by entering registration information directly into his smart TV using the remote user application (e.g. VOD application) in his smart TV. To this end, the remote user application may present to the user prompts similar to those contained in the form data 338 of the registration system 326 using the graphical user interface 600, and may receive the registration information from a user entering it using a user input device functioning as described above.
[00327] In the example, a user first starts the remote user application by, for example, opening the VOD application on a smart TV. From the remote user application, a log-in message is sent to the content management system 302. The processing logic 340 of the content access system 328 generates the log-in message using login information stored in local storage 342 or inputted by the user and causes the communication interface system 330 to transmit it to the content management system 302. It may also send a request for a content list, e.g. as described in relation with the description of the graphical user interface 600, or the log-in message itself may serve to prompt the content management system 302 to send the content access system 328 a content list.
[00328] In response, the content management system 302 identifies the user and the user credentials. The processing logic 316 of the content management system 302 uses the login information to identify a corresponding user account in the user database 322 and corresponding user credential.
[00329] In this example the content management system 302 comprises and stereoscopic content database 320 only temporarily, and does not store it long-term in the local storage 314. Nonetheless, it does obtain temporarily the stereoscopic content database 320 in order to provide the remote user application a list of stereoscopic content available. In this example, the stereoscopic content database 320 is essentially the whole list of stereoscopic content available to the remote user application.
[00330] This works as follows: upon identifying the user credentials, the content management system 302 queries the digital content library 308 based on those credentials. In particular the processing logic 316 of the content management system 302 generates a query message 506 for the control 310 of the content storage system 304 and instructs the communication interface system 318 to transmit the message to the content storage system 304 and more particularly to the communication interface system 312 of the content storage system 304. The query message includes information on the user credentials and requests records of all the stereoscopic content in the digital content library 308 that satisfies the credentials.
[00331] In response, the control 310 of the content storage system 304 compiles an stereoscopic content database 320 comprising all the stereoscopic content in the digital content library 308 satisfying the user credentials and returns it in message 508 to the content management system 302 via the communication interface systems 312 and 318 Once received, the processing logic may modify the information in the stereoscopic content database 320 to generate a list 510 to transmit to the remote user application or, if the stereoscopic content database 320 is already suitably formatted, it may transmit the stereoscopic content database 320 as the list directly.
[00332] The reader will appreciate that a stereoscopic content database 320 comprising records of all stereoscopic content in the digital content library 308 could be stored by the content management system 302 and that the processing logic 316 can compile the list of stereoscopic content available for the remote user application by consulting the stereoscopic content database 320 using the user credentials to identify content that should be included in the list.
[00333] Having received the list 510, the remote user application presents a choice of stereoscopic content to a user, for example as described in the discussion relating to Figures 6a and 6b. A user selection is received, also for example as described in the discussion relating to Figures 6a and 6B, and the selection, or an indication of the selected stereoscopic content is transmitted by the remote user application to the content management system 302. In particular the processing logic 340 of the content access system 328 generates a selection identification message 512 and instructs the communication interface system 330 of the content access system 328 to transmit it to the content management system 302 and more particularly to the communication interface system 318 of the content management system 302.
[00334] Now in this example, user viewing parameter data has not been provided at registration nor has it been provided before the selection of stereoscopic content. Instead, the content management system 302 requests viewing parameter data in response to receiving the request for stereoscopic content.
[00335] In particular, in this example the viewing parameter data is requested in the form of a selection of a version. Upon receiving the selection identification message 512, the content management system 302 identifies available version of the selected stereoscopic content (for example the processing logic queries the stereoscopic content database 320 kept since it was received in 508 or the content management system 302 sends another query (not shown) to content storage system 304 to identify available versions). Each version corresponds to difference configurations adapted to different viewing parameters.
[00336] In message 514, the content management system 302 sends to the content access system 328 a list of the versions of the requested stereoscopic content that are available. Upon receiving the message 514, the remote user application obtains presents to the user via graphical user interface 600 a representation of the versions available and receives from the user via the user input device a selection of a version. The selected version is considered indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version and may accordingly be consider digital viewing parameter data although even though it may be called a version-association representation of a viewing parameter.
[00337] In this example, the list of versions available may be, as represented in Figure 20b, an adult (or original) version and a child-safe version. In this example, each film is offered in an original version and a child-safe version the child-safe version being a reconfigured version of the original to adapt to a viewing environment where the viewer interocular distance is a child interocular distance. Since interocular distance is one of the parameters for which capture/synthesis parameters are typically adjusted most stereoscopic content is configured for, inter alia, a particular interocular distance. This is typically 65 mm or thereabouts. Indeed the typical interocular distance for adults is around 54-68mm. However, the typical interocular distance for children is much smaller, around 41-55 mm. This means that most stereoscopic content is ill-configured for a viewing environment where a child is the viewer. As a result, the depth perception may be skewed and even exaggerated leading to discomfort and forced over- or under-convergence. It is believed that forcing the eyes to focus beyond the usual range of convergence angles may be bad for vision. In young children who are still growing and whose anatomy is still developing in particular, it is feared that such strained focusing may lead to abnormal development of the eyes and eye muscles and cause vision problems on the long term. The development risk to children has led manufacturers of 3D equipment of all kinds (televisions, handheld game consoles, etc.. ) to recommend limits to exposure for children and even avoidance of 3D altogether for children below a certain age. However these restrictions reduces the demand for 3D content for children and even if dutifully followed, still allow children to be exposed to ill-configured and potentially harmful 3D viewing, albeit for a shorter period of time, the long-term consequences of which are not yet fully known.
[00338] In this example, each stereoscopic film in the digital content library 308 comprises an original version and a child-safe version reformatted from the original version to account for a smaller IOD. In the stereoscopic content database 320, both versions are identified, the one as an adult version, which indicates as a viewing parameter an adult IOD, and the other as a child-safe version, which indicates as a viewing parameter a child IOD.
[00339] The selection of the version (e.g. the selection or lack of selection of the child-safe version by the user using the user input device) is considered to be indicative of a user viewing parameter insofar as it is indicative of whether the viewer is an adult or a child and therefore is indicative of the IOD of the viewer and more specifically of whether the IOD is an adult IOD or a child IOD. This selection is placed in a message 516 sent from the content access system 328 to the content management system 302. In particular the processing logic 340 generates the message 516 containing an identification of a selected version of the stereoscopic content and instructs communication interface system 330 to transmit it to the content management system 302 or more particularly to the communications interface system 318 of the content management system 302.
[00340] It will be appreciated that this is merely one example of a possible implementation. For example, instead of providing in message 514 a list of versions, the message 514 may simply request viewer parameter data, in response to which the content access system 328 may provide in message 516 direct-representation digital viewing parameter data. In one such example, each film is still offered in an original version and a child-safe version but instead of sending the list of versions and requesting a selection, the content management system 302 transmits in message 514 a request for direct-representation digital viewing parameter data, and more particularly an indication of an IOD.
[00341] In response to the message 514, the remote user application may prompt a user for user viewing parameter data as described herein and more specifically for IOD data, or may find this data within local storage 342 if it has been previously recorded (as it should be mentioned may be the case not only with IOD data but indeed with any variation of digital viewing parameter data). The remote user application then generates the message 516 comprising digital viewing parameter data indicative of an interocular distance of a viewer. In particular, the digital viewing parameter data may be indicative of the age of a viewer, the age of the viewer being indicative of and interocular distance. Indeed, while the age indication may be a Boolean type of value (adult or child) it may also represent different age ranges characterized by different typical interocular distance. Indeed as children grow the interocular distance grows until they reach physical maturity and while it is very beneficial to offer a single child-safe mode intended to protect children so young as to have the smallest typical IOD for stereoscopic content viewers, it may be even more beneficial to provide a range of versions for different IODs typical of different age ranges such that children of all ages can safely enjoy the optimal depth-fidelity of image.
[00342] Once the content management system 302 has received message 516 from the remote user application, the processing logic 316 of the content management system 302 selects the appropriate version of the selected stereoscopic content on the basis of the received digital viewing parameter data. [00343] Once a version has been selected, the next steps are similar to those described in the example provided with Figure 17. In particular, the content management system 302 requests the content storage system 304 to transmit the selected stereoscopic content in the selected version to the remote user application. In particular the processing logic 316 of the content management system 302 generates a message 520 identifying the selected content and the selected version thereof, for example by providing a location indicator for the selected version of the selected content, which may have been previously included in the stereoscopic content database 320 provided by the content storage system 304 in message 508. Processing logic 316 then instructs communication interface 318 to transmit the message 520 to the content storage system 304, and more particularly to the communication interface 312 of the content storage system 304.
[00344] In response the content storage system 304 transmits the selected version of the selected content to the remote user application and more particularly control 310 causes the transfer of the selected version of the selected content from the digital content library 308 through the communication interface 3123 to the content access system 328 and more particularly to the communication interface system 330 of the content access system 328.
[00345] In this example the different versions are universal, that is, they are shared by all programs in the digital content library 308. As such, identifying the different version available can be simplified to simply knowing what the universal versions are. This can be stored in the local storage 314 of the content management system 302 or the content storage system 304 can provide this information upon being queried.
[00346] While in the above examples, the stereoscopic content is transmitted to the content access system 328 directly from the content storage system 304, it is to be appreciated that in order to afford greater control from the content management system 302, the stereoscopic content could be transferred to the user end 306 from the content management system 302. In such a case the content is transmitted from the content storage system 304 to the content management system 302 (via communication interfaces 312 and 318 prior to transmission by the content management system 302 (via communication interface 318) to the user end 306. This has the advantage of providing more control to the content management system 302 but requires a higher bandwidth both between the content management system 302 and the content storage system 304 and between the content management system 302 and the user end 306 (though the content storage system 304 no longer needs to be part of the content distribution network or distribute content a high bandwidth link to end user 306).
[00347] Alternatively, as illustrated in Figure 21, the functionality of the content management system 302 and the content storage system 304 may be combined in one single entity, a content management system 702 that contains the digital content library 308. The digital content library 308 is shown here in its own storage medium, presumably a server storage bank and is separate from the local storage 314 which still contains the user database 322. Of course the two could be in the same physical storage media. Since the content management system 702 comprises the digital content library 308, the stereoscopic content database 320 has been omitted since the processing logic 716 has access to the contents of the digital content library 308 directly. That said, in order to avoid having to browse the whole digital content library 308 for data on its content, the content management system 702 may still comprise the stereoscopic content database 320 (not shown), for example in the local storage 314 as was the case with content management system 302 . In this system, the content management system 702 communicates directly with the user end 306 and in particular with the content access system 328. The content access system 328 remains relatively unchanged, with the exception that communications that were previously described as being between it and the content management system 302 and the content storage system 304 are now both between it and the content management system 702. Accordingly the communication interface 718 of the content management system 702 embodies the functions of both the communication interfaces system 318 and 312 except, of course the function of communicating between the communication interfaces system 318 and 312.
[00348] In certain embodiments, a stereoscopic content distribution system 700 shown in Figure 21 may include a reconfigurator 704 for doing the reconfiguration of stereoscopic content for example in the manner taught by the aforementioned copending application.
[00349] In one embodiment, reconfigurator 704 is a real-time reconfigurator as taught in the aforementioned copending application and is used by the content management system 702 to reconfigure in rea-time the stereoscopic content contained in the digital content library 308. In this embodiment, the digital content library 308 needs only store one version of all stereoscopic content, e.g. an original version, and new versions are created on the fly in real-time in response to, and adapted for, the received digital viewing parameter data.
[00350] More particularly, the content management system 702 may receive the stereoscopic content in a first (e.g. original) configuration. This may be, for example, received as studio files as described above or otherwise inputted into the digital content library 308. The content management system 702 may then receive digital viewing parameter data from the content access system 328 as described above. Using the digital viewing parameter data the processing logic 716 may determine a configuration suitable for viewing in the remote environment at the user end 306. The processing logic 716 then causes the performing of a reconfiguration operation by the reconfigurator 704 to generate a second stereoscopic configuration of stereoscopic content, the second reconfiguration corresponding to at least one parameter from a set of user viewing parameters defining a viewing environment at the user end, and of which the digital viewing parameter data was indicative of at least one viewing parameter. Finally, the stereoscopic content in the second configuration is made to be transmitted to the content access system 328, in this case via the communication interface 718.
[00351] Advantageously, this allows the generation of versions tailored precisely to specific user viewing parameters received from the content access system 328. However, reformatting in real-time may prove overly burdensome if it is expected that many end-users may be requesting reconfigured stereoscopic content at the same time. In one embodiment, the stereoscopic content distribution system 700 may be a hybrid model whereby pre-reconfigured versions of stereoscopic data are stored in the digital content library 308 for the most common viewing environments and when uncommon digital viewing parameter data is received a special version of the stereoscopic content is reconfigured in real-time for the requestor.
[00352] Alternatively, the reconfigurator 704 may not be called upon to reconfigure in real- time but may simply be present to reconfigure stereoscopic content received at the content management system 702. Indeed, studio files being typically in an original version, it may be necessary to actually generate the reconfigured version of each program in order to be able to offer them. To this end, the stereoscopic content distribution system 700 may include a reconfigurator 704, for example in the content management system 702, in order to generate reconfigured version of the stereoscopic content to make available to end users. The reconfigurator may implement, for example the high-quality reconfiguration scheme provided in the aforementioned copending application. Reconfigured versions of stereoscopic content may be subjected to all the modules and process steps described in relation to the content storage/provisioning system 210 illustrated in Figure 16 including quality checks, and multibitrate coding and encryption. In addition, reconfigured versions of stereoscopic content may be subjected to an additional quality control step to verify the quality of the reconfiguration process itself and in particular to check for artefacts and infidelities that may be caused by the reconfiguration. This may include objective or subjective analyses.
[00353] It will be appreciated that the content access system 328 may be a hardware system such as the smart TV described above or other hardware system comprising a processing unit, and a network interface. The processing unit may be a programmable processing unit configured by software instructions physically residing on software storage media which may be the local storage media and instructing the processing unit to perform implement a remote user application and to perform as configured. That said, the content access system 328 may also be a software system implementing a remote user application, wherein the communication interface system is a set of software instruction residing on software storage media for instruction a software- programmable device having a network interface to communicate over the network interface, and wherein the processing logic comprises software instructions residing on software storage media instructing a processing unit in the software-programmable device to perform the configuration defined by the software instructions.
[00354] Although various embodiments have been illustrated, this was for the purpose of describing, but not limiting, the present invention. Various possible modifications and different configurations will become apparent to those skilled in the art and are within the scope of the present invention, which is defined more particularly by the attached claims.

Claims

What is claimed is:
1. A method for managing access to viewable stereoscopic content in a digital content library by a remote user application for viewing at a remote viewing environment characterised by a set of user viewing parameters, the method comprising the steps of: a. determining the presence in the digital content library of a first version of a stereoscopic content in a first stereoscopic configuration, the first stereoscopic configuration corresponding to a first set of viewing parameters; b. determining the presence in the digital content library of a second version of the stereoscopic content in a second stereoscopic configuration, the second stereoscopic configuration corresponding to a second set of viewing parameters; c. receiving digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters;
d. receiving from the remote user application a request for the stereoscopic content; e. selecting on the basis of the digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application; and f. provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
2. The method of claim 1, wherein selecting on the basis of the digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application comprises identifying a set of viewing parameters from among the first set of viewing parameters and the second set of viewing parameters which best correspond to the user viewing parameter and selecting the version of the stereoscopic content having the configuration best corresponding to the identified set of viewing parameters.
3. The method of claim 2, wherein the first stereoscopic configuration corresponds to adult viewing parameters and the second stereoscopic configuration corresponds to child viewing parameters, and wherein the digital viewing parameter data is indicative of whether the intended viewer is an adult or a child.
4. The method of claim 3, wherein the first set of viewing parameters and the second set of viewing parameters comprise first and second interocular distances respectively, the first interocular distance being an adult interocular distance and the second interocular distance being a child interocular distance, and wherein the digital viewing parameter data is indicative of a viewer interocular distance.
5. The method of claim 4, wherein the digital viewing parameter data is indicative of an age of a viewer, the age of the viewer being indicative of an interocular distance.
6. The method of claim 1, further comprising the steps of:
a. receiving service registration information, the service registration information being indicative of an intent to register for access to content in the digital content library and comprising unique user data;
b. generating a unique user account on the basis of the unique user data; and c. associating the at least one viewing parameter from the set of user viewing parameter to the unique user account
d. the service registration information further comprising the digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters
7. The method of claim 6, wherein the service registration information is received from the remote user application.
8. The method of claim 6, wherein the service registration information is entered by the user on a separate device distinct from the viewing environment and connected to the content management server by a network, the method further comprising the step of associating the registration information with the remote user application.
9. The method of claim 8, wherein the service registration information comprises the digital viewing parameter data.
10. The method of claim 8, wherein the digital viewing parameter data is received from the remote user application.
11. The method of claim 1, further comprising: transmitting to the remote user application an inquiry prompting the user application for the digital viewing parameter data.
12. The method of claim 11, wherein the inquiry prompts a display to the user of a request for information, the request for information inviting the user to provide information on the user viewing parameters, the digital viewing parameter data being containing information provided by the user in response to the request for information.
13. The method of claim 1, wherein the digital viewing parameter data comprises information indicative of at least one of:
a. a dimension of a display;
b. a display resolution;
c. a distance between a viewer and a display; and d. the interocular distance of a viewer.
14. The method of claim 1, wherein the digital viewing parameter data comprises device hardware identifier indicative of a particular viewing device, the particular viewing device being associated with at least one viewing parameter.
15. The method of claim 1, wherein the digital viewing parameter data comprises device software identifier indicative of the remote user application, the remote user application being associated with at least one viewing parameter.
16. The method of claim 1, wherein the digital viewing parameter data comprises an indication of a selection of a stereoscopic configuration corresponding to at least one parameter from the set of user viewing parameters.
17. The method of claim 16, further comprising transmitting to the remote user application the first and the second versions of the stereoscopic content, wherein the digital viewing parameter is indicative of a selection of one of the first and second versions of the stereoscopic content, the selection of one of the first and second versions of the stereoscopic content being indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version.
18. The method of claim 1, wherein the stereoscopic content is a stereoscopic program from among a plurality of stereoscopic programs contained in the digital content library, and wherein the request for the stereoscopic content comprises a selection of one of the stereoscopic programs from the list of stereoscopic programs the method further comprising: transmitting to the remote user application a list of stereoscopic programs contained in the digital content library for display to the user.
19. The method of claim 1, wherein determining the presence in the digital content library of a first version of a stereoscopic content comprises receiving and storing in management memory an indication of a location in the digital content library of the first version of a stereoscopic content and determining the presence in the digital content library of a second version of the stereoscopic content comprises receiving and storing in the memory an indication of a location in the digital content library of the second version of the stereoscopic content.
20. The method of claim 19, wherein the digital content library is located in a content storage server, and wherein provoking the transmission of the stereoscopic content comprises transmitting a signal to the content storage server instructing it to transmit the stereoscopic content in the selected version from the digital content library to the remote user application.
21. The method of claim 19, wherein provoking the transmission of the stereoscopic content comprises transmitting to the remote user application an authorisation token indicative of an authorization for viewing the stereoscopic content in the selected version.
22. The method of claim 1, wherein determining the presence in the digital content library of a first version of a stereoscopic content comprises storing in a content memory the first version of the stereoscopic content and determining the presence in the digital content library of a second version of the stereoscopic content comprises storing in a content memory the second version of the stereoscopic content, and wherein provoking the transmission of the stereoscopic content comprises transmitting to the remote user application comprises transmitting the stereoscopic content in the selected version to the remote user application.
23. The method of claim 22, wherein determining the presence in the digital content library of a first version of a stereoscopic content further comprises receiving the first version of the stereoscopic content, and wherein determining the presence in the digital content library of a second version of the stereoscopic content further comprises, reconfiguring the first version of the stereoscopic content to generate the second version of the stereoscopic content.
24. The method of claim 1, wherein the remote user application is one of a plurality of remote user applications, the method further comprising:
a. receiving from the remote user application a remote user identifier and determining a permission to access the stereoscopic content on the basis of the remote user identifier.
25. The method of claim 1, further comprising determining the presence in a digital library of at least one additional version of the stereoscopic content in a corresponding at least one additional stereoscopic configuration, each of the at least one additional stereoscopic configuration corresponding to a respective one of at least one additional set of viewing parameters.
26. The method of claim 25, wherein selecting on the basis of the digital viewing parameter data a version of the stereoscopic content to transmitted to the remote user application comprises identifying a set of viewing parameters from among the first set of viewing parameters, the second set of viewing parameters and the at least one additional set of viewing parameters which best correspond to the user viewing parameter and selecting the version of the stereoscopic content having the configuration best corresponding to the identified set of viewing parameters.
27. A method for managing access to viewable stereoscopic content in a digital content library by a remote user application at a remote viewing environment characterised by a set of user viewing parameters, the method comprising the steps of:
a. receiving a stereoscopic content in a first stereoscopic configuration, the first stereoscopic configuration corresponding to a first set of viewing parameters; b. receiving from a remote user application a digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter;
c. determining on the basis of the digital viewing parameter data whether the first stereoscopic configuration is suitable for viewing in the remote viewing environment;
d. upon determining that the first stereoscopic configuration is not suitable for viewing in the remote viewing environment, performing a reconfiguration operation to generate a second stereoscopic configuration corresponding to the at least one viewing parameter from the set of user viewing parameter; and e. provoking the transmission of the stereoscopic content in the second stereoscopic configuration to the remote user application.
28. The method of claim 27, wherein the reconfiguration operation is performed in real-time while the transmission of the stereoscopic content is being transmitted.
29. A method for accessing viewable stereoscopic content from a digital content library by a user application for a viewing device being part of a viewing environment, the viewing environment characterised by a set of user viewing parameters, the method comprising the steps of:
a. transmitting to a content management server a request for a particular stereoscopic content;
b. transmitting to the content management server digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter, the digital viewing parameter data being used for identifying a particular stereoscopic configuration corresponding to the viewing environment; c. receiving the stereoscopic content in the particular stereoscopic configuration; d. causing the stereoscopic content in the particular stereoscopic configuration to be displayed on a display associated with the viewing environment.
30. The method of claim 29, wherein the stereoscopic content is received from the content management server.
31. The method of claim 29, wherein receiving the stereoscopic content in the particular stereoscopic configuration comprises:
a. generating a request for the stereoscopic content;
b. transmitting the request for the stereoscopic content to a content storage server; and
c. receiving the stereoscopic content in the particular stereoscopic configuration from the content storage server.
32. The method of claim 31, further comprising receiving from the content management server a configuration identifier identifying the particular stereoscopic configuration, and wherein the request for the stereoscopic content comprises an indication of the configuration identifier.
33. The method of claim 31, wherein receiving the stereoscopic content in the particular stereoscopic configuration comprises receiving from the content management server an authorisation token, and wherein the request for the stereoscopic content comprises the authorisation token
34. The method of claim 29, further comprising:
a. receiving from the content management server a set of versions of the stereoscopic content, each version having a respective stereoscopic configuration each corresponding to a respective set of viewing parameters; and
b. identifying a selected version of the stereoscopic content that has a configuration corresponding to the user viewing parameters that best correspond to the user viewing parameters;
wherein the digital viewing parameter data comprises an indicator of the selected version, the indicator of the selected version being indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version.
35. The method of claim 34, wherein identifying selected version of the stereoscopic content that has a configuration corresponding to the user viewing parameters that best correspond to the user viewing parameters comprises:
a. providing on the viewing device a visual prompt requesting a user to make a selection of a version; and b. receiving at an input from a user input device an indication of the selection of a version that best corresponds to the user viewing parameters.
36. The method of claim 35, wherein the set of versions includes a child-safe version and an adult version, the visual prompt comprising a querry of whether the child-safe version is to be selected.
37. The method of claim 29, wherein the digital viewing parameter data is indicative of whether the intended viewer is an adult or a child.
38. The method of claim 29, wherein the digital viewing parameter data is indicative of an interocular distance of a viewer.
39. The method of claim 38, wherein the digital viewing parameter data is indicative of an age of a viewer, the age of the viewer being indicative of an interocular distance.
40. The method of claim 29, wherein the digital viewing parameter data comprises information indicative of at least one of:
a. a dimension of a display;
b. a display resolution;
c. a distance between a viewer and a display; and
d. the interocular distance of a viewer.
41. The method of claim 29, further comprising the steps of:
a. generating registration information, the service registration information being indicative of an intent to register for access to content in the digital content library and comprising unique user data, the service registration information further comprising the digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters; and
b. transmitting to the content management server the registration information.
42. The method of claim 41, wherein generating the registration information comprises: a. providing on the viewing device a visual prompt requesting specific user identification information; and
b. receiving at an input from a user input device the user identification information; wherein the registration information comprises the user identification information.
43. The method of claim 29, further comprising:
a. providing on the viewing device a visual prompt requesting the user to input an entered user viewing parameter; and
b. receiving at an input from a user input device the entered user viewing parameter; wherein the at least one viewing parameter indicated by the digital viewing parameter data comprises the entered user viewing parameter.
44. The method of claim 43, wherein the visual prompt prompts the user to enter at least one of:
a. a distance from the user to the viewing device;
b. a dimension of a display;
c. a display resolution; and
d. the interocular distance of a viewer.
45. The method of claim 29, wherein the digital viewing parameter data comprises device hardware identifier indicative of the viewing device, the viewing device being associated with at least one viewing parameter.
46. The method of claim 45, wherein the hardware identifier is a television model number identifier, the television model number being associated with, and indicative of, a dimension of the display and a display resolution.
47. The method of claim 29, wherein the digital viewing parameter data comprises device software identifier indicative of the user application, the user application being associated with at least one viewing parameter.
48. The method of claim 47, wherein the software identifier is indicative of a platform on which the user application is running, the platform being associated with at least one viewing parameter.
49. The method of claim 29, further comprising:
a. receiving form the content management server a list of one or more stereoscopic programs contained in the digital content library;
b. presenting a visual representation of the list of one or more stereoscopic programs to a user on the viewing device;
c. presenting a visual prompt to the user the prompting the user to make a selection of a program for display on the viewing device; and
d. receiving at an input from a user input device an indication of the selection of a program; and e. transmitting to the content management server an indication of the selected program.
50. A method for providing stereoscopic video-on-demand content to a remote user operating a remote user application, the method comprising:
a. at a content management server, providing the remote user a list of stereoscopic programs available in a digital content library for display on a viewing device to the remote user;
b. for a selected stereoscopic program in the list of stereoscopic programs, providing a regular version and a child-safe version, the regular version being an original configuration of the stereoscopic program and the child-safe version being a reconfigured version of the program reconfigured to adapt the program to a child interocular distance;
c. selecting on the basis of digital viewing parameter data received from the remote user application one of the regular version and the child safe version of the selected stereoscopic; and
d. causing the selected version of the selected stereoscopic program to be transmitted to the remote user application for display on the viewing device.
51. The method of claim 50, further comprising: e. receiving from the remote user application an indication of the selected stereoscopic program.
52. The method of claim 50, wherein causing the selected version of the selected stereoscopic program to be transmitted to the remote user application comprises streaming the selected version of the selected stereoscopic program to the viewing device via a content distribution network.
53. The method of claim 50, further comprising receiving the digital viewing parameter data from the remote user application.
54. The method of claim 53, wherein providing a regular version and a child-safe version comprises causing the remote user application to provide a visual prompt prompting the user to select one of the regular version and the child-safe version.
55. The method of claim 53, further comprising: receiving from the remote user application the viewing parameter data, the viewing parameter data being an indication of a selection of one of the regular version and the child-safe version provided in response to the visual prompt.
The method of claim 50, further comprising associating the digital viewing parameter data with a user account associated with one of a user and the remote user application.
57. The method of claim 56, further comprising receiving service registration information from the remote user application, the service registration information comprising the digital viewing parameter data, and generating a user account on the basis of the service registration information.
58. The method of claim 57 wherein selecting on the basis of digital viewing parameter data received from the remote user application one of the regular version and the child safe version of the selected stereoscopic comprises retrieving the digital viewing parameter data from a management memory.
A graphical user interface implemented with a viewing device for presenting to a user of the viewing device access to stereoscopic content in a digital content library offering for viewing at the viewing device, the graphical user interface comprising:
a. a first pane comprising:
i. a plurality of first visual elements, each of the first visual elements being representative of a category of stereoscopic program; and ii. for each first visual element, a first input element associated with the visual element, the first input element being operable by the user using an input device to select the category of stereoscopic program associated with the first visual element associated with the first input element;
b. a second pane comprising:
i. a plurality of second visual elements, each of the second visual elements being representative of a stereoscopic program; and
ii. for each second visual element, a second input element associated with the visual element, the second input element being operable by the user using the input device to select the stereoscopic program associated with the second visual element associated with the second visual element;
c. a third pane comprising:
i. a visual element displaying textual information about a particular stereoscopic program; and
ii. an third input element associated with the particular stereoscopic program, the third input element being operable to select for viewing the particular stereoscopic program; and d. a version visual element indicating the availability of a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters, the version visual element further providing for at least one of the plurality of versions information regarding the set of viewing parameters respective to corresponding stereoscopic configuration;
e. a version input element operable by the user using the input device to select a version from amongst the plurality of versions of the particular stereoscopic program.
60. The graphical user interface of claim 59, wherein the second pane is presented visually in the graphical user interface in response to operation of the first input element using the input device by the user.
61. The graphical user interface of claim 60, wherein the third pane is presented visually in the graphical user interface in response to operation of the second input element using the input device by the user.
62. The graphical user interface of claim 61, further comprising a stereoscopic program viewing pane for presenting a selected stereoscopic program in response to operation of the third input element using the input device by the user.
63. The graphical user interface of claim 62, further comprising a progress and control system comprising graphical user interface elements for allowing user interaction for modifying stereoscopic program presentation, wherein the version input element is in the progress and control system.
64. The graphical user interface of claim 63, wherein upon operation of the version input element by a user using the input device to select a version of the particular stereoscopic program that is different from a current version while the stereoscopic program is being present, the stereoscopic program being presented changes the current version to the selected version.
65. The graphical user interface of claim 59, wherein the plurality of versions comprise a regular version and a child-safe version, the regular version being in an original configuration of the stereoscopic program and the child-safe version being a reconfigured version of the program reconfigured to adapt the program to a child interocular distance.
66. The graphical user interface of claim 65, wherein information regarding the set of viewing parameters provided by the version visual element is an indication that the child- safe version is an indication that the child-safe version is for viewing by children, wherein viewing by children implies a child interocular distance.
67. The graphical user interface of claim 65, wherein the version visual element is a visual prompt prompting the user to select by operating the version input element whether the particular stereoscopic program is to be presented in the child-safe version.
68. The graphical user interface of claim 67, wherein the version visual element is a child- safe icon representing the availability of a child-safe mode and the version input element is the child-safe icon which can be selected and actuated by the user using the user input device to select the child-safe version.
69. The graphical user interface of claim 59, wherein the version visual element and the version input element are located in the third pane.
70. The graphical user interface of claim 59, wherein the version visual element is presented in a fourth pane, the fourth pane being presented visually in the graphical user interface in response to operation of the input element using the input device by the user.
71. The graphical user interface of claim 59, wherein the version visual element comprises a list of representations of selectable versions each representation of a selectable version corresponding to one of the plurality of versions of the particular stereoscopic program, and wherein the version input element is the list of representations of selectable version whereby each representation of a selectable version is selectable and actuatable by the user using the user input device to select the corresponding one of the plurality of version of the particular stereoscopic program.
72. The graphical user interface of claim 59, wherein the version visual element is a prompt prompting the user for information on the user viewing parameters, the prompt being indicative that different versions may be available for different viewing parameters, wherein the version input element is operable by the user using the user input device to provide information on the user viewing parameters.
73. The graphical user interface of claim 72, wherein the version input element comprises an input for entering an age of a viewer, the age of the viewer being indicative of an interocular distance.
74. The graphical user interface of claim 73, wherein the version visual element comprises a list of viewer ages, and wherein the version input element is the list of viewer ages whereby each viewer age is selectable and actuatable by the user using the user input device.
75. The graphical user interface of claim 72, wherein the version input element comprises an input for entering a viewing distance.
76. The graphical user interface of claim 75, wherein the version input element comprises a text box in which a user can enter using the user input device the viewing distance.
77. The graphical user interface of claim 72, wherein the version input element is operable by the user using the input device to provide two or more user viewing parameters. 78. A content management system for managing access to viewable stereoscopic content in a digital content library by a remote user application at a remote viewing environment characterised by a set of user viewing parameters, the content management entity comprising:
a. a stereoscopic content database comprising a set of records of stereoscopic content in a digital content library, the digital stereoscopic content database comprising for at least one of the records of stereoscopic content the identification of a plurality of versions of the stereoscopic content, each of the plurality of versions being in a different stereoscopic configuration, each stereoscopic configuration corresponding to a different set of viewing parameters; b. a communication interface system for communicating with a remote entity, the communication interface being suitable for receiving digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters;
c. processing logic configured for
i. accessing the records of stereoscopic content in the stereoscopic content database,
ii. accessing the digital viewing parameter data received from the remote entity,
iii. selecting on the basis of the received digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application; and
iv. provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application.
79. The content management system of claim 78, wherein the communication interface system is in communication with a content storage system, and wherein each of the records of stereoscopic content comprises an identifier of a respective stereoscopic content in the content storage system.
80. The content management system of claim 79, wherein each of the records of stereoscopic content comprises an address of the respective stereoscopic content.
81. The content management system of claim 79, wherein each of the at least one of the records of stereoscopic content comprises for each of the plurality of versions, an address of the version.
82. The content management system of claim 79, wherein for provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application, the processing logic is configured to cause the communication interface to transmit an instruction to the content storage system instructing it to transmit the stereoscopic content in the selected version from the digital content library to the remote user application.
83. The content management system of claim 79, wherein for provoking the transmission of the stereoscopic content in the selected version from the digital content library to the remote user application, the processing logic is configured to cause the communication interface to transmit to the remote user application an authorisation token indicative of an authorization for viewing the stereoscopic content in the selected version.
84. The content management system of claim 79, wherein the identification of a respective stereoscopic content in the content storage system is a location identifier identifying the location of the stereoscopic content.
85. The content management system of claim 79, wherein the content storage system stores the plurality of versions of the stereoscopic content, and wherein each of the records of stereoscopic content comprises a location identifier for each of the plurality of versions of the respective stereoscopic content.
86. The content management system of claim 78, wherein the processing logic is configured for causing the reconfiguration of one of the version of the stereoscopic content to generate another of the plurality of versions of the stereoscopic content.
87. The content management system of claim 86, wherein the processing logic is configured to access the one of the plurality of versions of the stereoscopic content and reconfiguring the one of the plurality of versions of the stereoscopic content to generate the another of the plurality of versions of the stereoscopic content.
88. The content management system of claim 86, wherein the processing logic is configured to do the reconfiguring on the basis of the step of selecting the another of the plurality of versions in the step of selecting on the basis of the received digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application.
89. The content management system of claim 78, wherein the identification of a plurality of versions of the stereoscopic content is shared among a plurality of records from the set of records of stereoscopic content.
90. The content management system of claim 78, further comprising a content storage system containing the digital content library, wherein the communication interface system is further suitable for transmitting viewable stereoscopic content to the remote entity.
91. The content management system of claim 90, wherein the stereoscopic content database comprises the stereoscopic content.
92. The content management system of claim 78, wherein the plurality of versions of the stereoscopic content for each of the at least one of the records of stereoscopic content include a first version in a first stereoscopic configuration the first stereoscopic configuration corresponding to a first set of viewing parameters and a second version in a second stereoscopic configuration, the second stereoscopic configuration corresponding to a second set of viewing parameters.
93. The content management system of claim 92, wherein selecting on the basis of the received digital viewing parameter data a version of the stereoscopic content to be transmitted to the remote user application, the processing logic is configured to identify a set of viewing parameters from among the first set of viewing parameters and the second set of viewing parameters which best correspond to the user viewing parameter and select the version of the stereoscopic content having the configuration best corresponding to the identified set of viewing parameters.
94. The content management system of claim 93, wherein the first stereoscopic configuration corresponds to adult viewing parameters and the second stereoscopic configuration corresponds to child viewing parameters, and wherein the digital viewing parameter data is indicative of whether the intended viewer is an adult or a child.
95. The content management system of claim 94, wherein the first set of viewing parameters and the second set of viewing parameters comprise first and second interocular distances respectively, the first interocular distance being an adult interocular distance and the second interocular distance being a child interocular distance, and wherein the digital viewing parameter data is indicative of a viewer interocular distance. 96. The content management system of claim 95, wherein the digital viewing parameter data is indicative of an age of a viewer, the age of the viewer being indicative of an interocular distance.
97. The content management system of claim 78, wherein the communication interface system is suitable for receiving the digital viewing parameter data from the remote user application.
98. The content management system of claim 78, further comprising a user database comprising a plurality of user records each user record corresponding to a particular user account and comprising user account data.
99. The content management system of claim 98, wherein the user account data comprises at least one viewing parameter. 100. The content management system of claim 98, wherein the communication interface system is further suitable for receiving service registration information, the service registration information being indicative of an intent to register for access to content in the digital content library and comprising unique user data and comprising the digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters, and wherein the processing logic is further configured to: a. generate a user record on the basis of the unique user data;
b. associating the at least one viewing parameter from the set of user viewing parameter to the user record.
101. The content management system of claim 100, the communication interface system is suitable for receiving the service registration information from the remote entity.
102. The content management system of claim 100, wherein the communication interface system is in communication with a registration server, the communication interface system being suitable for receiving the service registration information from the registration server.
103. The content management system of claim 99, wherein the at least one viewing parameter of the user account data indicates a user preference.
104. The content management system of claim 103, wherein the at least one viewing parameter of the user account data indicates a preferred configuration for stereoscopic content.
105. The content management system of claim 78, wherein the digital viewing parameter data comprises information indicative of at least one of:
a. a dimension of a display;
b. a display resolution;
c. a distance between a viewer and a display; and
d. the interocular distance of a viewer.
106. The content management system of claim 78, wherein the digital viewing parameter data comprises device hardware identifier indicative of a particular viewing device, the particular viewing device being associated with at least one viewing parameter.
107. The content management system of claim 78, wherein the digital viewing parameter data comprises device software identifier indicative of the remote user application, the remote user application being associated with at least one viewing parameter.
108. The content management system of claim 78, wherein the digital viewing parameter data comprises an indication of a selection of a stereoscopic configuration corresponding to at least one parameter from the set of user viewing parameters
109. The content management system of claim 78, wherein the set of records in the stereoscopic content database comprises a plurality of records.
110. The content management system of claim 78, wherein each record of stereoscopic content comprises an indication of a location in the digital content library of the first version of a stereoscopic content and determining the presence in the digital content library of a second version of the stereoscopic content comprises receiving and storing in the memory an indication of a location in the digital content library of the second version of the stereoscopic content. 111. The content management system of claim 78, wherein the content management system is a server.
12. A method for permitting access by a remote user application to viewable stereoscopic content in a configuration adapted for a set of viewing parameters characterizing a remote viewing environment, the method comprising:
a. establishing communication with a remote user device;
b. transmitting to the remote user device a registration form comprising queries prompting the inputting of registration information by a user at the user device, the queries including at least one query prompting the input of at least one viewing parameter;
c. receiving from the remote user device the registration information, the registration information comprising digital viewing parameter data comprising the at least one viewing parameter;
d. causing the association of the registration information with a unique user account at a content management system for selection by the content management system on the basis of the digital viewing parameter data of a version of stereoscopic from amongst a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters.
13. The method of claim 112, wherein causing the association of the registration information with a unique user account at the content management system comprises transmitting the registration information comprising the digital viewing parameter data together to the content management system for association by the content management system.
114. The method of claim 113, wherein the user account is generated at the content management system, and associated to the registration information by the content management system.
115. The method of claim 112, wherein causing the association of the registration information with a unique user account at a content management system comprises generating a unique user account and associating the registration information comprising the digital viewing parameter data to the unique user account.
116. The method of claim 112, further comprising communicating with a payment authority server to establish an eligibility of the user to register an account. 117. The method of claim 116, wherein the registration information comprises payment information, and wherein communicating with a payment authority server to establish an eligibility of the user to register an account comprises transmitting the payment information to the payment authority server and receiving from the registration server an indication of the eligibility of the user to register the account.
118. The method of claim 112, wherein communicating with a payment processing server to establish an eligibility of the user to register an account comprises:
a. transmitting to the remote user device a payment information form comprising queries prompting the inputting of registration payment information by the user at the user device for transmission from the user device to the payment authority server; and
b. receiving from the payment authority server the eligibility of the user to register the account. 119. The method of claim 112, wherein the remote user application runs on the user device.
120. The method of claim 112, wherein the user device is a separate user device from the one on which the remote user application runs.
121. A registration system for permitting access by a remote user application to viewable stereoscopic content in a configuration adapted for a set of viewing parameters characterizing a remote viewing environment, the system comprising:
a. a communication interface system for establishing bidirectional communication with a remote user device;
b. processing logic configured to:
i. cause the transmission using the communication interface system to the remote user device a registration form comprising queries prompting the inputting of registration information by a user at the user device, the queries including at least one query prompting the input of at least one viewing parameter;
ii. process registration information comprising digital viewing parameter data comprising the at least one viewing parameter received by the communication interface system from the remote user device to cause the association of the registration information with a unique user account at a content management system for selection by the content management system on the basis of the digital viewing parameter data of a version of stereoscopic content from amongst a plurality of versions of stereoscopic content, each of the plurality of versions corresponding to a different stereoscopic configuration corresponding to a respective set of viewing parameters.
122. The registration system claim 121, wherein the communication interface system is in communication with the content management system.
123. The registration system of claim 122, wherein to process the registration information, the processing logic is configured to cause the association of the registration information with a unique user account at the content management system comprises transmitting the registration information comprising the digital viewing parameter data together to the content management system for association by the content management system.
124. The registration system of claim 123, wherein the user account is generated at the content management system, and associated to the registration information by the content management system.
125. The registration system of claim 122, wherein to cause the association of the registration information with a unique user account, the processing logic is configured to generate a unique user account and associating the registration information comprising the digital viewing parameter data to the unique user account.
126. The registration system of claim 121, wherein the communication interface system is in communication with a payment authority server.
127. The registration system of claim 126, wherein the processing logic is further configured to establish an eligibility of the user to register an account on the basis of communication with the payment authority server. 128. The registration system of claim 127, wherein the registration information comprises payment information, and wherein to establish an eligibility of the user to register an account the processing logic is configured to cause the transmission over the communication interface system to the payment authority server of the payment information, and wherein the processing logic is further configured to process eligibility information received from the payment authority by the communication interface system to establish the eligibility of the user to register the account.
129. The registration system of claim 126, wherein the processing logic is further configured to:
a. cause the transmission to the remote user device over the communication interface system of a payment information form comprising queries prompting the inputting of registration payment information by the user at the user device for transmission from the user device to the payment authority server; and b. process eligibility information received from the payment authority server over the communication interface system to establish the eligibility of the user to register the account.
130. The registration system of claim 121, wherein the registration system is a web server.
131. The registration system of claim 121, wherein the at least one viewing parameter comprises information indicative of at least one of:
a. a dimension of a display;
b. a display resolution;
c. a distance between a viewer and a display; and
d. the interocular distance of a viewer.
132. The registration system of claim 121, wherein the at least one viewing parameter indicates a version of stereoscopic content.
133. The registration system of claim 121, wherein the at least one viewing parameter is indicative of a user preference. 134. A content access system for accessing viewable stereoscopic content from a digital content library for viewing in a viewing environment, the viewing environment characterised by a set of user viewing parameters, the system comprising:
a. a communication interface system for communicating with a content management system;
b. processing logic configured to:
i. cause the transmission using the communication interface system to the content management system of a request for a particular stereoscopic content; ii. cause the transmission using the communication interface system to the content management system of digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameter, the digital viewing parameter data being used for identifying a particular stereoscopic configuration corresponding to the viewing environment; iii. processing a received stereoscopic content received at the communication interface system in the particular stereoscopic configuration in response to the request to cause the received stereoscopic content to be displayed on a display associated with the viewing environment.
135. The content access system of claim 134, wherein the system is a hardware system comprising the display, a processing unit, and a network interface, wherein the communication interface system is the network interface, and wherein the processing logic is the processing unit, and wherein processing stereoscopic content received at the communication interface system in the particular stereoscopic configuration in response to the request to cause the received stereoscopic content to be displayed on a display comprises decoding the received stereoscopic content and displaying it on the display.
136. The content access system of claim 135, wherein the processing unit is a programmable processing unit configured by software instructions physically residing on software storage media and instructing the processing unit to implement a remote user application to perform as configured.
137. The content access system of claim 136, further comprising local storage media, wherein the the software storage media is the local storage media.
138. The content access system of claim 135, wherein the digital viewing parameter data comprises device hardware identifier indicative of the display, the display being associated with at least one viewing parameter
139. The content access system of claim 138, wherein the hardware identifier is a television model number identifier, the television model number being associated with, and indicative of, a dimension of the display and a display resolution
140. The content access system of claim 134, wherein the system is a software system implementing a remote user application, wherein the communication interface system is a set of software instructions residing on software storage media for instructing a software- programmable device having a network interface to communicate over the network interface, and wherein the processing logic comprises software instructions residing on software storage media instructing a processing unit in the software-programmable device to perform the configuration defined by the software instructions.
141. The content access system of claim 140, wherein the digital viewing parameter data comprises device software identifier indicative of the user application, the user application being associated with at least one viewing parameter.
142. The content access system of claim 141, wherein the software identifier is indicative of a platform on which the user application is running, the platform being associated with at least one viewing parameter.
143. The content access system of claim 134, wherein the communication interface system receives the stereoscopic content from the content management server. 144. The content access system of claim 134, further comprising a user interface for receiving input from the user, wherein the processing logic generates the request for the particular stereoscopic content in response to a user input.
145. The content access system of claim 134, wherein the communication interface system is in communication with a content storage server, and wherein the received stereoscopic content is received from the content storage server.
146. The content access system of claim 145, wherein the processing logic is further configured to process an authorisation token received at the communication interface system from the content management system and cause generate a request to the content storage server for the stereoscopic content comprising the authorisation token.
147. The content access system of claim 136, wherein the request for the stereoscopic content comprises the digital viewing parameter data.
148. The content access system of claim 147, wherein the digital viewing parameter data comprises a configuration identifier indicating a particular version to be selected from among a set of versions of the stereoscopic content, each version having a respective stereoscopic configuration each corresponding to a respective set of viewing parameters.
149. The content access system of claim 148, wherein the set of versions of the stereoscopic content is received at the communication interface system and the processing logic is further configured to identifying from among the set of versions a selected version of the stereoscopic content that has a configuration corresponding to the viewing parameters that best correspond to the user viewing parameters; and wherein the digital viewing parameter data comprises an indicator of the selected version, the indicator of the selected version being indicative that the user viewing parameters best correspond to the set of viewing parameters corresponding to the configuration of the selected version.
150. The content access system of claim 149, further comprising a user interface for receiving input from the user, wherein the processing logic identifies a selected version of the stereoscopic content in response to a user input.
151. The content access system of claim 150, wherein the set of versions includes a child-safe version and an adult version, and wherein the user input is indicative of a selection by the user of whether the child-safe version is to be selected.
152. The content access system of claim 136, wherein the digital viewing parameter data is indicative of whether the intended viewer is an adult or a child.
153. The content access system of claim 136, wherein the digital viewing parameter data is indicative of an interocular distance of a viewer.
154. The content access system of claim 153, wherein the digital viewing parameter data is indicative of an age of a viewer, the age of the viewer being indicative of an interocular distance.
155. The content access system of claim 136, wherein the digital viewing parameter data comprises information indicative of at least one of:
a. a dimension of a display;
b. a display resolution;
c. a distance between a viewer and a display; and
d. the interocular distance of a viewer.
156. The content access system of claim 136, further comprising further comprising a user interface for receiving input from the user, wherein the processing logic generates the digital viewing parameter data on the basis of input received from the user interface.
157. The content access system of claim 156, wherein the user interface includes a device for entering data, wherein the at least one viewing parameter indicated by the digital viewing parameter data comprises data entered by the user.
158. The content access system of claim 157, wherein the data entered by the user comprises at least one of:
a. a distance from the user to the viewing device;
b. a dimension of a display;
c. a display resolution; and
d. the interocular distance of a viewer.
159. A system for distributing stereoscopic video-on-demand content comprising: a. a content management server having a digital stereoscopic content database comprising a set of records of stereoscopic content held in a digital content library;
b. a digital content library storing the stereoscopic content;
c. a remote user application in communication with the content management server and the content storage server at a remote viewing environment characterised by a set of user viewing parameters;
wherein the remote user application is operative to send a request for a particular stereoscopic content from the content management server and send to the content management server digital viewing parameter data indicative of at least one viewing parameter from the set of user viewing parameters;
wherein in response to receiving the request for a particular stereoscopic content and the digital viewing parameter data, the content management server selects on the basis of the digital viewing parameter data one of a plurality of possible versions of the particular stereoscopic content each version having a respective stereoscopic configuration each corresponding to a respective set of viewing parameters; and wherein the content management server causes the particular stereoscopic content to be transmitted to the remote user application in the selected version.
160. The system of claim 159, wherein the digital content library is in the content management server, and wherein the content management server sends the particular stereoscopic content in the selected version to the remote user application.
161. The system of claim 160, wherein the content management server configures the particular stereoscopic content into the selected version at least in part the digital viewing parameter data.
162. The system of claim 160, wherein the digital content library comprises the particular stereoscopic content in each of the plurality of possible version.
163. The system of claim 162, wherein the digital content library is in a content storage server, and wherein the content management server instructs the content storage server to transmit the particular stereoscopic content in the selected version to the remote user application.
164. The system of claim 162, wherein the plurality of possible version includes a child-safe version, and wherein the digital viewing parameter data is a selection of the child-safe mode.
165. The system of claim 159 further comprising a stereoscopic reformater configured to generate the selected version of the particular stereoscopic content from an originating version, the stereoscopic reformater comprising:
a. a disparity map estimator having an image input for receiving a stereoscopic image pair and a disparity map output for outputting an estimated disparity map and logic configured for generating the estimated disparity map on the basis of the stereoscopic image pair;
b. an image transformation engine comprising processing logic configured for: i. receiving at least one image of the stereoscopic image pair and the estimated disparity map and computing an abstract dataset on the basis of first viewing parameters, the at least one image of the stereoscopic image pair and the estimated disparity map, the abstract dataset defining spatial relationships in the image;
ii. transforming the abstract dataset on the basis of second viewing parameters to generate a transformed abstract dataset; and iii. synthesizing at least one new image of a new stereoscopic image pair comprising the at least one new image on the basis of the transformed abstract dataset; and
c. an output for outputting the new stereoscopic image pair.
166. A stereoscopic reformater configured to generate the selected version of the particular stereoscopic content from an originating version, the stereoscopic reformater comprising:
a. a disparity map estimator having an image input for receiving a stereoscopic image pair and a disparity map output for outputting an estimated disparity map and logic configured for generating the estimated disparity map on the basis of the stereoscopic image pair;
b. an image transformation engine comprising processing logic configured for: i. receiving at least one image of the stereoscopic image pair and the estimated disparity map and computing an abstract dataset on the basis of first viewing parameters, the at least one image of the stereoscopic image pair and the estimated disparity map, the abstract dataset defining spatial relationships in the image;
ii. transforming the abstract dataset on the basis of second viewing parameters to generate a transformed abstract dataset; and
iii. synthesizing at least one new image of a new stereoscopic image pair comprising the at least one new image on the basis of the transformed abstract dataset; and
output for outputting the new stereoscopic image pair.
PCT/CA2014/051228 2014-01-06 2014-12-17 Reconfiguration of stereoscopic content and distribution for stereoscopic content in a configuration suited for a remote viewing environment WO2015100490A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201461924202P 2014-01-06 2014-01-06
US61/924,202 2014-01-06

Publications (1)

Publication Number Publication Date
WO2015100490A1 true WO2015100490A1 (en) 2015-07-09

Family

ID=53492856

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2014/051228 WO2015100490A1 (en) 2014-01-06 2014-12-17 Reconfiguration of stereoscopic content and distribution for stereoscopic content in a configuration suited for a remote viewing environment

Country Status (1)

Country Link
WO (1) WO2015100490A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018112738A (en) * 2016-12-30 2018-07-19 フィコサ アダス,ソシエダッド リミタダ ユニペルソナル Detecting correct or incorrect operation of display panel
WO2018144315A1 (en) * 2017-02-01 2018-08-09 Pcms Holdings, Inc. System and method for augmented reality content delivery in pre-captured environments
CN111684517A (en) * 2018-02-08 2020-09-18 蒙德多凯创新有限公司 Viewer adjusted stereoscopic image display
CN112585987A (en) * 2018-06-22 2021-03-30 皇家飞利浦有限公司 Apparatus and method for generating image data stream
CN113660330A (en) * 2021-08-16 2021-11-16 北京达佳互联信息技术有限公司 Method and device for receiving and sending information, electronic equipment and storage medium
WO2021242932A1 (en) * 2020-05-26 2021-12-02 Unify Medical Generation of three-dimensional images with digital magnification
US11240479B2 (en) 2017-08-30 2022-02-01 Innovations Mindtrick Inc. Viewer-adjusted stereoscopic image display

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013102790A2 (en) * 2012-01-04 2013-07-11 Thomson Licensing Processing 3d image sequences cross reference to related applications
EP2629536A1 (en) * 2010-10-12 2013-08-21 Sharp Kabushiki Kaisha Stereoscopic image conversion device and stereoscopic image display device
WO2013173548A2 (en) * 2012-05-17 2013-11-21 Thomson Licensing Adapting large format stereoscopic content to other platforms

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2629536A1 (en) * 2010-10-12 2013-08-21 Sharp Kabushiki Kaisha Stereoscopic image conversion device and stereoscopic image display device
WO2013102790A2 (en) * 2012-01-04 2013-07-11 Thomson Licensing Processing 3d image sequences cross reference to related applications
WO2013173548A2 (en) * 2012-05-17 2013-11-21 Thomson Licensing Adapting large format stereoscopic content to other platforms

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018112738A (en) * 2016-12-30 2018-07-19 フィコサ アダス,ソシエダッド リミタダ ユニペルソナル Detecting correct or incorrect operation of display panel
JP7036592B2 (en) 2016-12-30 2022-03-15 フィコサ アダス,ソシエダッド リミタダ ユニペルソナル Detection of normal or malfunction of display panel
WO2018144315A1 (en) * 2017-02-01 2018-08-09 Pcms Holdings, Inc. System and method for augmented reality content delivery in pre-captured environments
CN110249291A (en) * 2017-02-01 2019-09-17 Pcms控股公司 System and method for the augmented reality content delivery in pre-capture environment
US11024092B2 (en) 2017-02-01 2021-06-01 Pcms Holdings, Inc. System and method for augmented reality content delivery in pre-captured environments
US11240479B2 (en) 2017-08-30 2022-02-01 Innovations Mindtrick Inc. Viewer-adjusted stereoscopic image display
US11785197B2 (en) 2017-08-30 2023-10-10 Innovations Mindtrick Inc. Viewer-adjusted stereoscopic image display
CN111684517A (en) * 2018-02-08 2020-09-18 蒙德多凯创新有限公司 Viewer adjusted stereoscopic image display
CN112585987A (en) * 2018-06-22 2021-03-30 皇家飞利浦有限公司 Apparatus and method for generating image data stream
US11218680B2 (en) 2020-05-26 2022-01-04 Unify Medical, Inc. Generation of three-dimensional images with digital magnification
WO2021242932A1 (en) * 2020-05-26 2021-12-02 Unify Medical Generation of three-dimensional images with digital magnification
US11483531B2 (en) 2020-05-26 2022-10-25 Unify Medical, Inc. Generation of three-dimensional images with digital magnification
US11800078B2 (en) 2020-05-26 2023-10-24 Unify Medical, Inc. Generation of three-dimensional images with digital magnification
CN113660330A (en) * 2021-08-16 2021-11-16 北京达佳互联信息技术有限公司 Method and device for receiving and sending information, electronic equipment and storage medium
CN113660330B (en) * 2021-08-16 2022-10-04 北京达佳互联信息技术有限公司 Method and device for receiving and sending information, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
WO2015100490A1 (en) Reconfiguration of stereoscopic content and distribution for stereoscopic content in a configuration suited for a remote viewing environment
US8228327B2 (en) Non-linear depth rendering of stereoscopic animated images
TWI523488B (en) A method of processing parallax information comprised in a signal
US9445072B2 (en) Synthesizing views based on image domain warping
TWI444036B (en) 2d to 3d user interface content data conversion
KR101863767B1 (en) Pseudo-3d forced perspective methods and devices
US9031356B2 (en) Applying perceptually correct 3D film noise
KR101829345B1 (en) Method and apparatus for customizing 3-dimensional effects of stereo content
JP5933931B2 (en) Method for displaying setting menu and corresponding device
EP2323416A2 (en) Stereoscopic editing for video production, post-production and display adaptation
KR101487587B1 (en) Method, apparatus and computer program for selecting a stereoscopic imaging viewpoint pair
JP2012518317A (en) Transfer of 3D observer metadata
CN108141578A (en) Camera is presented
JP2007527665A (en) System and method for managing stereoscopic observation
CN102804169A (en) Viewer-centric User Interface For Stereoscopic Cinema
Masaoka et al. Spatial distortion prediction system for stereoscopic images
CN107209949B (en) Method and system for generating magnified 3D images
Minoli 3D television (3DTV) technology, systems, and deployment: Rolling out the infrastructure for next-generation entertainment
US20120086711A1 (en) Method of displaying content list using 3d gui and 3d display apparatus applied to the same
US9325963B2 (en) Device and method for rendering and delivering 3-D content
KR101826025B1 (en) System and method for generating 3d image contents that user interaction is possible
Li et al. On adjustment of stereo parameters in multiview synthesis for planar 3D displays
KR101433082B1 (en) Video conversing and reproducing method to provide medium feeling of two-dimensional video and three-dimensional video
JP7395725B2 (en) Media resource playback and text rendering methods, devices, equipment and storage media
US9547933B2 (en) Display apparatus and display method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14877293

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14877293

Country of ref document: EP

Kind code of ref document: A1