WO2011047343A2 - System and method of displaying three dimensional images using crystal sweep with freeze tag - Google Patents

System and method of displaying three dimensional images using crystal sweep with freeze tag Download PDF

Info

Publication number
WO2011047343A2
WO2011047343A2 PCT/US2010/052951 US2010052951W WO2011047343A2 WO 2011047343 A2 WO2011047343 A2 WO 2011047343A2 US 2010052951 W US2010052951 W US 2010052951W WO 2011047343 A2 WO2011047343 A2 WO 2011047343A2
Authority
WO
WIPO (PCT)
Prior art keywords
video
frame time
eye
eye image
display
Prior art date
Application number
PCT/US2010/052951
Other languages
French (fr)
Other versions
WO2011047343A3 (en
Inventor
James Mentz
Samuel Caldwell
Original Assignee
Bit Cauldron Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bit Cauldron Corporation filed Critical Bit Cauldron Corporation
Publication of WO2011047343A2 publication Critical patent/WO2011047343A2/en
Publication of WO2011047343A3 publication Critical patent/WO2011047343A3/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/332Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/341Displays for viewing with the aid of special glasses or head-mounted displays [HMD] using temporal multiplexing
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B30/00Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
    • G02B30/20Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes
    • G02B30/22Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type
    • G02B30/24Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images by providing first and second parallax images to an observer's left and right eyes of the stereoscopic type involving temporal multiplexing, e.g. using sequentially activated left and right shutters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/398Synchronisation thereof; Control thereof

Definitions

  • the present invention relates to stereoscopic 3D image viewing methods and apparatus. More particularly, the present invention relates to methods and devices that provide output to stereoscopic 3D image viewing devices.
  • One approach has been with the use of polarized glasses, where the left and right lenses have a fixed orthogonal polarization (e.g. clockwise-circular and counter-clockwise- circular polarization).
  • the inventors of the present invention have determined that such systems have a number of drawbacks.
  • One such drawback includes that such systems typically rely upon images provided by a light projector and thus such systems are limited for use in darkened environments.
  • Another drawback includes that such systems typically reply upon expensive silver or metalized reflective screens, that maintain the appropriate polarization of light from the projector to the right and left eye images. Such screens are often too expensive for the average consumer.
  • Additional drawbacks include that both left and right eye images are displayed to the user at the same time and polarizers are often imperfect, further, light can change polarization when it reflects off a screen, accordingly, despite the polarized glasses, right eye images are often visible to the left eye and left eye images are often visible to the right eye. This light pollution degrades the quality of the 3D images and can be termed as "ghosting" of 3D images.
  • Some techniques may include deliberate degradation of left eye images to account for right eye image ghosting and the deliberate degradation of right eye images to account for left eye image ghosting.
  • the inventors believe that such techniques are disadvantageous as they tend to reduce the contrast of objects in an image, and they may result in a visible halo around objects in the image.
  • 3D versions of features often do not appear as aesthetically pleasing as 2D versions of such features.
  • FIG. 1 Another approach to 3D visualization has included the use of stereoscopic shutter glasses that are based upon physical shutters, or more commonly liquid crystal display (LCD) technology.
  • LCD liquid crystal display
  • left and right images are alternatively displayed to the user, and the right and left LCD lenses alternate between a dark and transparent state.
  • the shutter glasses quickly alternate between transparency in the left then right eyes in synchronization with an image which presents alternating left and right points of view, the left and right eyes receive different 2D images and the observer experiences the perception of depth in a 3D image.
  • FIG. 1 illustrates a typical home user stereoscopic system.
  • such systems typically include a computer 10 including a display interface, such as a VGA port, or the like.
  • a graphics processing unit or graphics subsystem of computer 10 provides video signals via a VGA port, typically in the form of alternating left and right frames that make up alternating stereoscopic frame video signal.
  • the data rate e.g. pixel rate is relatively high.
  • the video signals are transmitted through the lenses to the viewer as the visible light sensed by the viewer.
  • the second signal sent from computer 10 is sequence information which provides information regarding when the transmission of a new frame is commencing and whether the new frame is a left frame or right frame.
  • This data is typically transmitted along display port lines at a relatively low rate, e.g. 60 Hz.
  • computer 10 / GPU must specifically determine and output both signals along VGA port at different data rates.
  • a dongle 30 is an adapter placed in-line with VGA port acts as a short VGA cable extender.
  • video and sequenced signals are determined by computer 10 / graphics processing unit (GPU) and are multiplexed together by the graphics processing unit (GPU) or graphics subsystem.
  • the signals are received at VGA port 20 by dongle 30 and are demultiplexed into separate video and sequence signals.
  • the video signal is passively passed through dongle 20 on male VGA connector 40 and is allows to propagate on to the display 50.
  • the right / left sequence information is extracted from the plug-and-play information present on the I2S bus (also known as the DDC2 bus) on the video cable.
  • the sequence information is then made available to electrical connector 60 which can be directly cabled to shutter glasses 70 or to a connector 80 following the 1997 VESA Connector & Signal Standards for Stereoscopic Display Hardware standard.
  • connector 80 may be coupled to an infra-red transmitter 90 which sends infra-red light to shutter glasses 95.
  • an IR transmitter In response to the signals from connector 60, an IR transmitter outputs infrared data that indicate when the right eye image is being output and when the left eye image is being output.
  • infrared data For example, one simple format for infrared data is a simple square wave with a high signal indicating left and a low signal indicating right; and another format includes a 8-bit word. Because of these different data formats, IR transmitters from one manufacturer often cannot be used with LCD glasses from another manufacturer.
  • infrared data is received by LCD glasses 95, and in response, for example, the right LCD of the LCD glasses 95 becomes opaque and the left LCD becomes translucent (e.g. clear,), or the left LCD of the LCD glasses 95 becomes opaque and the right LCD becomes translucent.
  • the right LCD becomes translucent display 50 is displaying a right eye image
  • the left LCD becomes translucent display 50 is displaying a left eye image.
  • shutter glasses 70 A similar process is performed by shutter glasses 70.
  • One such limitation includes the difficulty in synchronizing the glasses to the images that are displayed. Synchronization data is typically based upon when the images are provided to the 3D display. Limitations to such approaches, determined by the inventors includes that both latency and timing jitter are introduced as it is processed and rendered by the 3D display device. In various embodiments, jitter as little as 50 microseconds or 10 microseconds can affect the performance of the glasses. As a result of such latency and jitter information, the LCD lenses or shutters are often opened and closed often at improper times, e.g. out of phase, with some of the image intended for the left eye being shown to the right eye and vice versa. This is perceivable by the user as light pollution or ghosting effects.
  • phase difference is not constant and is subject to jitter
  • the user may see the image brightness change or flicker undesirably.
  • One approach to reduce such latency or jitter effects has been to reduce the amount of time the left LCD shutter and the amount of time the right LCD shutter are translucent.
  • the left shutter instead of the left shutter being open for example 50% of the time, the left shutter may be open 35% of the time, or the like. This reduction in open time should reduce the amount of ghosting.
  • the inventors recognize drawbacks to such approach to image ghosting.
  • One such drawback is the reduction in net amount of light transmitted to the user's eyes.
  • the user will perceive a darkening of the images for each eye. Accordingly, a 3D version of a feature will appear darker and duller compared to a 2D version of the feature when using shutter glasses.
  • IR LCD glasses based upon IR receivers often lose synchronization with the display as a result of stray reflections. For example, it has been observed by the inventors that IR LCD glasses may become confused as a result of sunlight reflecting from household objects; heat sources such candles, open flames, heat lamps; IR remote controls (e.g. television remotes, game controllers); light sources (e.g. florescent lights); and the like. Additionally, it has been observed by the inventors that IR LCD Glasses may also lose synchronization as a result of clothing, hair, portions of other users bodies (e.g. head), or the like, that temporarily obscure an IR receiver of the LCD glasses. The loss of synchronization may lead the user to seeing a series of flickering or rolling images and / or the left eye seeing the right eye image. The inventors believe these types of anomalies are highly disturbing to most users and should be inhibited or minimized.
  • manufacturers of such devices specifically instruct users to use IR LCD glasses in highly controlled environments. For example, they suggest that the 3D displays and glasses be used only in darkened rooms. The inventors believe such a solution limits the applicability and attractiveness of such 3D display devices to typical consumers. This is believed to be because most consumers do not have the luxury of a dedicated, light- controlled room for a home theater, and that most consumer entertainment rooms are multipurpose family rooms.
  • each 3D display system includes its own IR transmitter and 2D field timing and phase data. Then, when two such systems are in close proximity, a user's IR LCD glasses may receive IR transmissions from either of the 3D display systems. Because of this, although a user is viewing a first 3D display, the user's 3D glasses may be synchronizing to a different 3D display, causing the user to undesirably view flickering and rolling images.
  • the inventors of the present invention thus recognize that multiple 3D display systems cannot easily be used in applications such as for public gaming exhibitions, tournaments, or contests, trade shows, in stadiums, in restaurants or bars, or the like.
  • 3D source e.g. computer, DVD player, or the like
  • computer 10 / graphics processor specifically generates the switching signals and outputs these signals on VGA port 20.
  • the inventors believe this is undesirable as it requires sources of 3D images, e.g. PS3, to be modified in firmware, or physically, to output such switching information.
  • the present invention relates to stereoscopic 3D image viewing methods and apparatus. More particularly, the present invention relates to stereoscopic 3D image viewing devices incorporating robust synchronization capability.
  • a stereoscopic 3D image viewing device is based upon liquid crystal display (LCD) shutters that are synchronized to a source of 3D images.
  • the synchronization is based upon RF protocols such as Bluetooth, ZigBee radio (ZigBee Alliance), Z-Wave, IEEE Standard 802.11, IEEE Standard 802.15.4, or any other type of RF communications protocol.
  • the stereoscopic 3D image viewing device may transmit data back to the source of 3D images, via the RF communications mechanism or protocol, to increase the level of synchronization between the two devices.
  • a multitude of communications protocols e.g.
  • a system, method, and apparatus of perceiving stereoscopic 3D can be generated which improves the level of synchronization between the alternating images and the alternating action of shutter glasses.
  • a system, apparatus, method, and computer-readable media are provided to enable stereoscopic viewing.
  • the physical method of connecting the display system to stereoscopic glasses is the IEEE 802.11 wireless radio, , IEEE 802.15.4 wireless radio, ZigBee radio, Z-Wave, or Bluetooth technology. This allows a user to move one's head into positions that would normally lose reception of wireless transmissions (e.g. IR) thus simplifying the user experience of wearing stereoscopic glasses.
  • the wireless radio connection also has the advantage of replacing the infra-red light transmission method and its associated interference with remote controls and tendency to accept interference from natural and artificial light sources, thus enhancing the user experience.
  • a shutter glasses control timer and multi-layer timer feedback loop are provided to 3D glasses for improved stereoscopic viewing.
  • the control timer and multi-layer timer feedback loop operate the liquid crystal shutter action of the 3D glasses.
  • these components utilize the 3D source synchronization signal (e.g. system), in one example the VESA signal, along with RF- based communications mechanisms, as discussed herein, e.g. IEEE 802.15.4 wireless radio.
  • the RF-based communications channel between the display system and the 3D stereoscopic glasses allows a user to move his head into positions and to locations that would normally cause loss of reception of 3D glasses based upon infrared transmissions.
  • the shutter control timer and multi-layer feedback loop improves the three dimensional perception by eliminating jitter and noise in the system (3D source) synchronization signal.
  • the shutter control timer and multi-layer feedback loop of the 3D glasses can quickly synchronize with the system synchronization signal and can maintain the
  • synchronization may be temporarily lost. Such embodiments improve the user's 3D experience.
  • such shutter control timer includes hardware based upon a microprocessor in the LC shutter glasses.
  • the microprocessor receives the timing information (e.g. system synchronization signals) received from the 3D system synchronization source via wireless signal and the feedback loop synchronizes the localized control timer within the 3D glasses with the system synchronization signal.
  • the timer control system in the 3D glasses Based upon the localized clock, in the short term absence of input synchronization information or in short periods of high signal jitter, the timer control system in the 3D glasses does not adjust the frequency of phase of the LCD switching, and relies upon its own internal clock.
  • a method for synchronization between the video transmitter and the shutter glasses. Synchronization is provided via a protocol that provides timing information such as a beacon offset or any series of packets that is used as the energy to excite a clock. A precision timing protocol may be utilized to provide synchronization between the transmitter and the receiver.
  • the same right-eye image is output to a video display during two or more successive frame times, for example.
  • the image is transitioning between a left-eye image and the right-eye image, and thus the right-eye lens of the 3D glasses is set to an opaque state.
  • the image is transitioning from the right-eye image to the right image, and thus appears stable.
  • the right eye lens of the 3D glasses is set to a nonopaque state. The process may be repeated for the left-eye image.
  • Various embodiments of the present invention may display an image more than two times, as the refresh rate of the video display increases. In such embodiments, the brightness further increases.
  • Other embodiments of the present invention may be adapted to display other images than a left / right image pair, such as a 2D image, or different left / right image pairs.
  • the shutter glasses and the transmission device may include executable computer programs resident in a memory that instructs a respective processor to perform specific functions or operations, such as to transmit data, to determine a latency, or the like.
  • a method for operating a pair of shutter glasses including a right LCD shutter and a left LCD shutter includes receiving synchronization data via radio frequency transmissions from a radio frequency transmitter, and determining shutter timings for the right LCD shutter and the left LCD shutter in response to the synchronization data.
  • a technique may include applying the shutter timings to the right LCD shutter and the left LCD shutter to enable the viewer to view right-eye images via the right LCD shutter and left-eye images via the left LCD shutter.
  • a method for transmitting stereoscopic display information includes: converting one or more video synchronization signals into wireless radio signals; and decoding the wireless radio signal in a pair of shutter glasses or other device; wherein the wireless radio is the IEEE Standard 802.1 1, WiFi, or components thereof.
  • a method for transmitting stereoscopic display information includes: a pair of shutter glasses or other consumer electronics device which contains a localized clock such that the device remains synchronous to the video display system even when the connection to the source transmitting the synchronization information is interrupted or is not present.
  • the synchronization information between the display system and the glasses or other device are determined by a precision timing protocol in which bidirectional communication of timing information occurs.
  • a method for transmitting stereoscopic display information includes: a pair of shutter glasses or other consumer electronics device which receives synchronous information from the video display system, and a means and method of storing the delay and synchronization information in the transmitter or the video source generating computer, home theater system, or device.
  • the delay and synchronization information are stored and then transmitted to multiple devices to allow multiple users to simultaneously use the same system.
  • a method for transmitting stereoscopic display information includes: a pair of shutter glasses or other consumer electronics device which receives synchronous information from the video display system, and a means of determining the delay and synchronization information through information contained in the display and transmitter from the display via the video signal cable.
  • a method for transmitting stereoscopic display information including: a transmitter of synchronization information and a pair of shutter glasses or other consumer electronics device which is capable of receiving synchronization information from infrared, visible light and radio sources.
  • the shutter glasses or other receiving device can incorporate a computer program which allows the device to automatically determine which source or sources of
  • synchronization information are available and automatically use the best source or sources.
  • shutter glasses includes various radio frequency receiving capability along with a feedback mechanism and a localized clock.
  • the introduction of a synchronized timer in the shutter glasses improves the synchronization between the alternating source images and the alternating action of shutter glasses . It is with respect to these considerations that a LC shutter control timer and multi-layer timer feedback loop are provided for improved perception of stereoscopic 3D viewing.
  • a method of combining stereoscopic glasses with a wireless headset, Bluetooth headset, or stereo Bluetooth headset is disclosed.
  • a method of combining ordinary or automatically darkening sunglasses with a wireless headset, Bluetooth headset, or stereo Bluetooth headset is disclosed.
  • a three-dimensional viewing device for providing images to a user.
  • One apparatus includes a receiver configured to receive source 3D synchronization signals from a transmitting device, wherein the source 3D synchronization signals comprise a source frequency and a source phase.
  • a device may include a plurality of LCD shutters including a right LCD shutter and a left LCD shutter, wherein the right LCD shutter and the left LCD shutter are configured to alternatively enter a translucent state in response to local 3D synchronization signals.
  • a system may include a localized timing source coupled to the receiver and to the plurality of LCD shutters, wherein the localized timing source is configured to generate the local 3D synchronization signals in response to the source 3D synchronization signals, and an adjustment portion coupled to the localized timing source and to the receiver, wherein the adjustment circuit is configured to adjust parameters of the local 3D synchronization signals in response to parameters of the source 3D synchronization signals.
  • a method for operating a three-dimensional viewing device including a right LCD shutter and a left LCD shutter includes receiving source 3D synchronization signals from a transmitting device, wherein the source 3D synchronization signals comprise a source frequency and a source phase, and generating a local 3D synchronization signals in response to the source 3D synchronization signals.
  • a process may include adjusting parameters of the local 3D synchronization signals in response to parameters of the source 3D synchronization signals, and driving the right LCD shutter and the left LCD shutter with the local 3D synchronization signals, wherein the right LCD shutter and the left LCD shutter are configured to
  • a device for driving a pair of three- dimensional glasses includes an interface portion configured to receive 3D video output signals from a source of 3D video signals, and a decoding portion configured to determine a plurality of synchronization signals in response to the 3D video output signals.
  • a system may also include a radio frequency transmitter configured to output radio frequency signals to the pair of three-dimensional glasses in response to the plurality of synchronization signals.
  • a method for driving a pair of three- dimensional glasses includes receiving 3D video output signals from a source of 3D video signals, and determining a plurality of synchronization signals in response to the 3D video output signals.
  • a process may include outputting radio frequency signals to the pair of three-dimensional glasses in response to the plurality of synchronization signals.
  • a method for providing three-dimensional video images to a user on a display system programmed to implement the method includes while a right-eye lens of three dimensional glasses of the display system is in an opaque state, transferring with a video driver of the display system a first instance of a left-eye image to a video display device during a first frame time, while a left-eye lens of three-dimensional glasses of the display system is in an opaque state, and transferring with the video driver of the display system a second instance of the left-eye image to the video display device during a second frame time, while the left-eye lens of the three-dimensional glasses of the display system is in a non-opaque state, wherein the first frame time and the second frame time are adjacent.
  • a process includes while the left-eye lens of the three dimensional glasses of the display system is in the opaque state, transferring with the video driver of the display system a first instance of a right-eye image to the video display device during a third frame time, while the right-eye lens of the three-dimensional glasses of the display system is in the opaque state, and transferring with the video driver of the display system a second instance of the right-eye image to the video display device during a fourth frame time, while the right-eye lens of the three-dimensional glasses of the display system is in a non-opaque state, wherein the third frame time and the fourth frame time are adjacent.
  • a display system for providing three- dimensional video images to a user.
  • One apparatus includes a pair of three dimensional glasses having a right-eye lens configured to have an opaque state or a nonopaque state in response to right-eye timing data, a left-eye lens configured to have an opaque state or a non-opaque state in response to left-eye timing data, and a wireless communication receiver configured to receiving timing information from a video processor.
  • a pair of three dimensional glasses may include a processor coupled to the right-eye lens, the left-eye lens, and to the wireless communication receiver, wherein the processor is configured to receive timing information from the wireless communication receiver, wherein the processor is configured to provide left-eye timing data to the left-eye lens for a first frame time and a second frame time in response to the timing information, and wherein the processor is configured to provide right-eye timing data to the right-eye lens for a third frame time and a fourth frame time in response to the timing information, wherein the first frame time and the second frame time are sequential, and wherein the third frame time and the fourth frame time are sequential.
  • a device may include a video processor coupled to the pair of three dimensional glasses and to a video source including a processor configured to determine the timing information in response to video data from the video source, and a wireless communication transmitter configured to transmit the timing information to the pair of three dimensional glasses.
  • a video display system for driving a video display and a pair of 3D glasses.
  • One device may include a sequence memory configured to store an output image sequence, and a first double-buffer memory, wherein the first double-buffer memory is configured to simultaneously store and output a first left-eye image while receiving and storing a second left-eye image.
  • An apparatus may include a second double-buffer memory, wherein the second double-buffer memory is configured to simultaneously store and output a first right-eye image while receiving and storing a second right-eye image.
  • a system may include a sequencer coupled to the sequence memory, to the first double-buffer memory, and to the second double-buffer memory, wherein the sequencer is configured to output the first left-eye image to a video display during a first frame time, to output the first left-eye image to the video display during a second frame time, to output first timing information to instruct a left-eye lens of a pair of 3D glasses to enter an opaque state during the first frame time, and to output second timing information to instruct the left-eye lens of the pair of 3D glasses to enter a non-opaque state during the second frame time, in response to the output image sequence.
  • FIG. l is a block diagrams illustrating aspects of the prior art
  • FIGS. 2A-D include block diagrams of various embodiments of the present invention illustrating the process of elements of a system in which stereoscopic glasses are synchronized with the display device by incorporation of a wireless radio into the system;
  • FIG. 3 illustrates a block diagram of a process according to various embodiments of the present invention
  • FIG. 4 is a timing diagram of various embodiments of the present invention illustrating a method of sending image information to a display in which the frames which compose the image are sent sequentially;
  • FIG. 5 illustrates various embodiments incorporated into a mobile phone's hardware, firmware, and software and into a pair of stereoscopic shutter glasses
  • FIG. 6 illustrates various embodiments incorporated into a mobile phone, some of the methods are incorporated into a pair of stereoscopic shutter glasses, and other methods are incorporated into a cradle or other device that attaches to the mobile phone;
  • FIG. 7 illustrates various embodiments incorporated into a pair of stereoscopic shutter glasses combined with a mobile phone headset
  • FIG. 8A-C illustrate various embodiments of the present invention
  • FIG. 9 illustrates various embodiments of the present invention.
  • FIG. 10 is a block diagram illustrating aspects of the prior art
  • FIG. 11 is a block diagram illustrating various embodiments of the present invention
  • FIG. 12 is a block diagram illustrating various embodiments of the present invention.
  • FIG. 13 is a block diagram illustrating various embodiments of the present invention.
  • FIG. 14 is a block diagram illustrating various embodiments of the present invention.
  • FIG. 15 is a block diagram illustrating various embodiments of the present invention.
  • FIG. 16 is a block diagram illustrating various embodiments of the present invention.
  • FIGS. 2A-D illustrate various embodiments of the present invention.
  • FIGS. 2A-D illustrate various arrangements of embodiments of the present invention.
  • FIG. 2A includes a 3D source 34 of image data, a transmission device 37, a display 43, and shutter glasses 42.
  • 3D source 34 may be a computer, a Blu-ray or DVD player, a gaming console, a portable media player, set-top-box, home theater system, preamplifier, a graphics card of a computer, a cable box, or the like
  • 3D display 43 may be any 3D display device such as an LCD/Plasma/OLED display, a DLP display, a projection display, or the like.
  • transmission device 37 and shutter glasses 42 may be embodied by a product developed by the assignee of the current patent application, Bit Cauldron Corporation of Gainesville, FL.
  • shutter glasses 42 may be implemented with mechanical shutters or LCD shutters. For example, LCD shutters based upon twisted nemic, super-twisted nemic, or pi-cell technology, may be used.
  • 3D source 34 sends 3D display signals to display 43 through a video cable 35, typically through a standards-based interface such as VGA, DVI, HDMI, Display Port (DP), or the like.
  • Such 3D display signals are often configured as one or more interleaved full right-eye images then full left-eye images (e.g. field sequential); double wide (e.g. side by side) or double height (e.g. stacked) images including both left and right images; images interleaved with right-eye images and left-eye images on a pixel by pixel basis; or the like.
  • a transmission device 37 e.g. a radio transmitter may be inserted between the 3D source 34 or other video source and 3D display 43.
  • transmission device 37 determines 3D timing information by decoding the 3D display signals as they pass through to display 43 on signal line or cable 44.
  • transmission device 37 includes a transmitter based upon radio frequency (RF) signals.
  • the RF signals may use or may be combined with any conventional transmission protocol such as IEEE Standard 802.15.1 (e.g. Bluetooth), IEEE 802.11 (e.g. Wi-Fi), IEEE Standard 802.15.4 (e.g. ZigBee Alliance radio), Z-Wave, or the like.
  • synchronization signals 46 are then transmitted via antenna 39.
  • transmission device 37 may be a stand-alone device, e.g. a dongle, a USB "key,” or the like and transmission device 37 may be powered by power source 36 and 38, self-powered, powered from 3D data source, USB powered, or the like. In other embodiments, transmission device 37 may incorporated into another device, such as 3D source 34, 3D display 43, a pre-amplifier, or the like.
  • FIG. 2B illustrates additional embodiments of the present invention.
  • Fig. 2B includes a source of 3D images 100, a transmission device 1 10, and a 3D display 120.
  • 3D image source 100 provides 3D images (e.g. double-wide or double- height images including both right and left images) to 3D display 120 via a signal line 130 such as a VGA, DVI, Display Port (DP), cable, or the like.
  • 3D image source 100 provides a synchronization signal along signal line 140 to transmission device 1 10.
  • 3D image source 100 includes an industry standard interface such as a VESA miniDIN-3 connector, VESA 1997.1 1, USB connector, or the like, to which transmission device 1 10 may be coupled.
  • FIG. 2C illustrates additional embodiments of the present invention.
  • Fig. 2C includes a source of 3D images 160, a transmission device 170, and a 3D display 180.
  • 3D image source 160 provides 3D images (e.g. double-wide or double- height images including both right and left images) to 3D display 180 via a signal line 190 such as a VGA, DVI, HDMI cable, Display Port (DP), or the like.
  • 3D display 180 provides a synchronization signal along signal line 200 to transmission device 170.
  • 3D display 180 includes an industry standard interface such as a VESA miniDIN-3 connector, USB connector, or the like, to which transmission device 170 may be coupled.
  • FIG. 1C includes an industry standard interface such as a VESA miniDIN-3 connector, USB connector, or the like, to which transmission device 170 may be coupled.
  • FIG. 2D illustrates other additional embodiments of the present invention.
  • Fig. 2D includes a source of 3D images 220, a transmission device 230, and a 3D display 240.
  • 3D image source 220 provides 3D images (e.g. double-wide or double-height images including both right and left images) to 3D display 240 via a signal line 250 such as a VGA, DVI, HDMI cable, Display Port (DP), or the like.
  • a signal line 250 such as a VGA, DVI, HDMI cable, Display Port (DP), or the like.
  • transmission device 230 may be disposed within 3D display 240.
  • transmission device 230 may be installed within the manufacturing facility of 3D display 240, or the like.
  • 2D display 240 may also power transmission device 230.
  • 3D display 240 provides a (derived) synchronization signal along signal line 260 to transmission device 230.
  • shutter glasses 42 include a radio receiver 41 that receives the synchronization signals 40.
  • shutter glasses 42 alternatively changes the properties of one lens from translucent to opaque (e.g. dark) to translucent and of the other lens from opaque to translucent (e.g. clear) to opaque.
  • a user / viewer views 3D display images 45 from display 43 at the proper timing. More particularly, the user's right eye is then exposed to a right-eye image from 3D display images 45, and then the user's left eye is then exposed to a left-eye image from 3D display images 45, etc.
  • transmission device 37 based upon a radio frequency transmitter has several advantages over an infrared transmitter.
  • One advantage recognized is that radio signals can be received in many situations where an infrared signal would be blocked. For example this allows the user of a pair of 3D shutter glasses or the like, to move their head much farther away from the 3D display or transmission device than if IR were used, and allows the user to move throughout a room with a larger range of motion while maintaining synchronization with the 3D display.
  • RF transmitters allow other people or objects to pass in front the user / viewer without interrupting the signal.
  • shutter glasses 42 may include its own localized clock. Benefits to such a configuration include that it allows shutter glasses 42 to remain approximately synchronized to display 43 even though the connection to transmission device 37 is interrupted and / or synchronization signals 40 are not received.
  • a precision timing protocol can be used so that the clock that is local to shutter glasses 42 is synchronized with a clock within transmission device 37 and / or the 3D display signals.
  • a precision timing protocol may include the transmission of data packets with a time stamp time associated with the 3D display signals to shutter glasses 42.
  • the protocol may include transmission of a data packet with a time stamp associated with shutter glasses 42 to transmission device 37.
  • shutter glasses 42 receive the time stamp from the 3D data source, compares the received time stamp to its local clock and returns a data packet with its local time stamp. Using this information, transmission device 37 can determine a round-trip time for data between transmission device 37 and shutter glasses 42.
  • the round-trip time offset is evenly divided between transmission device 37 and shutter glasses 42.
  • one or both devices are capable of determining a difference in speed or lag between the two transmissions, then a more precise determination of the relative values of both clocks (offsets) can be determined. As a result, in various embodiments, more precise synchronization between the two clocks can be established.
  • the difference in rate (e.g. frequency) between the two clocks (transmission device 37 or 3D source 34 and shutter glasses 42) can be more precisely determined.
  • the period of time between the determination of a latency process may be made small, e.g. once a minute; and if there is a higher degree of consistency in the latencies, the period of time between the determination of a latency process may be increased, e.g. once every ten minutes.
  • Embodiments of the present invention enable the use of multiple pairs of shutter glasses 42.
  • a single pair of shutter glasses 42 may be used to determine delay and jitter as discussed was discussed above.
  • a simpler protocol such as a unidirectional or broadcast protocol, may be used by transmission device 37 to communicate this synchronization information to the remaining pairs of shutter glasses.
  • the delay and jitter information can be stored in transmission device 37, in 3D source 34, or other consumer electronics device generating the 3D data, either in a volatile or non-volatile manner.
  • other methods can be used to determine the synchronization and delay information.
  • this data may be determined using bidirectional communications on cable 44, such as the DisplayPort protocol, or the like, as illustrated in FIG. 2C.
  • Communications protocols such as display data channel (DDC and DDC2) protocols, PanelLink serial protocol or a similar protocols allows the display to communicate information back to the computer, home theater system, video source, or the like.
  • this serial protocol can be enhanced to provide the appropriate latency and synchronization characteristics of 3D display 43 back to 3D source 34 and / or transmission device 37.
  • these protocols can be used to determine the manufacturer, vendor, or other identifying information for 3D display 34, and a table of pre-determined synchronization information can be retrieved, either locally, across a local area network, across a network, or the like This information may include an appropriate delay and synchronization information for respective 3D displays.
  • FIG. 3 illustrates a block diagram of a process according to various embodiments of the present invention. More specifically, FIG. 3 illustrates a process for synchronizing shutter glasses to a source of 3D images.
  • a 3D data source provides 3D images, step L.
  • the 3D images may be provided in any number of specific formats, such as right and left images: sequentially transmitted, packed vertically or horizontally into a single image and transmitted, combined on a pixel by pixel basis into a single image and transmitted, or the like.
  • 3D data source may provide specific timing data.
  • synchronization data such as an identifier of a timing clock resident on 3D data source is determined, step 310.
  • this may include a packet of data including a source time stamp, or the like.
  • the synchronization data may then be transmitted through radio frequency transmissions to a first pair of shutter glasses, step 320.
  • the shutter glasses receive the source time stamp and synchronizes the operation of the right / left shutters to the synchronization data, step 330.
  • the synchronization data can then be maintained within the shutter glasses by an internal clock within such glasses, step 340.
  • the internal clock can be resynchronized.
  • Such embodiments are believed to be advantageous as the glasses need not wait for synchronization data from the 3D data source to be able to switch.
  • synchronization data from the transmission device may be dropped or lost while the shutter glasses continue to operate properly.
  • synchronization data is reestablished, the synchronization described above may be performed.
  • RF communications using the ZigBee radio occur at 2.4 GHz, the same band as most Wi-Fi transmissions.
  • embodiments of the present shutter glasses are designed to inhibit communications, and defer to such Wi-Fi signals.
  • the shutter glasses will continue to operate autonomously, until the interference stops and new synchronization data is received from the transmission server.
  • the shutter glasses may transmit data back to the RF transmission device. More specifically, the shutter glasses may transmit the received source time stamp and / or the glasses time stamp back to the transmission device via the same RF communications channel, or the like, step 350.
  • the transmission device may determine adjustments to subsequent synchronization data that will be sent to the shutter glasses, step 360.
  • the transmission device may determine that it should output synchronization data to the shutter glasses, even before the synchronization data is determined or received from the 3D data source.
  • the shutter glasses may trigger its shutters 100 microseconds before the expected arrival of a synchronization pulse.
  • synchronization data may be used to drive 3D glasses of other viewers of the 3D image.
  • 3D glasses of other viewers in the room may also have synchronization data adjusted using the process described above.
  • the transmission device may output the synchronization data at different times for different 3D glasses.
  • the shutter glasses may verify that they are in sync. If not, the shutter glasses may adjust the frequency of its own internal clocks until they are kept in a higher amount of synchronization.
  • the process may be repeated.
  • the synchronization process may be performed periodically, with the period dependent upon how well the 3D data source and the shutter clock stays remain in synchronization - if highly synchronized, the synchronization process may be performed at longer time periods apart (e.g. 2 minutes) than if these devices continually have synchronization problems (e.g. every 10 seconds). Further detail regarding the above synchronization process may be found in the provisional application referenced above.
  • Various embodiments of the present invention may include shutter glasses or other devices that includes multiple physical methods for receiving synchronization information.
  • some embodiments may contain both an infrared and radio receiver; an infrared and visible light receiver; a radio or visible light receiver; a combination of infrared, visible light and radio receivers; or the like.
  • the shutter glasses or other receiving device may include executable computer program that instructs a processor to automatically determine which communications channel or channels are available, and automatically use the communications channel having the strongest signal, lowest number of dropped data packets, or the like.
  • the combination of a visible light receiver (e.g. IR) with another synchronization transmission technology (e.g. RF) may be advantageous. More specifically, the information transmitted via visible light and the synchronization information transmitted via another transmission technology may be combined within shutter glasses 42 to deduce unknown elements of the delay in 3D display 43 and other synchronization information.
  • the data from the different communication channels are compared to more precisely synchronize 3D display 43 and shutter glasses 42.
  • the two communications channels can be used to verify that a left image displayed on 3D display 43 is going to the left eye and the right image displayed on 3D display 43 is going to the right eye. In such an example, this would preventing the error of a reversal of synchronization information somewhere in the system that results in the sending the left image to the right eye and vice versa.
  • shutter glasses 42 may be used to provide a variety of new functions.
  • FIG. 4 illustrates typical video output timing where frame one 26, frame two 28, frame three 30 and frame four 32 are output sequentially.
  • left images (frames) and right images (frames) are alternatively output.
  • frame one 26 is left
  • frame two 28 is right
  • frame three 30 is left
  • frame four 32 is right, creating the sequence L, R, L, R images to the user.
  • Various embodiments of the present invention may be applied to 3D displays having display rates on the order of 120Hz and higher.
  • the refresh rate is 120 Hz
  • right and left images will be displayed and refreshed at 60 Hz. Accordingly, the viewer should not be able to detect significant flickering, however, the viewer may detect a darkening of the images.
  • the higher refresh rate may enable new features, as described below.
  • more than one left image and right image may be output.
  • multiple viewers may view a 3D display, and different viewers may see different 3D images.
  • a two viewer sequence of output images may be user 1 left, user 1 right, user 2 left, user 2 right, etc. This could be represented as: LI, Rl, L2, R2.
  • shutter glasses of a first viewer will allow the first viewer will see images LI and Rl and a shutter glasses of a second viewer will allow the second viewer will see images L2 and R2.
  • other sequences are contemplated, such as LI, L2, Rl, R2, and the like.
  • refresh rate for a 3D display having a 240 Hz refresh rate, a viewer will see the respective right and left images at a refresh rate of 60Hz. As noted above, this frequency should be above the typical sensitivity of the eye, however, viewers may detect a darker image. Such artifacts may be mitigated by increasing the brightness of the images, or the like.
  • FIG. 1 Other embodiments may be extended to additional (e.g. three or more viewers).
  • Applications of such embodiments may include for computer or console gaming, or the like.
  • two or more viewers may initially see the same 3D image, and subsequently one or more viewers "break off to view a different 3D image.
  • three people could be playing a multiplayer game in which all three are traveling together and see the same 3D images.
  • one player then breaks away from the other players.
  • the player's glasses can be reprogrammed to allow the third person to see a different 3D image. Subsequently, the third person may return to the group, and then see the same 3D image.
  • a sequence of images output by the 3D display could begin with L0-R0-L0-R0, where 0 indicates everyone in the party.
  • the 3D display could switch and output images in a sequence such as L1&2, R1&2, L3, R3; L1&2, L3, R1&2, R3; or the like.
  • the sequence may revert to L0, R0, L0, R0.
  • switching back and forth may occur with little, if any, visible interruption in the 3D images viewed by the viewer.
  • the inventors recognize that the brightness of each frame may have to be adjusted to correct for the changes in overall viewing time.
  • sequences of images enable still other types functionality.
  • one sequence of frames can be sent such that viewers wearing 3D glasses see a stereo display and viewers without glasses see only one side of the image (e.g. left or right).
  • a three frame sequence may include: Left, Right, Left- minus-Right.
  • a user using embodiments of the present invention may see a stereoscopic image by viewing the left image in their left eye and the right image in their right eye. That user would be prevented from viewing the Left minus Right image.
  • L, R, (L-R) 2L, or only the left image with both eyes.
  • separate anti-left, anti-right images or both may also be sent.
  • theater-goers can decide whether they care to watch the same movie or feature with or without 3D glasses; game players can play in 3D while viewers watch the same display in 2D.
  • users not utilizing embodiments of the 3D glasses may view other arbitrary images.
  • the viewer with 3D glasses may see the left image in the left eye and the right image in the right eye, and may not see the Arbitrary image.
  • FIG. 5 illustrates additional embodiments of the present invention. More specifically, FIG. 5 illustrates a general purpose consumer device (e.g. mobile phone, personal media player, laptop, or the like) capable of 3D image output.
  • a general purpose consumer device e.g. mobile phone, personal media player, laptop, or the like
  • 3D image output e.g. 3D image output
  • the synchronization information to the shutter glasses may be provided by with the consumer device including embodiments of the RF transmitter described above, or unused or available transmitters available in the consumer device.
  • Various examples may use infrared, WiFi, Bluetooth, or the like, to provide synchronization signals to shutter glasses according to embodiments of the present invention.
  • Fig. 6 illustrates additional embodiments of the present invention wherein existing consumer devices (e.g. mobile phone) may be augmented to better support stereoscopic 3D viewing.
  • a cradle or dongle which attaches to the mobile device or holds the mobile device may be used.
  • the cradle or dongle may incorporate a projection system such that the image may be projected at a larger size than the screen on the mobile device.
  • the cradle or dongle or consumer device may also provide the synchronization signals to the shutter glasses.
  • the cradle or dongle may include a ZigBee radio-type transmitter (IEEE 802.15.4) that transmits the synchronization data to the shutter glasses, or the like.
  • IEEE 802.15.4 ZigBee radio-type transmitter
  • stereoscopic shutter glasses that are to be used with the consumer device described above, can be used for other purposes.
  • glasses incorporate a visible light sensor, they can be worn as ordinary sunglasses but make improved automatic decisions about the appropriate level of perceived darkening.
  • This information can be based on computer algorithms, information about the user and the environment that is stored on a mobile device; information retrieved from a computer network via the mobile device, and the like.
  • Fig. 7 illustrates yet another embodiment of the present invention.
  • a user of the consumer device may desire to perform multiple functions at the same time, such as: talk on a Bluetooth headset, view stereoscopic 3D content, and wear sunglasses.
  • Embodiments illustrated in Fig. 7 may include a pair of shutter glasses 57 combined with a pair of sunglasses and a Bluetooth or stereo mobile Bluetooth headset with a left earpiece 58 and a right earpiece 55, or the like.
  • FIGs. 8A-C illustrate various embodiments of the present invention.
  • Figs. 8A-C illustrate block diagrams of various embodiments of a dongle providing RF transmissions, for example, as described above.
  • a physical interface 710 is illustrated.
  • physical interface 710 may be a DVI port, HDMI port, Display Port (DP), USB, VESA 1997.11, or the like, for coupling to a source of 3D data 712(e.g. computer, DVD / BluRay player, HD display, monitor, etc.).
  • a port 712 may drive a display
  • a port 714 may drive a dongle 700.
  • an interface chip or block 720 may provide the electronic interface to physical interface 710. As illustrated, signals including horizontal synchronization signals (HSY C), vertical synchronization signals (VSY C), control signals, and a clock may be derived from the video data.
  • a processing device such as a CPLD (complex programmable logic device) 730 may be used to decode 3D synchronization data from 3D image data or 3D timing data. More specifically in various embodiments, sequence information is multiplexed directly into the video picture data from the source of 3D data 712.
  • the digital picture data is demultiplexed with receiver 720 into its component video and control data streams 722, described above, such that each stream can be examined, or the streams can be examined relative to each other.
  • the sequence information can be further demultiplexed from the digital video information.
  • CPLD 730 further decodes the sequence data.
  • CPLD 24 is useful in various embodiments as the 3D sequence data that is encoded into a digital signal is very fast, typically as fast as 165 MHz. This is much faster compared to the typical 8-32 MHz speed of microprocessor 750, accordingly, microprocessor 750 typically cannot decode this stream of data.
  • Control, VSYNC and HSYNC information, or the like 722 is not transmitted at every moment in time, CPLD 730 and microprocessor 750 further decode the sequence information into continuous information.
  • Control data, VSYNC and HSYNC information are provided at the frequency of the video data (e.g. HDMI, DVI) (e.g. within the range of 32 MHz to 165MHz to 340 MHz, although in the future, the pixel video data frequency may be even larger.
  • the sequence information must be converted to the appropriate left / right frame rate (e.g. 60Hz, 120Hz, 240Hz, 480Hz, or the like.)
  • microprocessor 750 includes an RF interface that references a clock 740.
  • microprocessor 450 is a TI CC2530 System on a Chip, that includes a 8051 MCU (processor), RAM, Flash memory, and a IEEE 802.14.4 ZigBee RF transceiver.
  • the flash memory is configured to store executable computer code or instructions that directs the processor to perform various functions, as described herein.
  • the flash memory includes computer code that directs the processor to transmit the 3D synchronization data to the 3D glasses, to receive timing data back from the 3D glasses, to determine a round-trip communication latency, to adjust 3D synchronization data in response to the round-trip communication latency, and the like, as described above.
  • 3D synchronization data can be sent to the wireless glasses.
  • an additional VESA 1997.1 1 port may also be provided for wired-glasses, as shown.
  • a physical interface 410 is illustrated.
  • physical interface 410 may be a DVI port, HDMI port, Display Port (DP), USB, VESA 1997.11, or the like, for coupling to a source of 3D data (e.g. computer, DVD / BluRay player, HD display, monitor, etc.).
  • a source of 3D data e.g. computer, DVD / BluRay player, HD display, monitor, etc.
  • the 3D data may include 3D image data.
  • Embodiments illustrated in Fig. 8B contemplate a pass-through type dongle, where, for example, a source of 3D data has only one physical interface 410, e.g. DVI port.
  • one or more interface chips or block 420 may provide the electronic decoding interface to physical interface 410. More specifically, similar to Fig. 8A, various data such as HSYNC, VSYNC, control lines, clocks, pixel data, or the like may be decoded by interface chips 420. Additionally, one or more interface chips or block 460 may be provided to reencode the HSYNC, VSYNC, pixel data, and the like into a format appropriate for the physical interface 460. In various embodiments, physical interface 460 may be a DVI port, HDMI port, Display Port (DP), USB, VESA 1997.11, or the like, for outputting data to a 3D display device (e.g. computer monitor, home entertainment monitor, projection device, etc.).
  • a 3D display device e.g. computer monitor, home entertainment monitor, projection device, etc.
  • CPLD 430 is used in similar manner described in Fig. 8 A, to decode data quickly.
  • 3D synchronization data is then provided to an RF interface device 450 that references a clock 440.
  • RF interface device 450 is a TI CC2530
  • the flash memory is configured to store executable computer code or instructions that directs the processor to perform various functions, as described herein.
  • the flash memory includes computer code that directs the processor to transmit the 3D synchronization data to the 3D glasses, to receive timing data back from the 3D glasses, to determine a round-trip communication latency, to adjust 3D synchronization data in response to the round-trip communication latency, and the like, as described above.
  • dongle 400 is illustrated to be placed in-line within a DVI or HDMI signal path and to allow the video signal to behave as if it has passed through dongle 400 undisturbed, when it has in fact been decoded and reencoded.
  • Fig. 8C illustrates another contemplated embodiment, similar to Fig. 8B. More specifically, a dongle 800 with pass-through DVI / HDMI input and output interfaces is illustrated. In Fig. 8C, video signals need not be decoded or reencoded as shown in Fig. 8B. In various embodiments, one or more high impedance devices, such as FET probes 810, may be used to tap the video signals to derive the HSYNC, VSYNC, Control data, or the like. The monitored data can them be processed and output via RF signals to 3D glasses as described herein. The embodiments illustrated in Fig. 8C thus avoid the encoding and decoding processes and hardware of Fig. 8B, illustrated above.
  • FET probes 810 may be used to tap the video signals to derive the HSYNC, VSYNC, Control data, or the like. The monitored data can them be processed and output via RF signals to 3D glasses as described herein.
  • the embodiments illustrated in Fig. 8C thus
  • data wires 820 coming from the source of 3D data includes an I2S clock and I2S data wires. As shown data wires 820 are also passed through dongle 800 to allow the display to communicate Plug and Play and configuration information back through dongle 800 and to the GPU.
  • the I2S bus also called the Display Data Channel 2 or Plug and Play bus, is a multi-drop bus meaning it is electrically capable of supporting communications between three or more devices.
  • the wireless link can now also be recognized by an application on the PC via the Display Data Channel 2. This allows other wireless communications such as upgrading or communicating with wireless shutter glasses, command and control, home automation, or the like.
  • any other wired-type interface may also be provided. Illustrated in these figures is a VESA 1997.11 port.
  • the physical port may be a USB port, a firewire port, or any other custom / proprietary port.
  • the interface may have a standard or proprietary signaling interface, for example, VESA 1997.1 1 specifies a specific signaling protocol. In other embodiments, in light of the present patent disclosure, one of ordinary skill in the art would recognize that a custom or proprietary signaling protocol may be implemented.
  • Fig. 9 illustrates various embodiments of the present invention.
  • Fig. 9 illustrates a block diagram of a pair of shutter glasses 500 according to various
  • Shutter glasses 500 is illustrated to include an RF interface device 510 that references a clock 540 and a pair of electronically controlled LCD shutter elements 520 and 530.
  • 3D synchronization data 550 is received in RF interface device 510.
  • RF interface device 510 is also an TI CC2530 System on a Chip, that includes a 8051 MCU (processor), RAM, Flash memory, and a IEEE 802.14.4 ZigBee RF transceiver.
  • the flash memory is configured to store executable computer code or instructions that directs the processor to perform various functions, as described herein.
  • the flash memory includes computer code that directs the processor to receive the 3D
  • Fig. 10 illustrates a timing diagram of a typical 3D system where stereoscopic viewing is achieved by sending alternating frames intended for viewing by a user's left and right eyes to a display.
  • a 3D display is driven with left-eye images in times 900, 910, etc. and right-eye images in times 920, 930, etc. and synchronized with a pair of electronic shutter 3D glasses.
  • the shutter 3D glasses alternatively turns the lens for each eye from opaque to clear to enable the frames with information intended only for the left eye to be transmitted to the user's left eye in times 940, 950, etc. while blocking these frames from the user's right eye. This process then typically continues vice- versa, such that information intended only for the right eye to be transmitted to the user's right eye in times 960, 970, etc.
  • a display having a 120Hz refresh rate (8.3ms period) and a 180Hz frame rate for a typical frame 900, 5.5ms is used for writing data to the display pixels and waiting for the pixel elements to change states.
  • the left-eye lens is opened in time 940 for only approximately 2.8ms.
  • this method is about 2.8/8.3 or 34% efficient. A consequence of this is that the user will perceive a dim and dull image when viewing 3D images.
  • Fig. 11 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 1 1 illustrates embodiments of a technique named "Crystal Sweep.” In such embodiments, a display is driven twice with each left or right frame or image. [0121] As can be seen in Fig. 1 1, in one example, in the first frame period 1000, the left- eye image is sent to the display, the display sweeps through each row and column of pixels and the display updates the formerly right-eye image with a left-eye image. In the subsequent period 1010, the left-eye image is again sent to the display, the display sweeps through each row and column of pixels and the display redisplays the left-eye image on the display. In such embodiments, in period 1010, as the display device redraws the left-eye image on the screen with an exact duplicate image, no visible distortion to the left-eye image occurs.
  • the left-eye image from a previous period is effectively frozen, or crystallized, onto the display.
  • period 1020 one period 1000 is spent drawing / outputting the left-eye image and one period 1010 is spent with a unique image.
  • this process repeats for the right-eye image, etc.
  • the left-eye shutter is non-opaque during period 1010, accordingly, to the user, the resulting left-eye image is visible 50% of the time allotted to the left-eye image.
  • embodiments of the Crystal Sweep method provide more viewing time than a display illustrated in Fig. 10. This is a significant improvement over the 34% calculated above in the prior art. Accordingly, to a user, using embodiments of the present invention, 3D images will appear brighter and more vibrant. As can be determined, this 50% visibility is independent of a display refresh rate, as exactly half of the periods are unique at any frequency.
  • the inventors of the present invention have also considered the human perception of stereoscopic 3D with regards to flicker. More specifically, if a left or right shutter of a pair of 3D glasses remains closed and the user's eye spends too much time viewing darkness between images, the user may perceive a strobing effect or flickering of the respective images. The inventors believe that such flickering of 3D images will decrease the quality of the user's 3D experience.
  • the interval of darkness 990 between the left eye opening twice is 13.8ms (5.5 (for right drawing) plus 2.8 (for right display) plus 5.5 (for left drawing)). Accordingly, it is expected that few users may experience flickering.
  • the interval of darkness 1030 between the left eye opening twice is in the order of 16.5 ms (5.5 (for right drawing) plus 5.5 (for right redrawing and right display) plus 5.5 (for left drawing)). Accordingly, it is expected that more users may experience flickering.
  • the Crystal Sweep embodiments trade a different point in the 13-16.7 ms flicker perception range for a superior brightness by virtue of the larger percentage of time spent with the lenses being open.
  • Fig. 12 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 12 illustrates embodiments of the present invention implemented upon a display with a quicker frame refresh rate.
  • florescent lighting typically flicker at the electrical system frequency. This lighting flicker would interfere with the television system images, if the television system refreshed at a frequency other than the electrical system frequency or multiple thereof.
  • a display is illustrated having a 240Hz refresh rate (4.2ms period).
  • the left- eye image is also visible to the user 50% of the time allotted to the left-eye image.
  • the interval of darkness 1 100 between the left eye opening twice is 12.6ms (4.2 (for right drawing) plus 4.2 (for right display) plus 4.2 (for left drawing)). Accordingly, it is expected that almost no users may experience flickering. In various embodiments, 3D images will appear brighter and have no perceivable flicker thus providing a user with an enjoyable 3D experience.
  • various methods may be used for the display writing an image in a first time period or frame time and re-writing the image in the second time period or frame time.
  • an additional innovation may include reducing the amount of redundant images that are communicated between a video source and a display.
  • the video source or rendering system may provide one copy or instance of an image for a first time period to a video display system.
  • the video display system may buffer the image, write the buffered image to the display during the first time frame, and rewrite the buffered image to the display during the second time frame.
  • the video source instead of the video source transmitting another instance of the image for the second time frame, the video source need only signal to the video display system that the information during the second frame time is redundant to the first frame time. This can leave the rendering system free to send information to another display, render addition detail for an upcoming image, perform other activities or conserve power, or the like.
  • the video display system can perform other activities or save power by receiving only the required information that the latter frame is redundant to the former frame.
  • this redundancy of frames can be communicated through a plurality of signals, including exerting the VSYNC or HSYNC signals or exerting another digital interface, such as the CONTROL signal in a DVI or HDMI connection, by including information about the anticipation of an upcoming redundant frame in the overscan area of a previous frame, or the like.
  • Other communication paths between the rendering system and the display subsystem are also contemplated.
  • Embodiments may be implemented with the examples in Figs. 21-D may implement the above mentioned features.
  • radio transmitter 37 in Fig. 2A, may provide the image buffering capability. More specifically, image output 34 may output one instance of a left-eye image to radio transmitter 37. In turn, in addition to rf communication with 3D glasses 42, discussed above, radio transmitter 37 may buffer the left-eye image, output one instance of the left-eye image to display 43 during a first frame time, and output another instance of the left-eye image to display 43 during a second frame time. Accordingly, the communication load of image output 34 is reduced.
  • Fig. 13 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 13 illustrates embodiments of Crystal Sweep in addition to embodiments of "Freeze" tag.
  • an image is displayed for multiple consecutive frame times.
  • the video source may provide multiple copies of the image to the display to write during the consecutive frame times.
  • the signaling that indicates that consecutive frames are going to retain identical and redundant information compared to an earlier frame is termed a Freeze tag. Accordingly, when embodiments of the Crystal Sweep technique are combined with the Freeze tag, the plurality of redundant frames can be increased, while reducing the amount of data transfer.
  • embodiments increase the percentage of the time in which each lens can be held open.
  • a display having a 480 Hz display (2.1ms period) is illustrated.
  • a respective lens of the pair of 3D glasses may be open for 75% of the time reserved for the left or right image.
  • the interval of darkness 1200 between the left eye opening twice is 10.5 ms (2.1 (for right drawing) plus 3x2.1 (for right redrawing and display) plus 2.1 (for left drawing)). Accordingly, it is expected that no users will experience flickering. In various embodiments, 3D images will appear very bright with no flicker thus providing a user with an enjoyable 3D experience.
  • Fig. 14 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 14 illustrates embodiments of Crystal Sweep with Freeze tag applied to displaying two separate video streams.
  • a plurality of video streams can be multiplexed such that one or a plurality of viewers can view one video stream while a separate plurality of viewers can receive a second video stream.
  • Each of these streams can be monoscopic or stereoscopic.
  • each monoscopic stream requires only half as many display periods as the stereoscopic streams because each monoscopic frame image is sent to both left and right eyes. More specifically, with half as many frames in time each eye gets the same total number of frames viewed and perceives the monoscopic stream at the same brightness as the stereoscopic stream.
  • the timing may be as follows: draw L, draw/view L, draw/view L, draw 2D, draw/view 2D, draw R, draw/view R, draw/view R, draw 2D, draw/view 2D, etc.
  • draw L draw/view L
  • draw 2D draw/view 2D
  • draw R draw/view R
  • draw/view R draw/view R
  • draw 2D draw/view 2D
  • draw/view 2D etc.
  • the time period 1300 between successive monoscopic images is 6.8ms (1.7 (for right drawing) plus 2x1.7 (for right redrawing and display) plus 1.7 (for monoscopic drawing)).
  • the time period 1310 between successive left or right eye images is 13.6 (1.7 (for monoscopic drawing), 1.7 (for monoscopic displaying), 1.7 (for right drawing), 2x1.7 (for right redrawing and display), 1.7 (for monoscopic drawing), 1.7 (for monoscopic displaying), 1.7 (for left drawing)).
  • the monoscopic images may be advertisements, monoscopic (2D) versions of the feature, text messages, or the like.
  • a video service provider, or the like may allow two users to watch a 3D feature, by enabling appropriate synchronization of two 3D shutter glasses with the output images.
  • Such limiting functionally may be provided by the video display system (e.g. cable box) the radio transmitter, or the like.
  • An additional user whose 3D glasses are not authorized e.g. has not paid for the pay-per view event
  • the video display system and / or the radio transmitter may then begin to provide the appropriate timing signals to the additional user's 3D glasses to view the 3D content.
  • the timing may be as follows: draw 1 st L, draw/view 1 st L, draw 2 nd L, draw/view 2 nd L, draw 1 st R, draw/view 1 st R, draw 2 nd R, draw/view 2 nd R, etc.
  • Freeze tag may be implemented to increase brightness, while keeping flickering low.
  • any number of viewing permutations be simply implemented, in light of the present patent application.
  • two viewers may view different 2D images, again using the Crystal Sweep technique: draw 1 st 2D, draw/view 1 st 2D, draw 2nd 2D, draw/view 2 nd 2D, etc.
  • the refresh frequency in the US, may be 120Hz, 180Hz, 240Hz, 480Hz, 600Hz, or the like (e.g. multiples of 60Hz); in Europe or other 50Hz countries, the refresh frequency may be 100Hz, 200Hz, 400Hz, 800Hz, or the like (e.g. multiples of 50Hz). In various embodiments, these frequencies are approximate.
  • a frame that is being transmitted to the display is often held in one buffer while the video rendering system is generating the video signal for the next frame in a second buffer.
  • This schema is called a double buffer.
  • the double buffer schema can be extended to a plurality of buffers, such as a triple buffer, in which the video rendering subsystem can start rendering a third frame if it completes the second frame and would otherwise need to become idle.
  • Figs. 15 and 16 illustrate various embodiments of the present invention. More specifically, Figs. 15 and 16 illustrate triple buffer embodiments.
  • one double buffer is required for the stream of left frames and a second buffer is required for the stream of right frames.
  • This schema is referred to as a side-by-side double buffer.
  • the sequencing mechanism 1400 that alternates between buffers to send frames to this display must transmit each frame more than once.
  • sequencer 1400 transmits each buffer one or a plurality of times such that the display produces the Crystal Sweep schema.
  • sequencer 1400 also may send a signal indicating that a frame is unchanged from the previous frame, thus transmitting the Crystal Sweep with Freeze tag capability.
  • sequencer 1400 is provided with a sequence pattern 1410 which describes how many frames from each stream are to be sent, which streams are used for sweeping a new image onto the display and which streams are used for crystallizing the existing image.
  • Sequencer 1400 references the double or triple buffers 1420 for the right and left images, and then outputs both the sequence of images or indications of redundant images to the display. For example, two of the same left images, two of the same right images, etc.
  • Sequencer 1400 also outputs the sequence signal to 3D glasses to enable the 3D glasses to properly switch the 3D glasses in the correct image output sequence, for example, close both lenses, open the left, close both lenses, open the right, and the like.
  • sequencer 1510 decodes a plurality of frames from a plurality of buffer stacks 1500 and transmits each to the display or transmits redundant frame signaling information at the appropriate times, as configured by a Sequence Pattern 1520. Similarly, in this example, sequencer 1510 outputs the sequence information to a pair of 3D glasses (to view images Ln and Rn), and to a pair of monoscopic 3D (2D) glasses (to view image "3").
  • the double / triple buffers illustrated in Figs. 15 and 16 may be implemented within a graphics processing unit (GPU), a Blu ray player, a stand-alone interface device, or the like.
  • GPU graphics processing unit
  • Blu ray player a stand-alone interface device
  • 3D glasses may be configured to transmit timing data back to the 3D source.
  • the ability for bidirectional communications also allows a more precise timing protocol to be implemented between the 3D source and 3D glasses.
  • the transmitter (3D source) and the receiver (3D glasses) each run local clocks that operate at a multiple of the frequency (e.g. 20KHz) of the sequence information (e.g. 120Hz) and are divided down to a lower speed which matches the sequence information.
  • the transmitter and the receiver exchange a series of messages at the higher frequency containing timestamps which indicate the value of their local clocks. By exchanging a series of these messages the transmitter and receiver can determine the difference in speed of their local clocks and compensate for these differences. The result is that synchronization is achieved with a much higher precision than the period of the sequence information.
  • a user using a pair of 3D glasses that perceives stereoscopic information from one 3D display should not also perceive stereoscopic information from another 3D display at the same time. This is unless each 3D display is synchronized in time and uses the same left/right sequence. For example, all the 3D displays must start the left frame at the same time and the right frame at the same time. In examples such as an event with multiple monitors, synchronization between these 3D displays may be performed by coordination of the 3D source devices or a single 3D source device.
  • Embodiments described above may be useful for hand-held consumer devices such as cell-phones, personal media players, mobile internet devices, or the like. Other embodiments may also be applied to higher-end devices such as laptop computers, desktop computers, DVRs. BluRay players, gaming consoles, hand-held portable devices, or the like. Other embodiments may take advantage of existing IR transmission devices for IR shutter glasses. More specifically, in such embodiments, an IR to RF conversion portion may be added to receive the IR 3D output instructions and to convert them to RF 3D transmission signals, described above. In some embodiments, an RF receiver is thus used. The RF 3D transmission signals are then transmitted to the RF 3D shutter glasses, described above. Such embodiments can therefore be a simple upgrade to available IR 3D glasses transmitters.
  • feedback from shutter glasses to the transmitter device described above with regards to synchronization may be used for additional purposes.
  • One such embodiment may allow the 3D image source (e.g. a cable box, computer, or the like) to take the indication that a pair of shutter glasses are currently synchronized to mean a person is viewing the 3D content, and to provide that data back to a marketing company such as Media Metrics, Nielsen Ratings, or the like. By doing this, such market research companies may determine the number of viewers of specific 3D features, or the like.
  • a marketing company such as Media Metrics, Nielsen Ratings, or the like.

Abstract

A method for providing 3D video images to a user includes while a right lens of 3D glasses is opaque, transferring a first left image to a display during a first frame, while a left lens of 3D glasses is opaque, and transferring the first left image to the display during a second frame time, while the left lens of the 3D glasses is not opaque, wherein the first frame and the second frame are adjacent, and while the left lens of the 3D glasses is opaque, transferring a first right image to the display during a third frame, while the right lens of the 3D glasses is opaque, and transferring the first right image to the display during a fourth frame, while the right lens of the 3D glasses is not opaque, wherein the third frame and the fourth frame are adjacent.

Description

SYSTEM AND METHOD OF DISPLAYING THREE DIMENSIONAL IMAGES USING CRYSTAL SWEEP WITH FREEZE TAG
BACKGROUND OF THE INVENTION
[0001] The present invention relates to stereoscopic 3D image viewing methods and apparatus. More particularly, the present invention relates to methods and devices that provide output to stereoscopic 3D image viewing devices.
[0002] When two-dimensional images that represent left and right points of view are sensed by respective left and right eyes of a user, the user typically experiences the perception of a 3D image from the two-dimensional images. The inventors are aware of several systems that allow users (e.g. individuals or groups) to perceive stereoscopic 3D depth in images, photos, pictures, moving pictures, videos, or the like, by the selective transmission of images to a users' eyes. Such systems include the use of display systems including light projection / reflection within a public or home theater or emissive or transmissive displays (e.g. LCD, plasma display, flat-panel display, or the like) to alternatively or simultaneously output right eye images and left eye images to a user. To view such 3D images, a variety of approaches have been provided to the user including prisms, static polarized glasses, LCD shutter glasses, or the like. The inventors of the present invention have recognized that existing approaches have many drawbacks, as will be discussed below.
[0003] One approach has been with the use of polarized glasses, where the left and right lenses have a fixed orthogonal polarization (e.g. clockwise-circular and counter-clockwise- circular polarization). The inventors of the present invention have determined that such systems have a number of drawbacks. One such drawback includes that such systems typically rely upon images provided by a light projector and thus such systems are limited for use in darkened environments. Another drawback includes that such systems typically reply upon expensive silver or metalized reflective screens, that maintain the appropriate polarization of light from the projector to the right and left eye images. Such screens are often too expensive for the average consumer. Additional drawbacks include that both left and right eye images are displayed to the user at the same time and polarizers are often imperfect, further, light can change polarization when it reflects off a screen, accordingly, despite the polarized glasses, right eye images are often visible to the left eye and left eye images are often visible to the right eye. This light pollution degrades the quality of the 3D images and can be termed as "ghosting" of 3D images.
[0004] The inventors are aware of a number of techniques that may be used to reduce this ghosting effect. Some techniques may include deliberate degradation of left eye images to account for right eye image ghosting and the deliberate degradation of right eye images to account for left eye image ghosting. The inventors believe that such techniques are disadvantageous as they tend to reduce the contrast of objects in an image, and they may result in a visible halo around objects in the image. As a result of using these circular or linear polarized glasses, the inventors have recognized that 3D versions of features often do not appear as aesthetically pleasing as 2D versions of such features.
[0005] Another approach to 3D visualization has included the use of stereoscopic shutter glasses that are based upon physical shutters, or more commonly liquid crystal display (LCD) technology. With such approaches, left and right images are alternatively displayed to the user, and the right and left LCD lenses alternate between a dark and transparent state. When the shutter glasses quickly alternate between transparency in the left then right eyes in synchronization with an image which presents alternating left and right points of view, the left and right eyes receive different 2D images and the observer experiences the perception of depth in a 3D image.
[0006] Fig. 1 illustrates a typical home user stereoscopic system. As illustrated, such systems typically include a computer 10 including a display interface, such as a VGA port, or the like. In such examples, two separate signals are transmitted from computer 10 downstream on a VGA port. Typically a graphics processing unit or graphics subsystem of computer 10 provides video signals via a VGA port, typically in the form of alternating left and right frames that make up alternating stereoscopic frame video signal. The data rate, e.g. pixel rate is relatively high. The video signals are transmitted through the lenses to the viewer as the visible light sensed by the viewer. The second signal sent from computer 10 is sequence information which provides information regarding when the transmission of a new frame is commencing and whether the new frame is a left frame or right frame. This data is typically transmitted along display port lines at a relatively low rate, e.g. 60 Hz.
Accordingly, computer 10 / GPU must specifically determine and output both signals along VGA port at different data rates.
[0007] As illustrated in Fig. 1, a dongle 30 is an adapter placed in-line with VGA port acts as a short VGA cable extender. As mentioned above, video and sequenced signals are determined by computer 10 / graphics processing unit (GPU) and are multiplexed together by the graphics processing unit (GPU) or graphics subsystem. The signals are received at VGA port 20 by dongle 30 and are demultiplexed into separate video and sequence signals.
[0008] In the present example, the video signal is passively passed through dongle 20 on male VGA connector 40 and is allows to propagate on to the display 50. The right / left sequence information is extracted from the plug-and-play information present on the I2S bus (also known as the DDC2 bus) on the video cable. The sequence information is then made available to electrical connector 60 which can be directly cabled to shutter glasses 70 or to a connector 80 following the 1997 VESA Connector & Signal Standards for Stereoscopic Display Hardware standard. As illustrated, connector 80 may be coupled to an infra-red transmitter 90 which sends infra-red light to shutter glasses 95.
[0009] In response to the signals from connector 60, an IR transmitter outputs infrared data that indicate when the right eye image is being output and when the left eye image is being output. The inventors note that many different manufacturers currently have different IR data packet definitions and protocols. For example, one simple format for infrared data is a simple square wave with a high signal indicating left and a low signal indicating right; and another format includes a 8-bit word. Because of these different data formats, IR transmitters from one manufacturer often cannot be used with LCD glasses from another manufacturer.
[0010] In various systems, infrared data is received by LCD glasses 95, and in response, for example, the right LCD of the LCD glasses 95 becomes opaque and the left LCD becomes translucent (e.g. clear,), or the left LCD of the LCD glasses 95 becomes opaque and the right LCD becomes translucent. Ideally, at the same time the right LCD becomes translucent, display 50 is displaying a right eye image, and when the left LCD becomes translucent, display 50 is displaying a left eye image. A similar process is performed by shutter glasses 70.
[0011] In theory, systems illustrated in Fig. 1 are expected to provide a workable, robust system. However, in practice, then inventors have determined that there are many limitations that degrade the performance of such systems and that limit the applicability of such systems from being successfully and widely adopted. As merely an example, wired shutter glasses, such as shutter glasses 70 are almost never used now, as Consumers do not want tethered devices, e.g. remote controls, game controllers, shutter glasses.
[0012] One such limitation includes the difficulty in synchronizing the glasses to the images that are displayed. Synchronization data is typically based upon when the images are provided to the 3D display. Limitations to such approaches, determined by the inventors includes that both latency and timing jitter are introduced as it is processed and rendered by the 3D display device. In various embodiments, jitter as little as 50 microseconds or 10 microseconds can affect the performance of the glasses. As a result of such latency and jitter information, the LCD lenses or shutters are often opened and closed often at improper times, e.g. out of phase, with some of the image intended for the left eye being shown to the right eye and vice versa. This is perceivable by the user as light pollution or ghosting effects.
Additionally, as the inventors have determined that the phase difference is not constant and is subject to jitter, the user may see the image brightness change or flicker undesirably.
[0013] One approach to reduce such latency or jitter effects has been to reduce the amount of time the left LCD shutter and the amount of time the right LCD shutter are translucent. In such approaches, instead of the left shutter being open for example 50% of the time, the left shutter may be open 35% of the time, or the like. This reduction in open time should reduce the amount of ghosting.
[0014] The inventors recognize drawbacks to such approach to image ghosting. One such drawback is the reduction in net amount of light transmitted to the user's eyes. In particular, as the exposure time for each eye is reduced, the user will perceive a darkening of the images for each eye. Accordingly, a 3D version of a feature will appear darker and duller compared to a 2D version of the feature when using shutter glasses.
[0015] Another limitation is the use of the IR communications channel itself. The inventors of the present invention have determined that LCD glasses based upon IR receivers often lose synchronization with the display as a result of stray reflections. For example, it has been observed by the inventors that IR LCD glasses may become confused as a result of sunlight reflecting from household objects; heat sources such candles, open flames, heat lamps; IR remote controls (e.g. television remotes, game controllers); light sources (e.g. florescent lights); and the like. Additionally, it has been observed by the inventors that IR LCD Glasses may also lose synchronization as a result of clothing, hair, portions of other users bodies (e.g. head), or the like, that temporarily obscure an IR receiver of the LCD glasses. The loss of synchronization may lead the user to seeing a series of flickering or rolling images and / or the left eye seeing the right eye image. The inventors believe these types of anomalies are highly disturbing to most users and should be inhibited or minimized.
[0016] In some cases manufacturers of such devices specifically instruct users to use IR LCD glasses in highly controlled environments. For example, they suggest that the 3D displays and glasses be used only in darkened rooms. The inventors believe such a solution limits the applicability and attractiveness of such 3D display devices to typical consumers. This is believed to be because most consumers do not have the luxury of a dedicated, light- controlled room for a home theater, and that most consumer entertainment rooms are multipurpose family rooms.
[0017] An additional drawback to such devices, determined by the inventors, is that multiple 3D display systems cannot easily be operated in the vicinity of each other. As described above, each 3D display system includes its own IR transmitter and 2D field timing and phase data. Then, when two such systems are in close proximity, a user's IR LCD glasses may receive IR transmissions from either of the 3D display systems. Because of this, although a user is viewing a first 3D display, the user's 3D glasses may be synchronizing to a different 3D display, causing the user to undesirably view flickering and rolling images. The inventors of the present invention thus recognize that multiple 3D display systems cannot easily be used in applications such as for public gaming exhibitions, tournaments, or contests, trade shows, in stadiums, in restaurants or bars, or the like.
[0018] An additional drawback to current 3D systems includes that the 3D source, e.g. computer, DVD player, or the like, must specifically generate and provide switching signals for the 3D glasses. As illustrated in Fig. 1, computer 10 / graphics processor specifically generates the switching signals and outputs these signals on VGA port 20. The inventors believe this is undesirable as it requires sources of 3D images, e.g. PS3, to be modified in firmware, or physically, to output such switching information.
[0019] Accordingly, what is desired are improved methods and apparatus for improved 3D image viewing without the drawbacks discussed above.
BRIEF SUMMARY OF THE INVENTION
[0020] The present invention relates to stereoscopic 3D image viewing methods and apparatus. More particularly, the present invention relates to stereoscopic 3D image viewing devices incorporating robust synchronization capability.
[0021] In various embodiments of the present invention, a stereoscopic 3D image viewing device is based upon liquid crystal display (LCD) shutters that are synchronized to a source of 3D images. In various embodiments, the synchronization is based upon RF protocols such as Bluetooth, ZigBee radio (ZigBee Alliance), Z-Wave, IEEE Standard 802.11, IEEE Standard 802.15.4, or any other type of RF communications protocol. In some embodiments of the present invention, the stereoscopic 3D image viewing device may transmit data back to the source of 3D images, via the RF communications mechanism or protocol, to increase the level of synchronization between the two devices. [0022] In various embodiments, by using a multitude of communications protocols (e.g. RF) and adding feedback from 3D shutter glasses back to the 3D image source, a system, method, and apparatus of perceiving stereoscopic 3D can be generated which improves the level of synchronization between the alternating images and the alternating action of shutter glasses. A system, apparatus, method, and computer-readable media are provided to enable stereoscopic viewing. In particular, according to one method, the physical method of connecting the display system to stereoscopic glasses is the IEEE 802.11 wireless radio, , IEEE 802.15.4 wireless radio, ZigBee radio, Z-Wave, or Bluetooth technology. This allows a user to move one's head into positions that would normally lose reception of wireless transmissions (e.g. IR) thus simplifying the user experience of wearing stereoscopic glasses. The wireless radio connection also has the advantage of replacing the infra-red light transmission method and its associated interference with remote controls and tendency to accept interference from natural and artificial light sources, thus enhancing the user experience.
[0023] In various embodiments, a shutter glasses control timer and multi-layer timer feedback loop are provided to 3D glasses for improved stereoscopic viewing. In particular, according to one embodiment, the control timer and multi-layer timer feedback loop operate the liquid crystal shutter action of the 3D glasses. Further, these components utilize the 3D source synchronization signal (e.g. system), in one example the VESA signal, along with RF- based communications mechanisms, as discussed herein, e.g. IEEE 802.15.4 wireless radio. The RF-based communications channel between the display system and the 3D stereoscopic glasses allows a user to move his head into positions and to locations that would normally cause loss of reception of 3D glasses based upon infrared transmissions. Further, the shutter control timer and multi-layer feedback loop improves the three dimensional perception by eliminating jitter and noise in the system (3D source) synchronization signal. In various embodiments, the shutter control timer and multi-layer feedback loop of the 3D glasses can quickly synchronize with the system synchronization signal and can maintain the
synchronization of the display and shutter action of the glasses although actual
synchronization may be temporarily lost. Such embodiments improve the user's 3D experience.
[0024] In various embodiments, such shutter control timer includes hardware based upon a microprocessor in the LC shutter glasses. In such embodiments, the microprocessor receives the timing information (e.g. system synchronization signals) received from the 3D system synchronization source via wireless signal and the feedback loop synchronizes the localized control timer within the 3D glasses with the system synchronization signal. Based upon the localized clock, in the short term absence of input synchronization information or in short periods of high signal jitter, the timer control system in the 3D glasses does not adjust the frequency of phase of the LCD switching, and relies upon its own internal clock.
Accordingly, in such conditions, the synchronization between display and shutter action is maintained.
[0025] According to other aspects, a method is provided for synchronization between the video transmitter and the shutter glasses. Synchronization is provided via a protocol that provides timing information such as a beacon offset or any series of packets that is used as the energy to excite a clock. A precision timing protocol may be utilized to provide synchronization between the transmitter and the receiver.
[0026] In various embodiments of the present invention, to increase the brightness of images sent to a user's eyes, the same right-eye image is output to a video display during two or more successive frame times, for example. In such embodiments, during the first frame time, the image is transitioning between a left-eye image and the right-eye image, and thus the right-eye lens of the 3D glasses is set to an opaque state. Then, during the second frame time, the image is transitioning from the right-eye image to the right image, and thus appears stable. During this second frame time, the right eye lens of the 3D glasses is set to a nonopaque state. The process may be repeated for the left-eye image.
[0027] Various embodiments of the present invention may display an image more than two times, as the refresh rate of the video display increases. In such embodiments, the brightness further increases. Other embodiments of the present invention may be adapted to display other images than a left / right image pair, such as a 2D image, or different left / right image pairs.
[0028] The above-described subject matter may also be implemented as a computer- controlled apparatus, a computer process, a computing system, or as an article of manufacture such as a pair of electronic glasses, an earbud or headset, a computer program product or a computer-readable medium. These and various other features will be apparent from a reading of the following Detailed Description and a review of the associated drawings. In various embodiments, the shutter glasses and the transmission device may include executable computer programs resident in a memory that instructs a respective processor to perform specific functions or operations, such as to transmit data, to determine a latency, or the like.
[0029] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
[0030] According to one aspect of the invention, a method for operating a pair of shutter glasses including a right LCD shutter and a left LCD shutter is disclosed. One process includes receiving synchronization data via radio frequency transmissions from a radio frequency transmitter, and determining shutter timings for the right LCD shutter and the left LCD shutter in response to the synchronization data. A technique may include applying the shutter timings to the right LCD shutter and the left LCD shutter to enable the viewer to view right-eye images via the right LCD shutter and left-eye images via the left LCD shutter.
[0031] According to another aspect of the invention, a method for transmitting stereoscopic display information includes: converting one or more video synchronization signals into wireless radio signals; and decoding the wireless radio signal in a pair of shutter glasses or other device; wherein the wireless radio is the IEEE Standard 802.1 1, WiFi, or components thereof.
[0032] According to another aspect of the invention, a method for transmitting stereoscopic display information includes: a pair of shutter glasses or other consumer electronics device which contains a localized clock such that the device remains synchronous to the video display system even when the connection to the source transmitting the synchronization information is interrupted or is not present. In some aspects, the synchronization information between the display system and the glasses or other device are determined by a precision timing protocol in which bidirectional communication of timing information occurs.
[0033] According to another aspect of the invention, a method for transmitting stereoscopic display information includes: a pair of shutter glasses or other consumer electronics device which receives synchronous information from the video display system, and a means and method of storing the delay and synchronization information in the transmitter or the video source generating computer, home theater system, or device. In some aspects, the delay and synchronization information are stored and then transmitted to multiple devices to allow multiple users to simultaneously use the same system.
[0034] According to another aspect of the invention, a method for transmitting stereoscopic display information includes: a pair of shutter glasses or other consumer electronics device which receives synchronous information from the video display system, and a means of determining the delay and synchronization information through information contained in the display and transmitter from the display via the video signal cable. [0035] According to another aspect of the invention, a method for transmitting stereoscopic display information, the method including: a transmitter of synchronization information and a pair of shutter glasses or other consumer electronics device which is capable of receiving synchronization information from infrared, visible light and radio sources. In various aspects, the shutter glasses or other receiving device can incorporate a computer program which allows the device to automatically determine which source or sources of
synchronization information are available and automatically use the best source or sources.
[0036] According to another aspect of the invention, shutter glasses includes various radio frequency receiving capability along with a feedback mechanism and a localized clock. The introduction of a synchronized timer in the shutter glasses improves the synchronization between the alternating source images and the alternating action of shutter glasses . It is with respect to these considerations that a LC shutter control timer and multi-layer timer feedback loop are provided for improved perception of stereoscopic 3D viewing.
[0037] According to another aspect of the invention, a method of combining stereoscopic glasses with a wireless headset, Bluetooth headset, or stereo Bluetooth headset is disclosed.
[0038] According to another aspect of the invention, a method of combining ordinary or automatically darkening sunglasses with a wireless headset, Bluetooth headset, or stereo Bluetooth headset is disclosed.
[0039] According to another aspect of the invention, a three-dimensional viewing device for providing images to a user is disclosed. One apparatus includes a receiver configured to receive source 3D synchronization signals from a transmitting device, wherein the source 3D synchronization signals comprise a source frequency and a source phase. A device may include a plurality of LCD shutters including a right LCD shutter and a left LCD shutter, wherein the right LCD shutter and the left LCD shutter are configured to alternatively enter a translucent state in response to local 3D synchronization signals. A system may include a localized timing source coupled to the receiver and to the plurality of LCD shutters, wherein the localized timing source is configured to generate the local 3D synchronization signals in response to the source 3D synchronization signals, and an adjustment portion coupled to the localized timing source and to the receiver, wherein the adjustment circuit is configured to adjust parameters of the local 3D synchronization signals in response to parameters of the source 3D synchronization signals.
[0040] According to yet another embodiment, a method for operating a three-dimensional viewing device including a right LCD shutter and a left LCD shutter is disclosed. One method includes receiving source 3D synchronization signals from a transmitting device, wherein the source 3D synchronization signals comprise a source frequency and a source phase, and generating a local 3D synchronization signals in response to the source 3D synchronization signals. A process may include adjusting parameters of the local 3D synchronization signals in response to parameters of the source 3D synchronization signals, and driving the right LCD shutter and the left LCD shutter with the local 3D synchronization signals, wherein the right LCD shutter and the left LCD shutter are configured to
alternatively enter a translucent state in response to local 3D synchronization signals.
[0041] According to one aspect of the invention, a device for driving a pair of three- dimensional glasses is disclosed. One apparatus includes an interface portion configured to receive 3D video output signals from a source of 3D video signals, and a decoding portion configured to determine a plurality of synchronization signals in response to the 3D video output signals. A system may also include a radio frequency transmitter configured to output radio frequency signals to the pair of three-dimensional glasses in response to the plurality of synchronization signals.
[0042] According to another aspect of the invention, a method for driving a pair of three- dimensional glasses is disclosed. One technique includes receiving 3D video output signals from a source of 3D video signals, and determining a plurality of synchronization signals in response to the 3D video output signals. A process may include outputting radio frequency signals to the pair of three-dimensional glasses in response to the plurality of synchronization signals.
[0043] According to one aspect of the invention, a method for providing three-dimensional video images to a user on a display system programmed to implement the method is disclosed. One technique includes while a right-eye lens of three dimensional glasses of the display system is in an opaque state, transferring with a video driver of the display system a first instance of a left-eye image to a video display device during a first frame time, while a left-eye lens of three-dimensional glasses of the display system is in an opaque state, and transferring with the video driver of the display system a second instance of the left-eye image to the video display device during a second frame time, while the left-eye lens of the three-dimensional glasses of the display system is in a non-opaque state, wherein the first frame time and the second frame time are adjacent. A process includes while the left-eye lens of the three dimensional glasses of the display system is in the opaque state, transferring with the video driver of the display system a first instance of a right-eye image to the video display device during a third frame time, while the right-eye lens of the three-dimensional glasses of the display system is in the opaque state, and transferring with the video driver of the display system a second instance of the right-eye image to the video display device during a fourth frame time, while the right-eye lens of the three-dimensional glasses of the display system is in a non-opaque state, wherein the third frame time and the fourth frame time are adjacent.
[0044] According to another aspect of the invention, a display system for providing three- dimensional video images to a user is disclosed. One apparatus includes a pair of three dimensional glasses having a right-eye lens configured to have an opaque state or a nonopaque state in response to right-eye timing data, a left-eye lens configured to have an opaque state or a non-opaque state in response to left-eye timing data, and a wireless communication receiver configured to receiving timing information from a video processor. A pair of three dimensional glasses may include a processor coupled to the right-eye lens, the left-eye lens, and to the wireless communication receiver, wherein the processor is configured to receive timing information from the wireless communication receiver, wherein the processor is configured to provide left-eye timing data to the left-eye lens for a first frame time and a second frame time in response to the timing information, and wherein the processor is configured to provide right-eye timing data to the right-eye lens for a third frame time and a fourth frame time in response to the timing information, wherein the first frame time and the second frame time are sequential, and wherein the third frame time and the fourth frame time are sequential. A device may include a video processor coupled to the pair of three dimensional glasses and to a video source including a processor configured to determine the timing information in response to video data from the video source, and a wireless communication transmitter configured to transmit the timing information to the pair of three dimensional glasses.
[0045] According to yet another aspect of the invention, a video display system for driving a video display and a pair of 3D glasses is described. One device may include a sequence memory configured to store an output image sequence, and a first double-buffer memory, wherein the first double-buffer memory is configured to simultaneously store and output a first left-eye image while receiving and storing a second left-eye image. An apparatus may include a second double-buffer memory, wherein the second double-buffer memory is configured to simultaneously store and output a first right-eye image while receiving and storing a second right-eye image. A system may include a sequencer coupled to the sequence memory, to the first double-buffer memory, and to the second double-buffer memory, wherein the sequencer is configured to output the first left-eye image to a video display during a first frame time, to output the first left-eye image to the video display during a second frame time, to output first timing information to instruct a left-eye lens of a pair of 3D glasses to enter an opaque state during the first frame time, and to output second timing information to instruct the left-eye lens of the pair of 3D glasses to enter a non-opaque state during the second frame time, in response to the output image sequence.
BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more fully understand the present invention, reference is made to the accompanying drawings. Understanding that these drawings are not to be considered limitations in the scope of the invention, the presently described embodiments and the presently understood best mode of the invention are described with additional detail through use of the accompanying drawings in which:
[0047] FIG. lis a block diagrams illustrating aspects of the prior art;
[0048] FIGS. 2A-D include block diagrams of various embodiments of the present invention illustrating the process of elements of a system in which stereoscopic glasses are synchronized with the display device by incorporation of a wireless radio into the system;
[0049] FIG. 3 illustrates a block diagram of a process according to various embodiments of the present invention;
[0050] FIG. 4 is a timing diagram of various embodiments of the present invention illustrating a method of sending image information to a display in which the frames which compose the image are sent sequentially;
[0051] FIG. 5 illustrates various embodiments incorporated into a mobile phone's hardware, firmware, and software and into a pair of stereoscopic shutter glasses;
[0052] FIG. 6 illustrates various embodiments incorporated into a mobile phone, some of the methods are incorporated into a pair of stereoscopic shutter glasses, and other methods are incorporated into a cradle or other device that attaches to the mobile phone;
[0053] FIG. 7 illustrates various embodiments incorporated into a pair of stereoscopic shutter glasses combined with a mobile phone headset;
[0054] Figs. 8A-C illustrate various embodiments of the present invention;
[0055] Fig. 9 illustrates various embodiments of the present invention;
[0056] FIG. 10 is a block diagram illustrating aspects of the prior art; [0057] FIG. 11 is a block diagram illustrating various embodiments of the present invention;
[0058] FIG. 12 is a block diagram illustrating various embodiments of the present invention;
[0059] FIG. 13 is a block diagram illustrating various embodiments of the present invention;
[0060] FIG. 14 is a block diagram illustrating various embodiments of the present invention;
[0061] FIG. 15 is a block diagram illustrating various embodiments of the present invention; and
[0062] FIG. 16is a block diagram illustrating various embodiments of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0063] FIGS. 2A-D illustrate various embodiments of the present invention. In particular, FIGS. 2A-D illustrate various arrangements of embodiments of the present invention.
[0064] FIG. 2A includes a 3D source 34 of image data, a transmission device 37, a display 43, and shutter glasses 42. In various embodiments, 3D source 34 may be a computer, a Blu-ray or DVD player, a gaming console, a portable media player, set-top-box, home theater system, preamplifier, a graphics card of a computer, a cable box, or the like, and 3D display 43 may be any 3D display device such as an LCD/Plasma/OLED display, a DLP display, a projection display, or the like. In various embodiments, transmission device 37 and shutter glasses 42 may be embodied by a product developed by the assignee of the current patent application, Bit Cauldron Corporation of Gainesville, FL. In some embodiments, shutter glasses 42 may be implemented with mechanical shutters or LCD shutters. For example, LCD shutters based upon twisted nemic, super-twisted nemic, or pi-cell technology, may be used.
[0065] In operation, 3D source 34 sends 3D display signals to display 43 through a video cable 35, typically through a standards-based interface such as VGA, DVI, HDMI, Display Port (DP), or the like. Such 3D display signals are often configured as one or more interleaved full right-eye images then full left-eye images (e.g. field sequential); double wide (e.g. side by side) or double height (e.g. stacked) images including both left and right images; images interleaved with right-eye images and left-eye images on a pixel by pixel basis; or the like. As shown in FIG. 2A, a transmission device 37, e.g. a radio transmitter may be inserted between the 3D source 34 or other video source and 3D display 43.
[0066] In various embodiments, in transmission device 37 determines 3D timing information by decoding the 3D display signals as they pass through to display 43 on signal line or cable 44. In FIG. 2A, transmission device 37 includes a transmitter based upon radio frequency (RF) signals. The RF signals may use or may be combined with any conventional transmission protocol such as IEEE Standard 802.15.1 (e.g. Bluetooth), IEEE 802.11 (e.g. Wi-Fi), IEEE Standard 802.15.4 (e.g. ZigBee Alliance radio), Z-Wave, or the like. In various embodiments, synchronization signals 46 are then transmitted via antenna 39.
[0067] In various embodiments, transmission device 37 may be a stand-alone device, e.g. a dongle, a USB "key," or the like and transmission device 37 may be powered by power source 36 and 38, self-powered, powered from 3D data source, USB powered, or the like. In other embodiments, transmission device 37 may incorporated into another device, such as 3D source 34, 3D display 43, a pre-amplifier, or the like.
[0068] FIG. 2B illustrates additional embodiments of the present invention. In particular, Fig. 2B includes a source of 3D images 100, a transmission device 1 10, and a 3D display 120. As illustrated, 3D image source 100 provides 3D images (e.g. double-wide or double- height images including both right and left images) to 3D display 120 via a signal line 130 such as a VGA, DVI, Display Port (DP), cable, or the like. Additionally 3D image source 100 provides a synchronization signal along signal line 140 to transmission device 1 10. In various embodiments, 3D image source 100 includes an industry standard interface such as a VESA miniDIN-3 connector, VESA 1997.1 1, USB connector, or the like, to which transmission device 1 10 may be coupled.
[0069] FIG. 2C illustrates additional embodiments of the present invention. In particular, Fig. 2C includes a source of 3D images 160, a transmission device 170, and a 3D display 180. As illustrated, 3D image source 160 provides 3D images (e.g. double-wide or double- height images including both right and left images) to 3D display 180 via a signal line 190 such as a VGA, DVI, HDMI cable, Display Port (DP), or the like. In turn, 3D display 180 provides a synchronization signal along signal line 200 to transmission device 170. In various embodiments, 3D display 180 includes an industry standard interface such as a VESA miniDIN-3 connector, USB connector, or the like, to which transmission device 170 may be coupled. [0070] FIG. 2D illustrates other additional embodiments of the present invention. In particular, Fig. 2D includes a source of 3D images 220, a transmission device 230, and a 3D display 240. As illustrated, 3D image source 220 provides 3D images (e.g. double-wide or double-height images including both right and left images) to 3D display 240 via a signal line 250 such as a VGA, DVI, HDMI cable, Display Port (DP), or the like. In these
embodiments, transmission device 230 may be disposed within 3D display 240. For example, transmission device 230 may be installed within the manufacturing facility of 3D display 240, or the like. In such embodiments, 2D display 240 may also power transmission device 230. Similar to the embodiments described above, 3D display 240 provides a (derived) synchronization signal along signal line 260 to transmission device 230.
[0071] In various embodiments described herein, shutter glasses 42 include a radio receiver 41 that receives the synchronization signals 40. In response to synchronization signals 40, shutter glasses 42 alternatively changes the properties of one lens from translucent to opaque (e.g. dark) to translucent and of the other lens from opaque to translucent (e.g. clear) to opaque. Because the shutters of shutter glasses 42 operate under the direction of synchronization signals 40, a user / viewer, views 3D display images 45 from display 43 at the proper timing. More particularly, the user's right eye is then exposed to a right-eye image from 3D display images 45, and then the user's left eye is then exposed to a left-eye image from 3D display images 45, etc.
[0072] The inventors of the present invention recognize that transmission device 37 based upon a radio frequency transmitter has several advantages over an infrared transmitter. One advantage recognized is that radio signals can be received in many situations where an infrared signal would be blocked. For example this allows the user of a pair of 3D shutter glasses or the like, to move their head much farther away from the 3D display or transmission device than if IR were used, and allows the user to move throughout a room with a larger range of motion while maintaining synchronization with the 3D display. As another example, RF transmitters allow other people or objects to pass in front the user / viewer without interrupting the signal.
[0073] Another advantage goes beyond the improved range and reliability of radio technology for synchronization purposes. For the example, the inventors believe that the avoidance of infrared is itself a benefit, as infrared signals can interfere with remote controls, such as those popular in households and home theater systems. Additionally, another benefit includes that IR receivers are often interfered with and are confused by IR remote controls, natural and artificial light sources, and video displays themselves [0074] In various embodiments of the present invention, shutter glasses 42 may include its own localized clock. Benefits to such a configuration include that it allows shutter glasses 42 to remain approximately synchronized to display 43 even though the connection to transmission device 37 is interrupted and / or synchronization signals 40 are not received.
[0075] In various embodiments, a precision timing protocol can be used so that the clock that is local to shutter glasses 42 is synchronized with a clock within transmission device 37 and / or the 3D display signals. A precision timing protocol may include the transmission of data packets with a time stamp time associated with the 3D display signals to shutter glasses 42. In other embodiments, the protocol may include transmission of a data packet with a time stamp associated with shutter glasses 42 to transmission device 37. In operation, shutter glasses 42 receive the time stamp from the 3D data source, compares the received time stamp to its local clock and returns a data packet with its local time stamp. Using this information, transmission device 37 can determine a round-trip time for data between transmission device 37 and shutter glasses 42. In some embodiments of the present invention, the round-trip time offset is evenly divided between transmission device 37 and shutter glasses 42. In other embodiments, if one or both devices are capable of determining a difference in speed or lag between the two transmissions, then a more precise determination of the relative values of both clocks (offsets) can be determined. As a result, in various embodiments, more precise synchronization between the two clocks can be established.
[0076] In various embodiments of the present invention, by repeating this process periodically, the difference in rate (e.g. frequency) between the two clocks (transmission device 37 or 3D source 34 and shutter glasses 42) can be more precisely determined. In some embodiments if there is a low degree of consistency in the latencies, the period of time between the determination of a latency process may be made small, e.g. once a minute; and if there is a higher degree of consistency in the latencies, the period of time between the determination of a latency process may be increased, e.g. once every ten minutes.
[0077] Embodiments of the present invention enable the use of multiple pairs of shutter glasses 42. In such embodiments, a single pair of shutter glasses 42 may be used to determine delay and jitter as discussed was discussed above. Next, a simpler protocol, such as a unidirectional or broadcast protocol, may be used by transmission device 37 to communicate this synchronization information to the remaining pairs of shutter glasses. In various embodiments, the delay and jitter information can be stored in transmission device 37, in 3D source 34, or other consumer electronics device generating the 3D data, either in a volatile or non-volatile manner. [0078] In other embodiments of the present invention other methods can be used to determine the synchronization and delay information. In various examples, this data may be determined using bidirectional communications on cable 44, such as the DisplayPort protocol, or the like, as illustrated in FIG. 2C. Communications protocols such as display data channel (DDC and DDC2) protocols, PanelLink serial protocol or a similar protocols allows the display to communicate information back to the computer, home theater system, video source, or the like. In various embodiments, this serial protocol can be enhanced to provide the appropriate latency and synchronization characteristics of 3D display 43 back to 3D source 34 and / or transmission device 37. In other embodiments, these protocols can be used to determine the manufacturer, vendor, or other identifying information for 3D display 34, and a table of pre-determined synchronization information can be retrieved, either locally, across a local area network, across a network, or the like This information may include an appropriate delay and synchronization information for respective 3D displays.
[0079] FIG. 3 illustrates a block diagram of a process according to various embodiments of the present invention. More specifically, FIG. 3 illustrates a process for synchronizing shutter glasses to a source of 3D images.
[0080] Initially, a 3D data source provides 3D images, step L. In various embodiments, the 3D images may be provided in any number of specific formats, such as right and left images: sequentially transmitted, packed vertically or horizontally into a single image and transmitted, combined on a pixel by pixel basis into a single image and transmitted, or the like. In other embodiments, as illustrated in Fig. 2B, 3D data source may provide specific timing data.
[0081] Next, in response to the data from 3D data source, synchronization data, such as an identifier of a timing clock resident on 3D data source is determined, step 310. In various embodiments, this may include a packet of data including a source time stamp, or the like.
The synchronization data may then be transmitted through radio frequency transmissions to a first pair of shutter glasses, step 320.
[0082] In various embodiments, the shutter glasses receive the source time stamp and synchronizes the operation of the right / left shutters to the synchronization data, step 330. The synchronization data can then be maintained within the shutter glasses by an internal clock within such glasses, step 340. As synchronization data is received, the internal clock can be resynchronized. Such embodiments are believed to be advantageous as the glasses need not wait for synchronization data from the 3D data source to be able to switch.
Accordingly, synchronization data from the transmission device may be dropped or lost while the shutter glasses continue to operate properly. When synchronization data is reestablished, the synchronization described above may be performed.
[0083] In various embodiments of the present invention, RF communications using the ZigBee radio (IEEE 802.15.4 standard) occur at 2.4 GHz, the same band as most Wi-Fi transmissions. In the case of interference with Wi-Fi transmissions, embodiments of the present shutter glasses are designed to inhibit communications, and defer to such Wi-Fi signals. As discussed above, in some embodiment, the shutter glasses will continue to operate autonomously, until the interference stops and new synchronization data is received from the transmission server.
[0084] In some embodiments of the present invention, the shutter glasses may transmit data back to the RF transmission device. More specifically, the shutter glasses may transmit the received source time stamp and / or the glasses time stamp back to the transmission device via the same RF communications channel, or the like, step 350.
[0085] In FIG. 3, in response to the received source time stamp and / or the glasses time stamp, and the source time stamp when these data are received, the transmission device may determine adjustments to subsequent synchronization data that will be sent to the shutter glasses, step 360. As an example, the transmission device may determine that it should output synchronization data to the shutter glasses, even before the synchronization data is determined or received from the 3D data source. As a numeric example, if it is determined that the shutter glasses lag the 3D data source by 100 microseconds, the shutter glasses may trigger its shutters 100 microseconds before the expected arrival of a synchronization pulse.
[0086] In various embodiments of the present invention, this adjustment to
synchronization data may be used to drive 3D glasses of other viewers of the 3D image. In other embodiments of the present invention, 3D glasses of other viewers in the room may also have synchronization data adjusted using the process described above. In such embodiments, the transmission device may output the synchronization data at different times for different 3D glasses.
[0087] In other embodiments, other adjustments may be performed by the shutter glasses. For example, based upon received time stamps and the shutter glasses own internal clocks, the shutter glasses may verify that they are in sync. If not, the shutter glasses may adjust the frequency of its own internal clocks until they are kept in a higher amount of synchronization.
[0088] As seen in FIG. 3, the process may be repeated. In various embodiments, the synchronization process may be performed periodically, with the period dependent upon how well the 3D data source and the shutter clock stays remain in synchronization - if highly synchronized, the synchronization process may be performed at longer time periods apart (e.g. 2 minutes) than if these devices continually have synchronization problems (e.g. every 10 seconds). Further detail regarding the above synchronization process may be found in the provisional application referenced above.
[0089] Various embodiments of the present invention may include shutter glasses or other devices that includes multiple physical methods for receiving synchronization information. For example, some embodiments may contain both an infrared and radio receiver; an infrared and visible light receiver; a radio or visible light receiver; a combination of infrared, visible light and radio receivers; or the like. In such embodiments, the shutter glasses or other receiving device may include executable computer program that instructs a processor to automatically determine which communications channel or channels are available, and automatically use the communications channel having the strongest signal, lowest number of dropped data packets, or the like.
[0090] In various embodiments, the combination of a visible light receiver (e.g. IR) with another synchronization transmission technology (e.g. RF) may be advantageous. More specifically, the information transmitted via visible light and the synchronization information transmitted via another transmission technology may be combined within shutter glasses 42 to deduce unknown elements of the delay in 3D display 43 and other synchronization information. In various embodiments, the data from the different communication channels are compared to more precisely synchronize 3D display 43 and shutter glasses 42. As merely an example, the two communications channels can be used to verify that a left image displayed on 3D display 43 is going to the left eye and the right image displayed on 3D display 43 is going to the right eye. In such an example, this would preventing the error of a reversal of synchronization information somewhere in the system that results in the sending the left image to the right eye and vice versa.
[0091] In various embodiments of the present invention, shutter glasses 42 may used to provide a variety of new functions. FIG. 4 illustrates typical video output timing where frame one 26, frame two 28, frame three 30 and frame four 32 are output sequentially. In some embodiments, left images (frames) and right images (frames) are alternatively output. For example, frame one 26 is left, frame two 28 is right, frame three 30 is left and frame four 32 is right, creating the sequence L, R, L, R images to the user.
[0092] Various embodiments of the present invention may be applied to 3D displays having display rates on the order of 120Hz and higher. In embodiments where the refresh rate is 120 Hz, right and left images will be displayed and refreshed at 60 Hz. Accordingly, the viewer should not be able to detect significant flickering, however, the viewer may detect a darkening of the images. As refresh rates for future televisions, projectors or the like, are increasing, the inventors have determined that the higher refresh rate may enable new features, as described below.
[0093] In various embodiments, depending upon the output frame rate of the 3D display, more than one left image and right image may be output. For example, in various embodiments, multiple viewers may view a 3D display, and different viewers may see different 3D images. For example, a two viewer sequence of output images may be user 1 left, user 1 right, user 2 left, user 2 right, etc. This could be represented as: LI, Rl, L2, R2. In such examples, shutter glasses of a first viewer will allow the first viewer will see images LI and Rl and a shutter glasses of a second viewer will allow the second viewer will see images L2 and R2. In other examples, other sequences are contemplated, such as LI, L2, Rl, R2, and the like. With respect to refresh rate, for a 3D display having a 240 Hz refresh rate, a viewer will see the respective right and left images at a refresh rate of 60Hz. As noted above, this frequency should be above the typical sensitivity of the eye, however, viewers may detect a darker image. Such artifacts may be mitigated by increasing the brightness of the images, or the like.
[0094] Other embodiments may be extended to additional (e.g. three or more viewers). Applications of such embodiments may include for computer or console gaming, or the like. As an example, two or more viewers may initially see the same 3D image, and subsequently one or more viewers "break off to view a different 3D image. For example, three people could be playing a multiplayer game in which all three are traveling together and see the same 3D images. Next, one player then breaks away from the other players. Using the additional communications protocols disclosed in various embodiments of the present invention, the player's glasses can be reprogrammed to allow the third person to see a different 3D image. Subsequently, the third person may return to the group, and then see the same 3D image. In such an example, a sequence of images output by the 3D display could begin with L0-R0-L0-R0, where 0 indicates everyone in the party. Next, when the third person leaves the party, the 3D display could switch and output images in a sequence such as L1&2, R1&2, L3, R3; L1&2, L3, R1&2, R3; or the like. When the third person returns to the party, the sequence may revert to L0, R0, L0, R0. In various embodiments, switching back and forth may occur with little, if any, visible interruption in the 3D images viewed by the viewer. In various embodiments, the inventors recognize that the brightness of each frame may have to be adjusted to correct for the changes in overall viewing time.
[0095] In other embodiments, other sequences of images enable still other types functionality. For example, one sequence of frames can be sent such that viewers wearing 3D glasses see a stereo display and viewers without glasses see only one side of the image (e.g. left or right). In such an example, a three frame sequence may include: Left, Right, Left- minus-Right. In response, a user using embodiments of the present invention may see a stereoscopic image by viewing the left image in their left eye and the right image in their right eye. That user would be prevented from viewing the Left minus Right image. To a viewer without the glasses, they would see in succession: L, R, (L-R) = 2L, or only the left image with both eyes. In other embodiments, separate anti-left, anti-right images or both may also be sent. With such embodiments, theater-goers can decide whether they care to watch the same movie or feature with or without 3D glasses; game players can play in 3D while viewers watch the same display in 2D.
[0096] In still other embodiments, users not utilizing embodiments of the 3D glasses may view other arbitrary images. As an example, a sequence may be: Left, Right, and Arbitrary- minus-Left-minus-Right = Arbitrary image. In operation, the viewer with 3D glasses may see the left image in the left eye and the right image in the right eye, and may not see the Arbitrary image. Further, the viewer without 3D glasses would see the arbitrary image, in succession: L, R, (A-L-R)=A, that may be a non-stereo version of the same program, a blank or solid color screen, or a completely different piece of content such as an advertisement, a copyright warning, or the like.
[0097] Fig. 5 illustrates additional embodiments of the present invention. More specifically, FIG. 5 illustrates a general purpose consumer device (e.g. mobile phone, personal media player, laptop, or the like) capable of 3D image output. In such
embodiments, the synchronization information to the shutter glasses may be provided by with the consumer device including embodiments of the RF transmitter described above, or unused or available transmitters available in the consumer device. Various examples may use infrared, WiFi, Bluetooth, or the like, to provide synchronization signals to shutter glasses according to embodiments of the present invention.
[0098] Fig. 6 illustrates additional embodiments of the present invention wherein existing consumer devices (e.g. mobile phone) may be augmented to better support stereoscopic 3D viewing. In various embodiments, a cradle or dongle which attaches to the mobile device or holds the mobile device may be used. In such examples, the cradle or dongle may incorporate a projection system such that the image may be projected at a larger size than the screen on the mobile device. The cradle or dongle or consumer device may also provide the synchronization signals to the shutter glasses. For example, the cradle or dongle may include a ZigBee radio-type transmitter (IEEE 802.15.4) that transmits the synchronization data to the shutter glasses, or the like.
[0099] In other embodiments of the present invention, stereoscopic shutter glasses that are to be used with the consumer device described above, can be used for other purposes. For example, if such glasses incorporate a visible light sensor, they can be worn as ordinary sunglasses but make improved automatic decisions about the appropriate level of perceived darkening. This information can be based on computer algorithms, information about the user and the environment that is stored on a mobile device; information retrieved from a computer network via the mobile device, and the like.
[0100] Fig. 7 illustrates yet another embodiment of the present invention. In such embodiments, a user of the consumer device may desire to perform multiple functions at the same time, such as: talk on a Bluetooth headset, view stereoscopic 3D content, and wear sunglasses. Embodiments illustrated in Fig. 7 may include a pair of shutter glasses 57 combined with a pair of sunglasses and a Bluetooth or stereo mobile Bluetooth headset with a left earpiece 58 and a right earpiece 55, or the like.
[0101] Figs. 8A-C illustrate various embodiments of the present invention. In particular, Figs. 8A-C illustrate block diagrams of various embodiments of a dongle providing RF transmissions, for example, as described above.
[0102] In Fig. 8A, a physical interface 710 is illustrated. In various embodiments, physical interface 710 may be a DVI port, HDMI port, Display Port (DP), USB, VESA 1997.11, or the like, for coupling to a source of 3D data 712(e.g. computer, DVD / BluRay player, HD display, monitor, etc.). In embodiments illustrated in Fig. 2B, a port 712 may drive a display, and a port 714 may drive a dongle 700.
[0103] In various embodiments of the present invention, an interface chip or block 720 may provide the electronic interface to physical interface 710. As illustrated, signals including horizontal synchronization signals (HSY C), vertical synchronization signals (VSY C), control signals, and a clock may be derived from the video data. Next, a processing device such as a CPLD (complex programmable logic device) 730 may be used to decode 3D synchronization data from 3D image data or 3D timing data. More specifically in various embodiments, sequence information is multiplexed directly into the video picture data from the source of 3D data 712. The digital picture data is demultiplexed with receiver 720 into its component video and control data streams 722, described above, such that each stream can be examined, or the streams can be examined relative to each other. In other embodiments, the sequence information can be further demultiplexed from the digital video information.
[0104] As mentioned above, Complex Programmable Logic Device (CPLD) 730 further decodes the sequence data. CPLD 24 is useful in various embodiments as the 3D sequence data that is encoded into a digital signal is very fast, typically as fast as 165 MHz. This is much faster compared to the typical 8-32 MHz speed of microprocessor 750, accordingly, microprocessor 750 typically cannot decode this stream of data. As the Control, VSYNC and HSYNC information, or the like 722 is not transmitted at every moment in time, CPLD 730 and microprocessor 750 further decode the sequence information into continuous information. In other words, the Control data, VSYNC and HSYNC information (sequence information) are provided at the frequency of the video data (e.g. HDMI, DVI) (e.g. within the range of 32 MHz to 165MHz to 340 MHz, although in the future, the pixel video data frequency may be even larger. However the sequence information must be converted to the appropriate left / right frame rate (e.g. 60Hz, 120Hz, 240Hz, 480Hz, or the like.)
[0105] In various embodiments of the present invention, microprocessor 750 includes an RF interface that references a clock 740. In some embodiments, microprocessor 450 is a TI CC2530 System on a Chip, that includes a 8051 MCU (processor), RAM, Flash memory, and a IEEE 802.14.4 ZigBee RF transceiver. The flash memory is configured to store executable computer code or instructions that directs the processor to perform various functions, as described herein. In various examples, the flash memory includes computer code that directs the processor to transmit the 3D synchronization data to the 3D glasses, to receive timing data back from the 3D glasses, to determine a round-trip communication latency, to adjust 3D synchronization data in response to the round-trip communication latency, and the like, as described above.
[0106] In various embodiments, using the IEEE 802.14.4 ZigBee RF transceiver of the like, 3D synchronization data can be sent to the wireless glasses. In other embodiments, an additional VESA 1997.1 1 port may also be provided for wired-glasses, as shown.
[0107] Fig. 8B, a physical interface 410 is illustrated. In various embodiments, physical interface 410 may be a DVI port, HDMI port, Display Port (DP), USB, VESA 1997.11, or the like, for coupling to a source of 3D data (e.g. computer, DVD / BluRay player, HD display, monitor, etc.). In embodiments illustrated in Fig. 2A or 2C for example, the 3D data may include 3D image data. Embodiments illustrated in Fig. 8B contemplate a pass-through type dongle, where, for example, a source of 3D data has only one physical interface 410, e.g. DVI port.
[0108] In various embodiments, one or more interface chips or block 420 may provide the electronic decoding interface to physical interface 410. More specifically, similar to Fig. 8A, various data such as HSYNC, VSYNC, control lines, clocks, pixel data, or the like may be decoded by interface chips 420. Additionally, one or more interface chips or block 460 may be provided to reencode the HSYNC, VSYNC, pixel data, and the like into a format appropriate for the physical interface 460. In various embodiments, physical interface 460 may be a DVI port, HDMI port, Display Port (DP), USB, VESA 1997.11, or the like, for outputting data to a 3D display device (e.g. computer monitor, home entertainment monitor, projection device, etc.).
[0109] In various embodiments of the present invention, CPLD 430 is used in similar manner described in Fig. 8 A, to decode data quickly. In various embodiments of the present invention, 3D synchronization data is then provided to an RF interface device 450 that references a clock 440. In some embodiments, RF interface device 450 is a TI CC2530
System on a Chip, that includes a 8051 MCU (processor), RAM, Flash memory, and a IEEE 802.14.4 ZigBee RF transceiver. The flash memory is configured to store executable computer code or instructions that directs the processor to perform various functions, as described herein. In various examples, the flash memory includes computer code that directs the processor to transmit the 3D synchronization data to the 3D glasses, to receive timing data back from the 3D glasses, to determine a round-trip communication latency, to adjust 3D synchronization data in response to the round-trip communication latency, and the like, as described above.
[0110] In Fig. 8B, dongle 400 is illustrated to be placed in-line within a DVI or HDMI signal path and to allow the video signal to behave as if it has passed through dongle 400 undisturbed, when it has in fact been decoded and reencoded.
[0111] Fig. 8C illustrates another contemplated embodiment, similar to Fig. 8B. More specifically, a dongle 800 with pass-through DVI / HDMI input and output interfaces is illustrated. In Fig. 8C, video signals need not be decoded or reencoded as shown in Fig. 8B. In various embodiments, one or more high impedance devices, such as FET probes 810, may be used to tap the video signals to derive the HSYNC, VSYNC, Control data, or the like. The monitored data can them be processed and output via RF signals to 3D glasses as described herein. The embodiments illustrated in Fig. 8C thus avoid the encoding and decoding processes and hardware of Fig. 8B, illustrated above. [0112] In various embodiments of the present invention, data wires 820 coming from the source of 3D data, e.g. a GPU, includes an I2S clock and I2S data wires. As shown data wires 820 are also passed through dongle 800 to allow the display to communicate Plug and Play and configuration information back through dongle 800 and to the GPU. In various embodiments, the I2S bus, also called the Display Data Channel 2 or Plug and Play bus, is a multi-drop bus meaning it is electrically capable of supporting communications between three or more devices. Thus, in addition to providing the sequence information via a wireless link, the wireless link can now also be recognized by an application on the PC via the Display Data Channel 2. This allows other wireless communications such as upgrading or communicating with wireless shutter glasses, command and control, home automation, or the like.
[0113] As illustrated in Figs. 8A-C, any other wired-type interface may also be provided. Illustrated in these figures is a VESA 1997.11 port. In other embodiments, the physical port may be a USB port, a firewire port, or any other custom / proprietary port. Additionally, in various embodiments, the interface may have a standard or proprietary signaling interface, for example, VESA 1997.1 1 specifies a specific signaling protocol. In other embodiments, in light of the present patent disclosure, one of ordinary skill in the art would recognize that a custom or proprietary signaling protocol may be implemented.
[0114] Fig. 9 illustrates various embodiments of the present invention. In particular, Fig. 9 illustrates a block diagram of a pair of shutter glasses 500 according to various
embodiments of the present invention. Shutter glasses 500 is illustrated to include an RF interface device 510 that references a clock 540 and a pair of electronically controlled LCD shutter elements 520 and 530.
[0115] In various embodiments of the present invention, 3D synchronization data 550, typically radio frequency signals, is received in RF interface device 510. In various embodiments, RF interface device 510 is also an TI CC2530 System on a Chip, that includes a 8051 MCU (processor), RAM, Flash memory, and a IEEE 802.14.4 ZigBee RF transceiver. The flash memory is configured to store executable computer code or instructions that directs the processor to perform various functions, as described herein. In various examples, the flash memory includes computer code that directs the processor to receive the 3D
synchronization data, to change the states of / drive shutter elements 520 and 530 at the appropriate timing (e.g. LI and Rl in the sequence LI, L2, Rl, R2), to send clock or timing data back to a transmission device via RF communications, and the like. [0116] In addition to the problems discussed above for 3D glasses based upon infrared communications links, the inventors of the present invention recognize that typical 3D glasses and displays (3D system) together suffer from performance issues.
[0117] Fig. 10 illustrates a timing diagram of a typical 3D system where stereoscopic viewing is achieved by sending alternating frames intended for viewing by a user's left and right eyes to a display. In such a system, a 3D display is driven with left-eye images in times 900, 910, etc. and right-eye images in times 920, 930, etc. and synchronized with a pair of electronic shutter 3D glasses. As discussed above, the shutter 3D glasses alternatively turns the lens for each eye from opaque to clear to enable the frames with information intended only for the left eye to be transmitted to the user's left eye in times 940, 950, etc. while blocking these frames from the user's right eye. This process then typically continues vice- versa, such that information intended only for the right eye to be transmitted to the user's right eye in times 960, 970, etc.
[0118] One limitation with typical systems is that all video displays do not
instantaneously switch from displaying one frame to the next. As illustrated in Fig. 10, for a display having a 120Hz refresh rate (8.3ms period) and a 180Hz frame rate, for a typical frame 900, 5.5ms is used for writing data to the display pixels and waiting for the pixel elements to change states. In this example, after the image is stable, the left-eye lens is opened in time 940 for only approximately 2.8ms. As the image is unique for about 2.8 ms out of the 8.3 ms allotted for the frame, this method is about 2.8/8.3 or 34% efficient. A consequence of this is that the user will perceive a dim and dull image when viewing 3D images.
[0119] In the worse case, some displays require almost the entire period of the frame to sweep through the display (row-by -row and column-by -column) and for the display elements to properly display the frame. In this worse-case there would never be a moment in which the image had a unique display, and thus the shutters could never open, leaving the glasses opaque. The inventors of the present invention believe that a functioning system must have better performance than never providing a unique image, accordingly, the inventors believe that the time allotted for the display to sweep through the rows and columns of pixels should be shorter compared to the time allotted to the shutters being open.
[0120] Fig. 11 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 1 1 illustrates embodiments of a technique named "Crystal Sweep." In such embodiments, a display is driven twice with each left or right frame or image. [0121] As can be seen in Fig. 1 1, in one example, in the first frame period 1000, the left- eye image is sent to the display, the display sweeps through each row and column of pixels and the display updates the formerly right-eye image with a left-eye image. In the subsequent period 1010, the left-eye image is again sent to the display, the display sweeps through each row and column of pixels and the display redisplays the left-eye image on the display. In such embodiments, in period 1010, as the display device redraws the left-eye image on the screen with an exact duplicate image, no visible distortion to the left-eye image occurs.
Accordingly, the left-eye image from a previous period is effectively frozen, or crystallized, onto the display. The result is that during period 1020, one period 1000 is spent drawing / outputting the left-eye image and one period 1010 is spent with a unique image. As illustrated in Fig. 11, this process repeats for the right-eye image, etc.
[0122] In this example, the left-eye shutter is non-opaque during period 1010, accordingly, to the user, the resulting left-eye image is visible 50% of the time allotted to the left-eye image. In the example of FIG. 1 1, for a display capable of refreshing an image at 180 Hz, embodiments of the Crystal Sweep method provide more viewing time than a display illustrated in Fig. 10. This is a significant improvement over the 34% calculated above in the prior art. Accordingly, to a user, using embodiments of the present invention, 3D images will appear brighter and more vibrant. As can be determined, this 50% visibility is independent of a display refresh rate, as exactly half of the periods are unique at any frequency.
[0123] The inventors of the present invention have also considered the human perception of stereoscopic 3D with regards to flicker. More specifically, if a left or right shutter of a pair of 3D glasses remains closed and the user's eye spends too much time viewing darkness between images, the user may perceive a strobing effect or flickering of the respective images. The inventors believe that such flickering of 3D images will decrease the quality of the user's 3D experience.
[0124] According to available literature, it is believed that most humans perceive flicker as a result of periods of darkness longer than 16.7ms. As the dark time periods increase, the perceivable flicker increases Further, as the dark time periods decrease, the perceivable flicker decreases until the periods of darkness reach about 13 ms. For time periods shorter than 13ms, a typical user will not be able to perceive any flicker, thus in some embodiments, there is less perceivable gain in further shortening the darkness periods.
[0125] In the example illustrated in FIG. 10, the interval of darkness 990 between the left eye opening twice is 13.8ms (5.5 (for right drawing) plus 2.8 (for right display) plus 5.5 (for left drawing)). Accordingly, it is expected that few users may experience flickering. In the example in FIG. 11, the interval of darkness 1030 between the left eye opening twice is in the order of 16.5 ms (5.5 (for right drawing) plus 5.5 (for right redrawing and right display) plus 5.5 (for left drawing)). Accordingly, it is expected that more users may experience flickering. Therefore, for the same display draw rate of 180 Hz or (5.5 ms), the Crystal Sweep embodiments trade a different point in the 13-16.7 ms flicker perception range for a superior brightness by virtue of the larger percentage of time spent with the lenses being open.
[0126] Fig. 12 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 12 illustrates embodiments of the present invention implemented upon a display with a quicker frame refresh rate.
[0127] In various embodiments, it is desirable to operate a stereoscopic display having a refresh rate at a multiple of same frequency of the electrical system. In such cases, florescent lighting typically flicker at the electrical system frequency. This lighting flicker would interfere with the television system images, if the television system refreshed at a frequency other than the electrical system frequency or multiple thereof.
[0128] In the embodiment illustrated in FIG. 12, a display is illustrated having a 240Hz refresh rate (4.2ms period). Using the Crystal Sweep embodiment discussed above, the left- eye image is also visible to the user 50% of the time allotted to the left-eye image.
Additionally, the interval of darkness 1 100 between the left eye opening twice is 12.6ms (4.2 (for right drawing) plus 4.2 (for right display) plus 4.2 (for left drawing)). Accordingly, it is expected that almost no users may experience flickering. In various embodiments, 3D images will appear brighter and have no perceivable flicker thus providing a user with an enjoyable 3D experience.
[0129] In various embodiments of the present invention, various methods may be used for the display writing an image in a first time period or frame time and re-writing the image in the second time period or frame time. As the image written in the first and the second time period are typically identical, in various embodiments, an additional innovation may include reducing the amount of redundant images that are communicated between a video source and a display.
[0130] In one embodiment, the video source or rendering system (e.g. a graphics processing unit, a Blu Ray player, etc.) may provide one copy or instance of an image for a first time period to a video display system. In turn, the video display system may buffer the image, write the buffered image to the display during the first time frame, and rewrite the buffered image to the display during the second time frame. In such cases, instead of the video source transmitting another instance of the image for the second time frame, the video source need only signal to the video display system that the information during the second frame time is redundant to the first frame time. This can leave the rendering system free to send information to another display, render addition detail for an upcoming image, perform other activities or conserve power, or the like. Similarly, the video display system can perform other activities or save power by receiving only the required information that the latter frame is redundant to the former frame. In various embodiments, this redundancy of frames can be communicated through a plurality of signals, including exerting the VSYNC or HSYNC signals or exerting another digital interface, such as the CONTROL signal in a DVI or HDMI connection, by including information about the anticipation of an upcoming redundant frame in the overscan area of a previous frame, or the like. Other communication paths between the rendering system and the display subsystem are also contemplated.
Embodiments may be implemented with the examples in Figs. 21-D may implement the above mentioned features.
[0131] In other embodiments, radio transmitter 37, in Fig. 2A, for example, may provide the image buffering capability. More specifically, image output 34 may output one instance of a left-eye image to radio transmitter 37. In turn, in addition to rf communication with 3D glasses 42, discussed above, radio transmitter 37 may buffer the left-eye image, output one instance of the left-eye image to display 43 during a first frame time, and output another instance of the left-eye image to display 43 during a second frame time. Accordingly, the communication load of image output 34 is reduced.
[0132] Fig. 13 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 13 illustrates embodiments of Crystal Sweep in addition to embodiments of "Freeze" tag.
[0133] In various embodiments while one eye of the shutter glasses are in a non-opaque state, an image is displayed for multiple consecutive frame times. In some embodiments, the video source may provide multiple copies of the image to the display to write during the consecutive frame times. In other embodiments, the signaling that indicates that consecutive frames are going to retain identical and redundant information compared to an earlier frame is termed a Freeze tag. Accordingly, when embodiments of the Crystal Sweep technique are combined with the Freeze tag, the plurality of redundant frames can be increased, while reducing the amount of data transfer.
[0134] As can be seen, embodiments increase the percentage of the time in which each lens can be held open. In the example in FIG. 13 a display having a 480 Hz display (2.1ms period) is illustrated. Additionally, if four left images or frames or four right images or frames are displayed in a row, a respective lens of the pair of 3D glasses may be open for 75% of the time reserved for the left or right image. Additionally, the interval of darkness 1200 between the left eye opening twice is 10.5 ms (2.1 (for right drawing) plus 3x2.1 (for right redrawing and display) plus 2.1 (for left drawing)). Accordingly, it is expected that no users will experience flickering. In various embodiments, 3D images will appear very bright with no flicker thus providing a user with an enjoyable 3D experience.
[0135] Fig. 14 illustrates a timing diagram according to various embodiments of the present invention. More specifically, Fig. 14 illustrates embodiments of Crystal Sweep with Freeze tag applied to displaying two separate video streams.
[0136] In various embodiments, a plurality of video streams can be multiplexed such that one or a plurality of viewers can view one video stream while a separate plurality of viewers can receive a second video stream. Each of these streams can be monoscopic or stereoscopic. In the case of monoscopic stream (illustrated in Fig. 14), each monoscopic stream requires only half as many display periods as the stereoscopic streams because each monoscopic frame image is sent to both left and right eyes. More specifically, with half as many frames in time each eye gets the same total number of frames viewed and perceives the monoscopic stream at the same brightness as the stereoscopic stream. In one embodiment, the timing may be as follows: draw L, draw/view L, draw/view L, draw 2D, draw/view 2D, draw R, draw/view R, draw/view R, draw 2D, draw/view 2D, etc. In this example of a video display with a 600Hz refresh rate (1.7ms period), the time period 1300 between successive monoscopic images is 6.8ms (1.7 (for right drawing) plus 2x1.7 (for right redrawing and display) plus 1.7 (for monoscopic drawing)). The time period 1310 between successive left or right eye images is 13.6 (1.7 (for monoscopic drawing), 1.7 (for monoscopic displaying), 1.7 (for right drawing), 2x1.7 (for right redrawing and display), 1.7 (for monoscopic drawing), 1.7 (for monoscopic displaying), 1.7 (for left drawing)).
[0137] In various embodiments, as discussed above, the monoscopic images may be advertisements, monoscopic (2D) versions of the feature, text messages, or the like. As an example of the latter, for a pay-per-view event, a video service provider, or the like may allow two users to watch a 3D feature, by enabling appropriate synchronization of two 3D shutter glasses with the output images. Such limiting functionally may be provided by the video display system (e.g. cable box) the radio transmitter, or the like. An additional user whose 3D glasses are not authorized (e.g. has not paid for the pay-per view event) may only see a 2D version of the movie, a 2D message indicating that they need to call a number, hit a button on the cable box or the like. When the additional user performs that operation, the video display system and / or the radio transmitter may then begin to provide the appropriate timing signals to the additional user's 3D glasses to view the 3D content.
[0138] In various embodiments where two 3D images are displayed using the Crystal Sweep technique, the timing may be as follows: draw 1st L, draw/view 1st L, draw 2nd L, draw/view 2nd L, draw 1st R, draw/view 1st R, draw 2nd R, draw/view 2nd R, etc. In such an example, the period of darkness for any eye is expected to be approximately 1 1.9ms (7 x 1.7=(1.7 (for 2nd Left drawing), 1.7 (for 2nd Left redrawing and display), 1.7 (for 1st right drawing), 1.7 (for 1st right redrawing and display), 1.7 (for 2nd right drawing), 1.7 (for 2nd right redrawing and displaying), 1.7 (for 1st left drawing)). In future embodiments, as the refresh rate of displays are expected to increase, Freeze tag may be implemented to increase brightness, while keeping flickering low.
[0139] In still other embodiments, any number of viewing permutations be simply implemented, in light of the present patent application. For example, two viewers may view different 2D images, again using the Crystal Sweep technique: draw 1st 2D, draw/view 1st 2D, draw 2nd 2D, draw/view 2nd 2D, etc.
[0140] In various embodiments of the present invention, in the US, the refresh frequency may be 120Hz, 180Hz, 240Hz, 480Hz, 600Hz, or the like (e.g. multiples of 60Hz); in Europe or other 50Hz countries, the refresh frequency may be 100Hz, 200Hz, 400Hz, 800Hz, or the like (e.g. multiples of 50Hz). In various embodiments, these frequencies are approximate. For example, in the US, according to SMPTE, the refresh frequency may be a multiple of 59.94Hz (59.94= 2*29.97Hz) not 60Hz, per se.
[0141] In a modern video sources, a frame that is being transmitted to the display is often held in one buffer while the video rendering system is generating the video signal for the next frame in a second buffer. This schema is called a double buffer. The double buffer schema can be extended to a plurality of buffers, such as a triple buffer, in which the video rendering subsystem can start rendering a third frame if it completes the second frame and would otherwise need to become idle.
[0142] Figs. 15 and 16 illustrate various embodiments of the present invention. More specifically, Figs. 15 and 16 illustrate triple buffer embodiments.
[0143] In various embodiments of the present invention, to render alternating frames of stereoscopic video, one double buffer is required for the stream of left frames and a second buffer is required for the stream of right frames. This schema is referred to as a side-by-side double buffer. In the prior art, it is sufficient to know when each frame is complete to motivate the shutter glasses to open or close each eye. However, when using Crystal Sweep embodiments, the sequencing mechanism 1400 that alternates between buffers to send frames to this display must transmit each frame more than once. In another embodiment, sequencer 1400 transmits each buffer one or a plurality of times such that the display produces the Crystal Sweep schema.
[0144] In various embodiments, sequencer 1400 also may send a signal indicating that a frame is unchanged from the previous frame, thus transmitting the Crystal Sweep with Freeze tag capability. In various embodiments, sequencer 1400 is provided with a sequence pattern 1410 which describes how many frames from each stream are to be sent, which streams are used for sweeping a new image onto the display and which streams are used for crystallizing the existing image. Sequencer 1400 references the double or triple buffers 1420 for the right and left images, and then outputs both the sequence of images or indications of redundant images to the display. For example, two of the same left images, two of the same right images, etc. Sequencer 1400 also outputs the sequence signal to 3D glasses to enable the 3D glasses to properly switch the 3D glasses in the correct image output sequence, for example, close both lenses, open the left, close both lenses, open the right, and the like.
[0145] In Fig. 16, an embodiment of a system in which includes a plurality of double or triple buffers 1500 is referred to as a Townhouse Buffer enabled the Crystal Sweep with Freeze tag and Multiview, as illustrated in FIG. 14. In this example, sequencer 1510 decodes a plurality of frames from a plurality of buffer stacks 1500 and transmits each to the display or transmits redundant frame signaling information at the appropriate times, as configured by a Sequence Pattern 1520. Similarly, in this example, sequencer 1510 outputs the sequence information to a pair of 3D glasses (to view images Ln and Rn), and to a pair of monoscopic 3D (2D) glasses (to view image "3").
[0146] In various embodiments of the present invention, the double / triple buffers illustrated in Figs. 15 and 16 may be implemented within a graphics processing unit (GPU), a Blu ray player, a stand-alone interface device, or the like.
[0147] In other embodiments of the present invention, 3D glasses may be configured to transmit timing data back to the 3D source. The ability for bidirectional communications also allows a more precise timing protocol to be implemented between the 3D source and 3D glasses. As an example, the transmitter (3D source) and the receiver (3D glasses) each run local clocks that operate at a multiple of the frequency (e.g. 20KHz) of the sequence information (e.g. 120Hz) and are divided down to a lower speed which matches the sequence information. Then, the transmitter and the receiver exchange a series of messages at the higher frequency containing timestamps which indicate the value of their local clocks. By exchanging a series of these messages the transmitter and receiver can determine the difference in speed of their local clocks and compensate for these differences. The result is that synchronization is achieved with a much higher precision than the period of the sequence information.
[0148] In various embodiment, when there are multiple 3D displays and 3D transmitters present, a user using a pair of 3D glasses that perceives stereoscopic information from one 3D display should not also perceive stereoscopic information from another 3D display at the same time. This is unless each 3D display is synchronized in time and uses the same left/right sequence. For example, all the 3D displays must start the left frame at the same time and the right frame at the same time. In examples such as an event with multiple monitors, synchronization between these 3D displays may be performed by coordination of the 3D source devices or a single 3D source device.
[0149] In light of the above disclosure, one of ordinary skill in the art would recognize that many variations may be implemented based upon the discussed embodiments. Embodiments described above may be useful for hand-held consumer devices such as cell-phones, personal media players, mobile internet devices, or the like. Other embodiments may also be applied to higher-end devices such as laptop computers, desktop computers, DVRs. BluRay players, gaming consoles, hand-held portable devices, or the like. Other embodiments may take advantage of existing IR transmission devices for IR shutter glasses. More specifically, in such embodiments, an IR to RF conversion portion may be added to receive the IR 3D output instructions and to convert them to RF 3D transmission signals, described above. In some embodiments, an RF receiver is thus used. The RF 3D transmission signals are then transmitted to the RF 3D shutter glasses, described above. Such embodiments can therefore be a simple upgrade to available IR 3D glasses transmitters.
[0150] In other embodiments of the present invention, feedback from shutter glasses to the transmitter device described above with regards to synchronization, may be used for additional purposes. One such embodiment may allow the 3D image source (e.g. a cable box, computer, or the like) to take the indication that a pair of shutter glasses are currently synchronized to mean a person is viewing the 3D content, and to provide that data back to a marketing company such as Media Metrics, Nielsen Ratings, or the like. By doing this, such market research companies may determine the number of viewers of specific 3D features, or the like. [0151] The above detailed description is directed to systems, methods, and computer- readable media for stereoscopic viewing. While the subject matter described herein is presented in the general context of program modules that execute in conjunction with the execution of an application program or an operating system on a 3D source, consumer electronics device, and a pair of stereoscopic glasses, those skilled in the art will recognize that other implementations may be performed in combination with other program modules or devices.
[0152] Further embodiments can be envisioned to one of ordinary skill in the art after reading this disclosure. In other embodiments, combinations or sub-combinations of the above disclosed invention can be advantageously made. The block diagrams of the architecture and flow charts are grouped for ease of understanding. However it should be understood that combinations of blocks, additions of new blocks, re-arrangement of blocks, and the like are contemplated in alternative embodiments of the present invention.
[0153] The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereunto without departing from the broader spirit and scope.

Claims

WHAT IS CLAIMED IS:
1. A method for providing three-dimensional video images to a user on a display system programmed to implement the method comprising:
while a right-eye lens of three dimensional glasses of the display system is in an opaque state, the method comprising:
transferring with a video driver of the display system a first instance of a left-eye image to a video display device during a first frame time, while a left-eye lens of three-dimensional glasses of the display system is in an opaque state; and
transferring with the video driver of the display system a second instance of the left-eye image to the video display device during a second frame time, while the left-eye lens of the three-dimensional glasses of the display system is in a non-opaque state, wherein the first frame time and the second frame time are adjacent; and thereafter while the left-eye lens of the three dimensional glasses of the display system is in the opaque state, the method comprising:
transferring with the video driver of the display system a first instance of a right-eye image to the video display device during a third frame time, while the right-eye lens of the three-dimensional glasses of the display system is in the opaque state; and
transferring with the video driver of the display system a second instance of the right-eye image to the video display device during a fourth frame time, while the right-eye lens of the three-dimensional glasses of the display system is in a non-opaque state, wherein the third frame time and the fourth frame time are adjacent.
2. The method of claim 1 wherein the first frame time and the second frame time are equal and each have a duration selected from a group consisting of: 1/120 second, 1/180 second, 1/240 second, 1/480 second, and 1/600 second.
3. The method of claim 1 wherein the second frame time and third frame time are adjacent.
4. The method of claim 1 while the right-eye lens of the three dimensional glasses of the display system is in the opaque state, the method further comprising:
transferring with the video driver of the display system a third instance of the left-eye image to the video display device of the display system during a fifth frame time; and transferring with the video driver of the display system a fourth instance of the left-eye image to the video display device of the display system during a sixth frame time;
wherein the second frame time is adjacent to the fifth frame time; and wherein the fifth frame time is adjacent to the sixth frame time.
5. The method of claim 1 further comprising:
storing by the video driver, the left-eye image into a memory of the video driver;
accessing with the video driver, the left-eye image stored in the memory with the video driver to provide the first instance of the left-eye image;
reaccessing with the video driver, the left-eye image stored in the memory with the video driver to provide the second instance of the left-eye image.
6. The method of claim 1 further comprising:
receiving with the video driver, the left-eye image stored from a video source with the video driver to provide the first instance of the left-eye image; and
receiving with the video driver, the left-eye image stored from the video source with the video driver to provide the second instance of the left-eye image.
7. The method of claim 1 further comprising instructing with the video driver, the left-eye lens of the three dimensional glasses to enter the opaque state during the first time frame and the left-eye lens of the three dimensional glasses to enter the non-opaque state during the second time frame.
8. The method of claim 7 wherein the instructing with the video driver comprises wirelessly transmitting timing data to the three dimensional glasses.
9. The method of claim 7
wherein the instructing with the video driver comprises transmitting the timing data via a radio frequency transmitter to the three dimensional glasses; and
wherein radio signals associated with the radio frequency transmitter are selected from a group consisting of: IEEE Standard 802.15.1, 802.15.4, 802.1 1, Bluetooth, ZigBee, WiFi, Z-Wave.
10. A display system for providing three-dimensional video images to a user comprising: a pair of three dimensional glasses comprising:
a right-eye lens configured to have an opaque state or a non-opaque state in response to right-eye timing data;
a left-eye lens configured to have an opaque state or a non-opaque state in response to left-eye timing data;
a wireless communication receiver configured to receiving timing information from a video processor; and
a processor coupled to the right-eye lens, the left-eye lens, and to the wireless communication receiver, wherein the processor is configured to receive timing information from the wireless communication receiver, wherein the processor is configured to provide left-eye timing data to the left-eye lens for a first frame time and a second frame time in response to the timing information, and wherein the processor is configured to provide right-eye timing data to the right-eye lens for a third frame time and a fourth frame time in response to the timing information, wherein the first frame time and the second frame time are sequential, and wherein the third frame time and the fourth frame time are sequential; and
the video processor coupled to the pair of three dimensional glasses and to a video source comprising:
a processor configured to determine the timing information in response to video data from the video source; and
a wireless communication transmitter configured to transmit the timing information to the pair of three dimensional glasses.
1 1. The display system of claim 10 wherein the first frame time and the second frame time are equal and each have a duration selected from a group consisting of: approximately: 1/120 second, 1/180 second, 1/240 second, 1/480 second, 1/600 second, 1/100 second, 1/200 second, 1/400 second, and 1/800 second.
12. The display system of claim 10 wherein the second frame time and third frame time are adjacent.
13. The display system of claim 10
wherein the processor is configured to provide left-eye timing data to the left- eye lens for a fifth frame time and a sixth frame time in response to the timing information, wherein the second frame time and the fifth frame time are sequential.
14. The display system of claim 10 further comprising a video output device coupled to the video source, wherein the video output device is configured to receive the video data from the video source comprising a left-eye image and a right-eye image, wherein the video output device is configured to write the left-eye image to a display portion during the first frame time and during the second frame time, and wherein the video output device is configured to write the right-eye image to the display portion during the third frame time and during the fourth frame time.
15. The display system of claim 14 wherein the video output device is configured to receive an instance of the left-eye image from the video source for output during the first frame time and configured to receive another instance of the left-eye image from the video source for output during the second frame time.
16. The display system of claim 14 wherein the video output device is configured to receive a single instance of the left-eye image from the video source for output during the first frame time and for output during the second frame time.
17. The display system of claim 10 wherein the wireless communication receiver comprises a radio frequency receiver, wherein wireless communication radio signals are selected from a group consisting of: IEEE Standard 802.15.1, 802.15.4, 802.1 1,
Bluetooth, ZigBee, WiFi, Z-Wave.
18. The display system of claim 11 wherein the video processor and the video source are both integrated into a single unit.
19. A video display system for driving a video display and a pair of 3D glasses comprises:
a sequence memory configured to store an output image sequence;
a first double-buffer memory, wherein the first double-buffer memory is configured to simultaneously store and output a first left-eye image while receiving and storing a second left-eye image;
a second double-buffer memory, wherein the second double-buffer memory is configured to simultaneously store and output a first right-eye image while receiving and storing a second right-eye image; and
a sequencer coupled to the sequence memory, to the first double-buffer memory, and to the second double-buffer memory, wherein the sequencer is configured to output the first left-eye image to a video display during a first frame time, to output the first left-eye image to the video display during a second frame time, to output first timing information to instruct a left-eye lens of a pair of 3D glasses to enter an opaque state during the first frame time, and to output second timing information to instruct the left-eye lens of the pair of 3D glasses to enter a non-opaque state during the second frame time, in response to the output image sequence.
20. The video display system of claim 19 further comprising a graphics processing unit configured to determine the first left-eye image, to determine the second left-eye image, to determine the first right-eye image, and to determine the second right-eye image.
PCT/US2010/052951 2009-10-15 2010-10-15 System and method of displaying three dimensional images using crystal sweep with freeze tag WO2011047343A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US25173909P 2009-10-15 2009-10-15
US61/251,739 2009-10-15
US30096110P 2010-02-03 2010-02-03
US61/300,961 2010-02-03

Publications (2)

Publication Number Publication Date
WO2011047343A2 true WO2011047343A2 (en) 2011-04-21
WO2011047343A3 WO2011047343A3 (en) 2011-07-28

Family

ID=43876917

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2010/052951 WO2011047343A2 (en) 2009-10-15 2010-10-15 System and method of displaying three dimensional images using crystal sweep with freeze tag

Country Status (1)

Country Link
WO (1) WO2011047343A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102865996A (en) * 2011-07-05 2013-01-09 纬创资通股份有限公司 Test method, device and system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7218339B2 (en) * 2002-01-22 2007-05-15 Kenneth Jacobs Eternalism, a method for creating an appearance of sustained three-dimensional motion-direction of unlimited duration, using a finite number of pictures
KR20070089023A (en) * 2006-02-27 2007-08-30 삼성전자주식회사 High resolution 2d-3d switchable autostereoscopic display apparatus
US20080151040A1 (en) * 2006-12-26 2008-06-26 Samsung Electronics Co., Ltd. Three-dimensional image display apparatus and method and system for processing three-dimensional image signal
KR100914806B1 (en) * 2001-06-23 2009-09-02 톰슨 라이센싱 Method and device for processing video frames for stereoscopic display on a display device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100914806B1 (en) * 2001-06-23 2009-09-02 톰슨 라이센싱 Method and device for processing video frames for stereoscopic display on a display device
US7218339B2 (en) * 2002-01-22 2007-05-15 Kenneth Jacobs Eternalism, a method for creating an appearance of sustained three-dimensional motion-direction of unlimited duration, using a finite number of pictures
KR20070089023A (en) * 2006-02-27 2007-08-30 삼성전자주식회사 High resolution 2d-3d switchable autostereoscopic display apparatus
US20080151040A1 (en) * 2006-12-26 2008-06-26 Samsung Electronics Co., Ltd. Three-dimensional image display apparatus and method and system for processing three-dimensional image signal

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102865996A (en) * 2011-07-05 2013-01-09 纬创资通股份有限公司 Test method, device and system

Also Published As

Publication number Publication date
WO2011047343A3 (en) 2011-07-28

Similar Documents

Publication Publication Date Title
US20110090324A1 (en) System and method of displaying three dimensional images using crystal sweep with freeze tag
US20140184762A1 (en) Method of stereoscopic synchronization of active shutter glasses
US20100194857A1 (en) Method of stereoscopic 3d viewing using wireless or multiple protocol capable shutter glasses
US20110001805A1 (en) System and method of transmitting and decoding stereoscopic sequence information
EP2438763A2 (en) Method of stereoscopic synchronization of active shutter glasses
JP4818469B2 (en) Video viewing glasses and control method for video viewing glasses
US9179136B2 (en) Method and system for synchronizing 3D shutter glasses to a television refresh rate
US8896676B2 (en) Method and system for determining transmittance intervals in 3D shutter eyewear based on display panel response time
US9143767B2 (en) Method and apparatus for adaptive stable image timing in stereoscopic 3D systems
US20120190439A1 (en) Multiple simultaneous programs on a display
JP4958993B2 (en) Video display device and control method thereof
US20110254829A1 (en) Wearable electronic device, viewing system and display device as well as method for operating a wearable electronic device and method for operating a viewing system
JP2011193460A (en) Method for adjusting 3d-image quality, 3d-display apparatus, 3d-glasses, and system for providing 3d-image
EP2394195A2 (en) Method of stereoscopic 3d image capture and viewing
US20130194399A1 (en) Synchronization of shutter signals for multiple 3d displays/devices
US9167238B2 (en) 3D display apparatus for use in synchronization with 3D glasses and 3D display method thereof
WO2011047343A2 (en) System and method of displaying three dimensional images using crystal sweep with freeze tag
KR101768538B1 (en) Method for adjusting 3-Dimension image quality, 3D display apparatus, 3D glasses and System for providing 3D image
EP2334086A1 (en) Stereoscopic display device
TWI517668B (en) Stereoscopic glasses control method, system and computer program
KR20140112122A (en) Multiple display game providing apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10824220

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10824220

Country of ref document: EP

Kind code of ref document: A2