US10558036B2 - Using oscillation of optical components to reduce fixed pattern noise in a virtual reality headset - Google Patents
Using oscillation of optical components to reduce fixed pattern noise in a virtual reality headset Download PDFInfo
- Publication number
- US10558036B2 US10558036B2 US15/376,844 US201615376844A US10558036B2 US 10558036 B2 US10558036 B2 US 10558036B2 US 201615376844 A US201615376844 A US 201615376844A US 10558036 B2 US10558036 B2 US 10558036B2
- Authority
- US
- United States
- Prior art keywords
- sub
- electronic display
- image light
- pixels
- display element
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0025—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for optical correction, e.g. distorsion, aberration
- G02B27/0068—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for optical correction, e.g. distorsion, aberration having means for controlling the degree of correction, e.g. using phase modulators, movable elements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
- G02B26/101—Scanning systems with both horizontal and vertical deflecting means, e.g. raster or XY scanners
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
- G06F1/1637—Details related to the display arrangement, including those related to the mounting of the display in the housing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0118—Head-up displays characterised by optical features comprising devices for improving the contrast of the display / brillance control visibility
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/64—Imaging systems using optical elements for stabilisation of the lateral and angular position of the image
- G02B27/646—Imaging systems using optical elements for stabilisation of the lateral and angular position of the image compensating for small deviations, e.g. due to vibration or shake
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
- G09G2340/04—Changes in size, position or resolution of an image
- G09G2340/0457—Improvement of perceived resolution by subpixel rendering
Definitions
- the present disclosure generally relates to reducing fixed pattern noise, and specifically relates to reducing fixed pattern noise using piezoelectric materials to oscillate components of a display.
- Electronic displays include a plurality of pixels, which may each include a plurality of sub-pixels (e.g., a red sub-pixel, a green sub-pixel, etc.). Arrangement of individual sub-pixels may affect the appearance and performance of an electronic display device. Some arrangements of sub-pixels may increase fixed pattern noise under certain conditions. For example, magnification of a pixel may result in boundaries between individual sub-pixels of the pixel becoming visible to the user, resulting in a “screen door” pattern (i.e., an increase in fixed pattern noise) in an image presented to a user.
- a “screen door” pattern i.e., an increase in fixed pattern noise
- An image presented by an electronic display element within a virtual reality (VR) headset of a VR system is magnified beyond a threshold amount (e.g., magnified several-fold).
- a threshold amount e.g., magnified several-fold.
- an optics block included in the VR headset may cause a user viewing the image through the VR headset to see individual sub-pixels of a pixel in the image, as well as dark space between the sub-pixels (i.e., fixed pattern noise or the “screen door effect”).
- the electronic display element of the VR headset or an optics block of the VR headset is oscillated to blur light from the sub-pixels to mask the fixed pattern noise.
- the VR headset includes an electronic display including an electronic display element that outputs image light via a plurality of sub-pixels.
- various sub-pixels have different colors and are separated from each other by a dark space.
- the electronic display includes different electronic display elements for each eye of the user. Alternatively, a single electronic display element is used to display images to both eyes of the user.
- an optics block included in the VR headset is oscillated to reduce fixed pattern noise.
- a piezoelectric material is coupled to the optics block that vibrates the optics block.
- the optics block includes a lens made from a piezoelectric material that causes the lens to vibrate.
- a piezoelectric material is coupled to the electronic display element in the VR headset, causing the electronic display element to vibrate and reduce fixed pattern noise in image data presented to a user via the VR headset.
- FIG. 1 is a block diagram of a system environment including a virtual reality system, in accordance with an embodiment.
- FIG. 2A is a wire diagram of a virtual reality headset, in accordance with an embodiment.
- FIG. 2B is a cross section of a front rigid body of the VR headset in FIG. 2A , in accordance with an embodiment.
- FIG. 2C is a cross section of a front rigid body of the VR headset in FIG. 2A with a piezoelectric element coupled to the electronic display element, in accordance with an embodiment.
- FIG. 3A is an example array of sub-pixels on an electronic display element, in accordance with an embodiment.
- FIG. 3B is an example of image light emitted by an array of sub-pixels and adjusted by oscillation of an optics block or of an electronic display element, in accordance with an embodiment.
- FIG. 1 is a block diagram of a virtual reality (VR) system environment 100 in which a VR console 110 operates.
- the system environment 100 shown by FIG. 1 comprises a VR headset 105 , an imaging device 135 , and a VR input interface 140 that are each coupled to the VR console 110 .
- FIG. 1 shows an example system 100 including one VR headset 105 , one imaging device 135 , and one VR input interface 140 , in other embodiments any number of these components may be included in the system 100 .
- different and/or additional components may be included in the system environment 100 .
- the VR headset 105 is a head-mounted display that presents media to a user. Examples of media presented by the VR head set include one or more images, video, audio, or some combination thereof. In some embodiments, audio is presented via an external device (e.g., speakers and/or headphones) that receives audio information from the VR headset 105 , the VR console 110 , or both, and presents audio data based on the audio information.
- An embodiment of the VR headset 105 is further described below in conjunction with FIGS. 2A and 2B .
- the VR headset 105 may comprise one or more rigid bodies, which may be rigidly or non-rigidly coupled to each other together. A rigid coupling between rigid bodies causes the coupled rigid bodies to act as a single rigid entity. In contrast, a non-rigid coupling between rigid bodies allows the rigid bodies to move relative to each other.
- the VR headset 105 includes an electronic display 115 , a corrective element 116 , an optics block 118 , one or more locators 120 , one or more position sensors 125 , and an inertial measurement unit (IMU) 130 .
- the electronic display 115 displays images to the user in accordance with data received from the VR console 110 .
- the electronic display 115 may comprise a single electronic display or multiple electronic displays (e.g., a display for each eye of a user). Examples of the electronic display 115 include: a liquid crystal display (LCD), an organic light emitting diode (OLED) display, an active-matrix organic light-emitting diode display (AMOLED), some other display, or some combination thereof.
- LCD liquid crystal display
- OLED organic light emitting diode
- AMOLED active-matrix organic light-emitting diode display
- the electronic display 115 includes a display area comprising a plurality of sub-pixels, where a sub-pixel is a discrete light emitting component. Different sub-pixels are separated from each other by dark space. For example, a sub-pixel emits red light, yellow light, blue light, green light, white light, or any other suitable color of light. In some embodiments, images projected by the electronic display 115 are rendered on the sub-pixel level.
- RGB red-green-blue
- each pixel in the RGB layout includes a red sub-pixel, which is adjacent to a green sub-pixel that is adjacent to a blue sub-pixel; the red, green, and blue sub-pixels operate together to form different colors.
- a sub-pixel in a pixel is restricted to working within that pixel.
- sub-pixels in the electronic display operate within multiple “logical” pixels in their surrounding vicinity to form different colors.
- the sub-pixels are arranged on the display area of the electronic display 115 in a sub-pixel array. Examples of a sub-pixel array include PENTILE® RGBG, PENTILE® RGBW, some another suitable arrangement of sub-pixels that renders images at the sub-pixel level.
- one or more adjacent sub-pixels are of the same color.
- the display area of the electronic display 115 arranges sub-pixels in a hexagonal layout, in contrast to a rectangular layout used by conventional RGB type systems. Moreover, some users are more comfortable viewing images which appear to have been generated via a rectangular layout of sub-pixels.
- the corrective element 116 may be configured to generate amounts of blur that causes the array of sub-pixels in the image presented to the user to appear to be arrayed in a rectangular layout.
- the optics block 118 magnifies received light, corrects optical errors associated with the image light, and presents the corrected image light is presented to a user of the VR headset 105 .
- An optical element may be an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, or any other suitable optical element that affects the blurred image light.
- the optics block 118 may include combinations of different optical elements.
- one or more of the optical elements in the optics block 118 may have one or more coatings, such as anti-reflective coatings.
- Magnification of the image light by the optics block 118 allows the electronic display 115 to be physically smaller, weigh less, and consume less power than larger displays. Additionally, magnification may increase a field of view of the displayed media. For example, the field of view of the displayed media is such that the displayed media is presented using almost all (e.g., 110 degrees diagonal), and in some cases all, of the user's field of view.
- magnification of image light may cause an increase in fixed pattern noise, also referred to as the “screen door effect,” which is a visual artifact where dark space separating pixels and/or sub-pixels of a display become visible to a user in an image presented by the display.
- the optics block 118 vibrates to blur received image light to cover the dark space between the adjacent sub-pixels.
- a piezoelectric material is coupled to the optics block 118 and vibrates the optics block 118 to blur image light received by the optics block 118 .
- the optics block 118 includes one or more components, such as lenses, made from piezoelectric material that vibrates to blur received image light, as further described below in conjunction with FIG. 2B .
- the optics block 118 may be designed so its effective focal length is larger than the spacing to the electronic display 115 , which magnifies the image light projected by the electronic display 115 .
- the amount of magnification may be adjusted by adding or removing optical elements.
- the optics block 118 may be designed to correct one or more types of optical error in addition to fixed pattern noise (i.e., the screen door effect).
- optical error include: two dimensional optical errors, three dimensional optical errors, or some combination thereof.
- Two dimensional errors are optical aberrations that occur in two dimensions.
- Example types of two dimensional errors include: barrel distortion, pincushion distortion, longitudinal chromatic aberration, transverse chromatic aberration, or any other type of two-dimensional optical error.
- Three dimensional errors are optical errors that occur in three dimensions.
- Example types of three dimensional errors include spherical aberration, comatic aberration, field curvature, astigmatism, or any other type of three-dimensional optical error.
- content provided to the electronic display 115 for display is pre-distorted, and the optics block 118 corrects the distortion when is receives image light from the electronic display 115 generated based on the content.
- the locators 120 are objects located in specific positions on the VR headset 105 relative to one another and relative to a specific reference point on the VR headset 105 .
- a locator 120 may be a light emitting diode (LED), a corner cube reflector, a reflective marker, a type of light source that contrasts with an environment in which the VR headset 105 operates, or some combination thereof.
- the locators 120 may emit light in the visible band ( ⁇ 380 nm to 750 nm), in the infrared (IR) band ( ⁇ 750 nm to 1 mm), in the ultraviolet band (10 nm to 380 nm), some other portion of the electromagnetic spectrum, or some combination thereof.
- the locators 120 are located beneath an outer surface of the VR headset 105 , which is transparent to the wavelengths of light emitted or reflected by the locators 120 or is thin enough to not substantially attenuate the wavelengths of light emitted or reflected by the locators 120 . Additionally, in some embodiments, the outer surface or other portions of the VR headset 105 are opaque in the visible band of wavelengths of light. Thus, the locators 120 may emit light in the IR band under an outer surface that is transparent in the IR band but opaque in the visible band.
- the IMU 130 is an electronic device that generates fast calibration data based on measurement signals received from one or more of the position sensors 125 .
- a position sensor 125 generates one or more measurement signals in response to motion of the VR headset 105 .
- Examples of position sensors 125 include: one or more accelerometers, one or more gyroscopes, one or more magnetometers, another suitable type of sensor that detects motion, a type of sensor used for error correction of the IMU 130 , or some combination thereof.
- the position sensors 125 may be located external to the IMU 130 , internal to the IMU 130 , or some combination thereof.
- the IMU 130 Based on the one or more measurement signals from one or more position sensors 125 , the IMU 130 generates fast calibration data indicating an estimated position of the VR headset 105 relative to an initial position of the VR headset 105 .
- the position sensors 125 include multiple accelerometers to measure translational motion (forward/back, up/down, left/right) and multiple gyroscopes to measure rotational motion (e.g., pitch, yaw, roll).
- the IMU 130 rapidly samples the measurement signals and calculates the estimated position of the VR headset 105 from the sampled data.
- the IMU 130 integrates the measurement signals received from the accelerometers over time to estimate a velocity vector and integrates the velocity vector over time to determine an estimated position of a reference point on the VR headset 105 .
- the IMU 130 provides the sampled measurement signals to the VR console 110 , which determines the fast calibration data.
- the reference point is a point that may be used to describe the position of the VR headset 105 . While the reference point may generally be defined as a point in space; however, in practice the reference point is defined as a point within the VR headset 105 (e.g., a center of the IMU 130 ).
- the IMU 130 receives one or more calibration parameters from the VR console 110 . As further discussed below, the one or more calibration parameters are used to maintain tracking of the VR headset 105 . Based on a received calibration parameter, the IMU 130 may adjust one or more IMU parameters (e.g., sample rate). In some embodiments, certain calibration parameters cause the IMU 130 to update an initial position of the reference point so it corresponds to a next calibrated position of the reference point. Updating the initial position of the reference point as the next calibrated position of the reference point helps reduce accumulated error associated with the determined estimated position. The accumulated error, also referred to as drift error, causes the estimated position of the reference point to “drift” away from the actual position of the reference point over time.
- drift error causes the estimated position of the reference point to “drift” away from the actual position of the reference point over time.
- the imaging device 135 generates slow calibration data in accordance with calibration parameters received from the VR console 110 .
- Slow calibration data includes one or more images showing observed positions of the locators 120 that are detectable by the imaging device 135 .
- the imaging device 135 may include one or more cameras, one or more video cameras, any other device capable of capturing images including one or more of the locators 120 , or some combination thereof. Additionally, the imaging device 135 may include one or more filters (e.g., used to increase signal to noise ratio).
- the imaging device 135 is configured to detect light emitted or reflected from locators 120 in a field of view of the imaging device 135 .
- the imaging device 135 may include a light source that illuminates some or all of the locators 120 , which retro-reflect the light towards the light source in the imaging device 135 .
- Slow calibration data is communicated from the imaging device 135 to the VR console 110 , and the imaging device 135 receives one or more calibration parameters from the VR console 110 to adjust one or more imaging parameters (e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.).
- the VR input interface 140 is a device that allows a user to send action requests to the VR console 110 .
- An action request is a request to perform a particular action.
- An action request may be to start or end an application or to perform a particular action within the application.
- the VR input interface 140 may include one or more input devices.
- Example input devices include: a keyboard, a mouse, a game controller, or any other suitable device for receiving action requests and communicating the received action requests to the VR console 110 .
- An action request received by the VR input interface 140 is communicated to the VR console 110 , which performs an action corresponding to the action request.
- the VR input interface 140 may provide haptic feedback to the user in accordance with instructions received from the VR console 110 . For example, haptic feedback is provided when an action request is received, or the VR console 110 communicates instructions to the VR input interface 140 causing the VR input interface 140 to generate haptic feedback when the VR console 110 performs an action.
- the VR console 110 provides media to the VR headset 105 for presentation to the user in accordance with information received from one or more of: the imaging device 135 , the VR headset 105 , and the VR input interface 140 .
- the VR console 110 includes an application store 145 , a tracking module 150 , and a virtual reality (VR) engine 155 .
- Some embodiments of the VR console 110 have different modules than those described in conjunction with FIG. 1 .
- the functions further described below may be distributed among components of the VR console 110 in a different manner than is described here.
- the application store 145 stores one or more applications for execution by the VR console 110 .
- An application is a group of instructions, that when executed by a processor, generates content for presentation to the user. Content generated by an application may be in response to inputs received from the user via movement of the HR headset 105 or the VR interface device 140 . Examples of applications include: gaming applications, conferencing applications, video playback application, or other suitable applications.
- the tracking module 150 calibrates the VR system 100 using one or more calibration parameters and may adjust one or more calibration parameters to reduce error in determination of the position of the VR headset 105 . For example, the tracking module 150 adjusts the focus of the imaging device 135 to obtain a more accurate position for observed locators on the VR headset 105 . Moreover, calibration performed by the tracking module 150 also accounts for information received from the IMU 130 . Additionally, if tracking of the VR headset 105 is lost (e.g., the imaging device 135 loses line of sight of at least a threshold number of the locators 120 ), the tracking module 140 re-calibrates some or all of the system environment 100 .
- the tracking module 140 re-calibrates some or all of the system environment 100 .
- the tracking module 150 tracks movements of the VR headset 105 using slow calibration information from the imaging device 135 . As further described below in conjunction with FIG. 3 , the tracking module 150 determines positions of a reference point of the VR headset 105 using observed locators from the slow calibration information and a model of the VR headset 105 . The tracking module 150 also determines positions of a reference point of the VR headset 105 using position information from the fast calibration information. Additionally, in some embodiments, the tracking module 150 may use portions of the fast calibration information, the slow calibration information, or some combination thereof, to predict a future location of the headset 105 . The tracking module 150 provides the estimated or predicted future position of the VR headset 105 to the VR engine 155 .
- the VR engine 155 executes applications within the system environment 100 and receives position information, acceleration information, velocity information, predicted future positions, or some combination thereof of the VR headset 105 from the tracking module 150 . Based on the received information, the VR engine 155 determines content to provide to the VR headset 105 for presentation to the user. For example, if the received information indicates that the user has looked to the left, the VR engine 155 generates content for the VR headset 105 that mirrors the user's movement in a virtual environment. Additionally, the VR engine 155 performs an action within an application executing on the VR console 110 in response to an action request received from the VR input interface 140 and provides feedback to the user that the action was performed. The provided feedback may be visual or audible feedback via the VR headset 105 or haptic feedback via the VR input interface 140 .
- FIG. 2A is a wire diagram of a virtual reality (VR) headset, in accordance with an embodiment.
- the VR headset 200 is an embodiment of the VR headset 105 , and includes a front rigid body 205 and a band 210 .
- the front rigid body 205 includes one or more electronic display elements of the electronic display 115 (not shown), the IMU 130 , the one or more position sensors 125 , and the locators 120 .
- the position sensors 125 are located within the IMU 130 , and neither the IMU 130 nor the position sensors 125 are visible to the user.
- the locators 120 are located in fixed positions on the front rigid body 205 relative to one another and relative to a reference point 215 .
- the reference point 215 is located at the center of the IMU 130 .
- Each of the locators 120 emit light that is detectable by the imaging device 135 .
- Locators 120 or portions of locators 120 , are located on a front side 220 A, a top side 220 B, a bottom side 220 C, a right side 220 D, and a left side 220 E of the front rigid body 205 in the example of FIG. 2A .
- FIG. 2B is a cross section 225 of the front rigid body 205 of the embodiment of a VR headset 200 shown in FIG. 2A .
- the front rigid body 205 includes an optical block 230 that provides altered image light to an exit pupil 250 .
- the exit pupil 250 is the location of the front rigid body 205 where a user's eye 245 is positioned.
- FIG. 2B shows a cross section 225 associated with a single eye 245 , but another optical block, separate from the optical block 230 , provides altered image light to another eye of the user.
- the optical block 230 includes an electronic display element 235 of the electronic display 115 , the optics block 118 , and a piezoelectric element 240 coupled to the optics block 118 .
- the piezoelectric element 240 is a piezoelectric material, such as polyvinylidene difluoride (PEVDF) or another suitable piezoelectric material.
- PEVDF polyvinylidene difluoride
- a voltage is applied to the piezoelectric element 240 , causing the piezoelectric material comprising the piezoelectric element 240 to vibrate.
- the piezoelectric element 240 vibrates at a frequency matching a frequency of the fixed pattern noise but out of phase with the fixed pattern noise to attenuate or cancel the fixed pattern noise.
- the piezoelectric element 240 vibrates at a frequency that modulates the fixed pattern noise.
- the piezoelectric element 240 vibrates at a frequency that spreads a spectrum of the fixed pattern noise or moves the fixed pattern noise to one or more frequencies to which a user is less visually sensitive.
- vibration of the piezoelectric element causes the optics block 118 to oscillate and blur image light emitted by the electronic display element 235 and directed towards the optics block 118 .
- Blurring of the image light by the oscillating optics block 118 corrects for fixed pattern noise, and the blurred image light is directed towards the exit pupil 250 for presentation to the user.
- the optics block 118 may also correct for one or more additional optical errors (e.g., distortion, astigmatism, etc.).
- the optics block 118 directs the corrected image light to the exit pupil 250 for presentation to the user.
- the piezoelectric element 240 is coupled to the optics block 118 so the optics block 118 oscillates perpendicularly to the exit pupil 250 .
- vibration of the piezoelectric element 240 causes the optical block 230 to oscillate horizontally.
- the piezoelectric element 230 is coupled to the optics block 118 so the optics block 118 oscillates parallel to the exit pupil 250 .
- the optics block 118 in FIG. 2B oscillates vertically in certain embodiments.
- FIG. 2B shows an embodiment where the piezoelectric element 240 is coupled to the optics block 118
- one or more components in the optics block 118 comprise piezoelectric materials.
- a lens included in the optics block 118 comprises a piezoelectric material. Applying a voltage to the lens causes the lens to vibrate and blur image light from the electronic display element 235 to compensate for fixed pattern noise.
- Various piezoelectric materials, such as those described above, may comprise the lens in various embodiments.
- FIG. 2C shows an alternative embodiment where the piezoelectric element 240 is coupled to the electronic display element 235 .
- the piezoelectric element 240 vibrates when a voltage is applied to it, which causes the electronic display element 235 to oscillate.
- the oscillation of the electronic display element blurs image light transmitted towards the optics block 118 by the electronic display element 235 , which compensates for fixed pattern noise.
- the optics block 118 magnifies the blurred image light to generate an image that is corrected for fixed pattern noise and that is directed to the exit pupil 250 .
- vibration of the piezoelectric element 240 causes the electronic display element 235 to oscillate perpendicularly to the exit pupil 250 (e.g., horizontally) or to oscillate parallel to the exit pupil 250 (e.g., vertically).
- a piezoelectric element 240 coupled to the electronic display element 235 oscillates the electronic display element as a piezoelectric element 240 coupled to the optics block 118 causes the optics block 118 to oscillate.
- the piezoelectric element 240 comprises different materials when coupled to the electronic display element 235 and when coupled to the optics block 118 ; alternatively, the piezoelectric element 240 comprises a common material when coupled to the electronic display element 235 and when coupled to the optics block 118 .
- any suitable device having high activation bandwidth and controllable force effects may be used in the piezoelectric element 240 .
- Example devices include a linear motor, voice coil actuator, thermo electric driver, and a media phase change driver.
- FIGS. 2B and 2C show embodiments where a piezoelectric element 240 is coupled to the optics block 118 and to the electronic display element, respectively, in other embodiments, any suitable vibrating element may be coupled to the optics block or to the electronic display element 235 .
- a micro electrical mechanical system (MEMS) oscillator or resonator is coupled to the optics block 118 or to the electronic display element 235 .
- MEMS micro electrical mechanical system
- any suitable component capable of inducing oscillation may be coupled to the optics block 118 or to the electronic display element 235 .
- FIG. 3A is an example array 300 of sub-pixels on the electronic display element 235 .
- the example array 300 shown in FIG. 3A includes red sub-pixels 310 , blue sub-pixels 320 , and green sub-pixels 330 .
- the array 300 is portion of a PENTILE® display.
- a dark space 340 separates each sub-pixel from one or more adjacent sub-pixels.
- the dark space 340 is a portion of the array 300 that does not emit light, and may become visible to a user under certain circumstances (e.g., magnification) causing the screen door effect that degrades image quality.
- magnification magnification
- the optics block 118 or the electronic display element 235 may be oscillated to blur image light transmitted by the electronic display element to reduce fixed pattern noise so the dark space 340 between the sub-pixels is not visible to the user.
- the electronic display element 235 or the optics block 118 may oscillate so image light generated by different sub pixels is blurred to generate blur spots that are large enough to mask the dark space 340 between adjacent sub-pixels, while still allowing the image light from adjacent sub-pixels to be resolved by a user as image light from separate sub-pixels.
- FIG. 3B is an example of image light emitted by an array of sub-pixels (e.g., array 300 ) and adjusted by oscillation of the optics block 118 or of the electronic display element 235 .
- each of the sub-pixels has an associated blur spot.
- the red sub-pixels 310 have a corresponding red blur spot 360
- the blue sub-pixels 320 have a corresponding blue blur spot 370
- the green sub-pixels 330 have a corresponding green blur spot 380 .
- a blur spot is an area filled with an image of a blurred sub-pixel.
- a blur spot does not overlap with a point of maximum intensity of an adjacent blur spot, the two blur spots are resolvable as two adjacent sub-pixels by a user.
- FIG. 3B shows blur spots a circular
- a blur spot is an area including the blurred image of a sub-pixel and may have any suitable shape.
- the electronic display element 235 or the optics block 118 is configured to vibrate so that after magnification by the optics block 118 , blur spots associated with sub-pixels mask the dark space 340 between adjacent sub-pixels.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Optics & Photonics (AREA)
Abstract
Description
Claims (19)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/376,844 US10558036B2 (en) | 2015-12-15 | 2016-12-13 | Using oscillation of optical components to reduce fixed pattern noise in a virtual reality headset |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201562267748P | 2015-12-15 | 2015-12-15 | |
| US15/376,844 US10558036B2 (en) | 2015-12-15 | 2016-12-13 | Using oscillation of optical components to reduce fixed pattern noise in a virtual reality headset |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20170192228A1 US20170192228A1 (en) | 2017-07-06 |
| US10558036B2 true US10558036B2 (en) | 2020-02-11 |
Family
ID=59235485
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US15/376,844 Expired - Fee Related US10558036B2 (en) | 2015-12-15 | 2016-12-13 | Using oscillation of optical components to reduce fixed pattern noise in a virtual reality headset |
Country Status (1)
| Country | Link |
|---|---|
| US (1) | US10558036B2 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11579692B2 (en) | 2013-07-05 | 2023-02-14 | Haptx, Inc. | Whole-body human-computer interface |
| US11816268B2 (en) | 2020-10-22 | 2023-11-14 | Haptx, Inc. | Actuator and retraction mechanism for force feedback exoskeleton |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI811416B (en) * | 2019-07-31 | 2023-08-11 | 友達光電股份有限公司 | Near-eye display device |
| CN114114692A (en) * | 2021-11-29 | 2022-03-01 | 维沃移动通信有限公司 | Wearable device |
| TWI842227B (en) * | 2022-06-28 | 2024-05-11 | 宏達國際電子股份有限公司 | Head-mounted display device and display method therefor |
Citations (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5777588A (en) * | 1994-12-16 | 1998-07-07 | Sharp Kabushiki Kaisha | Autostereoscopic display having a high resolution 2D mode |
| US6078038A (en) * | 1994-02-15 | 2000-06-20 | Cooper; J. Carl | Apparatus and method for displaying a high resolution image with relatively lower resolution display device |
| US6243055B1 (en) * | 1994-10-25 | 2001-06-05 | James L. Fergason | Optical display system and method with optical shifting of pixel position including conversion of pixel layout to form delta to stripe pattern by time base multiplexing |
| US6425669B1 (en) * | 2000-05-24 | 2002-07-30 | Ball Semiconductor, Inc. | Maskless exposure system |
| US20030218618A1 (en) * | 1997-09-13 | 2003-11-27 | Phan Gia Chuong | Dynamic pixel resolution, brightness and contrast for displays using spatial elements |
| US20040150651A1 (en) * | 1997-09-13 | 2004-08-05 | Phan Gia Chuong | Dynamic pixel resolution, brightness and contrast for displays using spatial elements |
| US6963454B1 (en) * | 2002-03-01 | 2005-11-08 | Research Foundation Of The University Of Central Florida | Head-mounted display by integration of phase-conjugate material |
| US20060176362A1 (en) * | 2005-02-04 | 2006-08-10 | Penn Steven M | Optical system and method for increasing image resolution and/or dithering in printing applications |
| US20060176323A1 (en) * | 2005-02-04 | 2006-08-10 | Bommersbach William M | Optical system and method for increasing image resolution and/or dithering in projection applications |
| US7097311B2 (en) * | 2003-04-19 | 2006-08-29 | University Of Kentucky Research Foundation | Super-resolution overlay in multi-projector displays |
| US20060250696A1 (en) * | 2005-05-03 | 2006-11-09 | Mcguire James P | Head mounted display devices |
| US20080180648A1 (en) * | 2006-09-16 | 2008-07-31 | Wenhui Mei | Divided sub-image array scanning and exposing system |
| US20100026173A1 (en) * | 2008-08-04 | 2010-02-04 | Samsung Mobile Display Co., Ltd. | Method of depositing light emitting layer of organic el device, method of manufacturing organic el device, and organic el device manufactured by the method |
| US20110141297A1 (en) * | 2008-07-31 | 2011-06-16 | Fujifilm Corporation | Compound-eye imaging device |
| US20110211097A1 (en) * | 2008-11-12 | 2011-09-01 | Sharp Kabushiki Kaisha | Imaging device |
| US20120105582A1 (en) * | 2010-10-29 | 2012-05-03 | Sony Corporation | Super-resolution from 3d (3d to 2d conversion) for high quality 2d playback |
| US20130207951A1 (en) * | 2010-04-28 | 2013-08-15 | Universitat Des Saarlandes | Apparent display resolution enhancement for moving images |
| US20140178861A1 (en) * | 2006-03-10 | 2014-06-26 | Reuven Duer | Waveguide-based detection system with scanning light source |
| US20150077618A1 (en) * | 2013-09-18 | 2015-03-19 | Kabushiki Kaisha Toshiba | Imaging lens and solid state imaging device |
| US20150205014A1 (en) * | 2014-01-21 | 2015-07-23 | Seiko Epson Corporation | Lens array, method for manufacturing lens array, electro-optical device, and electronic apparatus |
| US20150277121A1 (en) * | 2014-03-29 | 2015-10-01 | Ron Fridental | Method and apparatus for displaying video data |
| US20150338660A1 (en) * | 2013-01-10 | 2015-11-26 | Sony Corporation | Image display apparatus, image generating device, and transmissive spatial light modulating device |
| US20160116979A1 (en) * | 2014-01-21 | 2016-04-28 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
| US20170221266A1 (en) * | 2016-02-03 | 2017-08-03 | Google Inc. | Super-resolution virtual reality head-mounted displays and methods of operating the same |
-
2016
- 2016-12-13 US US15/376,844 patent/US10558036B2/en not_active Expired - Fee Related
Patent Citations (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6078038A (en) * | 1994-02-15 | 2000-06-20 | Cooper; J. Carl | Apparatus and method for displaying a high resolution image with relatively lower resolution display device |
| US6243055B1 (en) * | 1994-10-25 | 2001-06-05 | James L. Fergason | Optical display system and method with optical shifting of pixel position including conversion of pixel layout to form delta to stripe pattern by time base multiplexing |
| US5777588A (en) * | 1994-12-16 | 1998-07-07 | Sharp Kabushiki Kaisha | Autostereoscopic display having a high resolution 2D mode |
| US20030218618A1 (en) * | 1997-09-13 | 2003-11-27 | Phan Gia Chuong | Dynamic pixel resolution, brightness and contrast for displays using spatial elements |
| US20040150651A1 (en) * | 1997-09-13 | 2004-08-05 | Phan Gia Chuong | Dynamic pixel resolution, brightness and contrast for displays using spatial elements |
| US6425669B1 (en) * | 2000-05-24 | 2002-07-30 | Ball Semiconductor, Inc. | Maskless exposure system |
| US6963454B1 (en) * | 2002-03-01 | 2005-11-08 | Research Foundation Of The University Of Central Florida | Head-mounted display by integration of phase-conjugate material |
| US7097311B2 (en) * | 2003-04-19 | 2006-08-29 | University Of Kentucky Research Foundation | Super-resolution overlay in multi-projector displays |
| US20060176362A1 (en) * | 2005-02-04 | 2006-08-10 | Penn Steven M | Optical system and method for increasing image resolution and/or dithering in printing applications |
| US20060176323A1 (en) * | 2005-02-04 | 2006-08-10 | Bommersbach William M | Optical system and method for increasing image resolution and/or dithering in projection applications |
| US20060250696A1 (en) * | 2005-05-03 | 2006-11-09 | Mcguire James P | Head mounted display devices |
| US20140178861A1 (en) * | 2006-03-10 | 2014-06-26 | Reuven Duer | Waveguide-based detection system with scanning light source |
| US20080180648A1 (en) * | 2006-09-16 | 2008-07-31 | Wenhui Mei | Divided sub-image array scanning and exposing system |
| US20110141297A1 (en) * | 2008-07-31 | 2011-06-16 | Fujifilm Corporation | Compound-eye imaging device |
| US20100026173A1 (en) * | 2008-08-04 | 2010-02-04 | Samsung Mobile Display Co., Ltd. | Method of depositing light emitting layer of organic el device, method of manufacturing organic el device, and organic el device manufactured by the method |
| US20110211097A1 (en) * | 2008-11-12 | 2011-09-01 | Sharp Kabushiki Kaisha | Imaging device |
| US20130207951A1 (en) * | 2010-04-28 | 2013-08-15 | Universitat Des Saarlandes | Apparent display resolution enhancement for moving images |
| US20120105582A1 (en) * | 2010-10-29 | 2012-05-03 | Sony Corporation | Super-resolution from 3d (3d to 2d conversion) for high quality 2d playback |
| US20150338660A1 (en) * | 2013-01-10 | 2015-11-26 | Sony Corporation | Image display apparatus, image generating device, and transmissive spatial light modulating device |
| US20150077618A1 (en) * | 2013-09-18 | 2015-03-19 | Kabushiki Kaisha Toshiba | Imaging lens and solid state imaging device |
| US20150205014A1 (en) * | 2014-01-21 | 2015-07-23 | Seiko Epson Corporation | Lens array, method for manufacturing lens array, electro-optical device, and electronic apparatus |
| US20160116979A1 (en) * | 2014-01-21 | 2016-04-28 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
| US20150277121A1 (en) * | 2014-03-29 | 2015-10-01 | Ron Fridental | Method and apparatus for displaying video data |
| US20170221266A1 (en) * | 2016-02-03 | 2017-08-03 | Google Inc. | Super-resolution virtual reality head-mounted displays and methods of operating the same |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11579692B2 (en) | 2013-07-05 | 2023-02-14 | Haptx, Inc. | Whole-body human-computer interface |
| US11816261B2 (en) | 2013-07-05 | 2023-11-14 | Haptx, Inc. | Whole-body human-computer interface |
| US11816268B2 (en) | 2020-10-22 | 2023-11-14 | Haptx, Inc. | Actuator and retraction mechanism for force feedback exoskeleton |
Also Published As
| Publication number | Publication date |
|---|---|
| US20170192228A1 (en) | 2017-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11598966B2 (en) | Light projection system including an optical assembly for correction of differential distortion | |
| US11163165B1 (en) | Directed display architecture | |
| US9599822B2 (en) | Corrective optics for reducing fixed pattern noise in a virtual reality headset | |
| US10198032B2 (en) | Passive locators for a virtual reality headset | |
| US10345517B2 (en) | Curved electronic display element | |
| US10152121B2 (en) | Eye tracking through illumination by head-mounted displays | |
| US9472025B2 (en) | Compressible eyecup assemblies in a virtual reality headset | |
| US9640120B2 (en) | Diffractive element for reducing fixed pattern noise in a virtual reality headset | |
| US10274728B2 (en) | Stacked display panels for image enhancement | |
| US9910282B2 (en) | Increasing field of view of head-mounted display using a mirror | |
| US10558036B2 (en) | Using oscillation of optical components to reduce fixed pattern noise in a virtual reality headset | |
| US10437326B2 (en) | Backlight modulation for liquid crystal display with eyetracking for virtual reality | |
| US10613323B1 (en) | Transition feature for framing multizone optics | |
| US20160341953A1 (en) | Laminated alignment structure for eyecup assemblies in a virtual reality headset | |
| CN115280218B (en) | High-resolution LCD display | |
| US10585479B2 (en) | Control for a virtual reality system including opposing portions for interacting with virtual objects and providing tactile feedback to a user | |
| US10522094B1 (en) | Global illumination mode liquid crystal display for virtual reality | |
| US20170097680A1 (en) | Using anisotropic weaves of materials in input interfaces for a virtual reality system | |
| US10714023B2 (en) | Display including liquid crystal layer with organic light emitting diode backlight | |
| US10209775B2 (en) | Using a magnetic actuation mechanism to provide tactile feedback to a user interacting with a virtual environment | |
| US10698218B1 (en) | Display system with oscillating element | |
| US20160217613A1 (en) | Extendable eyecups for a virtual reality headset | |
| US10679416B1 (en) | Crosstalk mitigation for virtual reality | |
| US10690923B1 (en) | Scanning system for a tiling display | |
| US10436975B2 (en) | Segmented backlight for liquid crystal display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: OCULUS VR, LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KELLER, SEAN JASON;TRUTNA, TRISTAN THOMAS;PEREK, DAVID R.;AND OTHERS;SIGNING DATES FROM 20161217 TO 20170117;REEL/FRAME:041012/0224 |
|
| AS | Assignment |
Owner name: FACEBOOK TECHNOLOGIES, LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:OCULUS VR, LLC;REEL/FRAME:047178/0616 Effective date: 20180903 |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| AS | Assignment |
Owner name: META PLATFORMS TECHNOLOGIES, LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:FACEBOOK TECHNOLOGIES, LLC;REEL/FRAME:060315/0224 Effective date: 20220318 |
|
| FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
| FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20240211 |