EP3522150A3 - Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays - Google Patents

Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays Download PDF

Info

Publication number
EP3522150A3
EP3522150A3 EP18187455.3A EP18187455A EP3522150A3 EP 3522150 A3 EP3522150 A3 EP 3522150A3 EP 18187455 A EP18187455 A EP 18187455A EP 3522150 A3 EP3522150 A3 EP 3522150A3
Authority
EP
European Patent Office
Prior art keywords
head
frame
special
purpose hardware
hardware device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP18187455.3A
Other languages
German (de)
French (fr)
Other versions
EP3522150A2 (en
Inventor
Vlad Fruchter
Ohad Meitav
David Vakrat
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Meta Platforms Technologies LLC
Original Assignee
Facebook Technologies LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/006,434 external-priority patent/US10559276B2/en
Application filed by Facebook Technologies LLC filed Critical Facebook Technologies LLC
Publication of EP3522150A2 publication Critical patent/EP3522150A2/en
Publication of EP3522150A3 publication Critical patent/EP3522150A3/en
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/395—Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen
    • G09G5/397—Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/12—Synchronisation between the display unit and other units, e.g. other display units, video-disc players
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/02—Improving the quality of display appearance
    • G09G2320/0242—Compensation of deficiencies in the appearance of colours
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/02—Improving the quality of display appearance
    • G09G2320/0252—Improving the response speed
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

A special-purpose hardware device for mitigating motion-to-photon latency in head-mounted displays may include an image signal processor that receives at least one image frame (402) captured by a camera device of a head-mounted-display system. The special-purpose hardware device may also include an input-formatting component that receives the computer-generated imagery. (404) The special-purpose hardware device may further include a blending component (106) that generates at least one mixed-reality frame by overlaying the computer-generated imagery onto the image frame. The special-purpose hardware device may additionally include a frame-output interface that feeds the mixed-reality frame generated by the blending component to a display device (318) of the head-mounted-display system to facilitate displaying the mixed-reality frame for presentation to a user wearing the head-mounted-display system.
EP18187455.3A 2018-02-03 2018-08-06 Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays Withdrawn EP3522150A3 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862626004P 2018-02-03 2018-02-03
US16/006,434 US10559276B2 (en) 2018-02-03 2018-06-12 Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays
PCT/US2018/038224 WO2019152067A1 (en) 2018-02-03 2018-06-19 Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays

Publications (2)

Publication Number Publication Date
EP3522150A2 EP3522150A2 (en) 2019-08-07
EP3522150A3 true EP3522150A3 (en) 2020-01-01

Family

ID=63371464

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18187455.3A Withdrawn EP3522150A3 (en) 2018-02-03 2018-08-06 Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays

Country Status (1)

Country Link
EP (1) EP3522150A3 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10559276B2 (en) 2018-02-03 2020-02-11 Facebook Technologies, Llc Apparatus, system, and method for mitigating motion-to-photon latency in head-mounted displays
US11455705B2 (en) 2018-09-27 2022-09-27 Qualcomm Incorporated Asynchronous space warp for remotely rendered VR

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6445815B1 (en) * 1998-05-08 2002-09-03 Canon Kabushiki Kaisha Measurement of depth image considering time delay
FR2840753A1 (en) * 2002-06-06 2003-12-12 Artabel Digital image computer assembly processing having video signals decoded producing pixel/clock synchronisation signal and pixel signals generated applying OR logic two pixel signals
US20070188522A1 (en) * 2006-02-15 2007-08-16 Canon Kabushiki Kaisha Mixed reality display system
US20080297437A1 (en) * 2007-05-31 2008-12-04 Canon Kabushiki Kaisha Head mounted display and control method therefor
US20100207957A1 (en) * 2009-02-18 2010-08-19 Stmicroelectronics Pvt. Ltd. Overlaying videos on a display device
US20150029218A1 (en) * 2013-07-25 2015-01-29 Oliver Michael Christian Williams Late stage reprojection
US20160037085A1 (en) * 2014-07-31 2016-02-04 Apple Inc. Auto filter extent management
US20160314564A1 (en) * 2015-04-22 2016-10-27 Esight Corp. Methods and devices for optical aberration correction
US20160328882A1 (en) * 2015-05-04 2016-11-10 Google Inc. Pass-through display of captured imagery
US20160364904A1 (en) * 2015-06-12 2016-12-15 Google Inc. Electronic display stabilization for head mounted display
US20170243324A1 (en) * 2016-02-22 2017-08-24 Google Inc. Separate time-warping for a scene and an object for display of virtual reality content

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6445815B1 (en) * 1998-05-08 2002-09-03 Canon Kabushiki Kaisha Measurement of depth image considering time delay
FR2840753A1 (en) * 2002-06-06 2003-12-12 Artabel Digital image computer assembly processing having video signals decoded producing pixel/clock synchronisation signal and pixel signals generated applying OR logic two pixel signals
US20070188522A1 (en) * 2006-02-15 2007-08-16 Canon Kabushiki Kaisha Mixed reality display system
US20080297437A1 (en) * 2007-05-31 2008-12-04 Canon Kabushiki Kaisha Head mounted display and control method therefor
US20100207957A1 (en) * 2009-02-18 2010-08-19 Stmicroelectronics Pvt. Ltd. Overlaying videos on a display device
US20150029218A1 (en) * 2013-07-25 2015-01-29 Oliver Michael Christian Williams Late stage reprojection
US20160037085A1 (en) * 2014-07-31 2016-02-04 Apple Inc. Auto filter extent management
US20160314564A1 (en) * 2015-04-22 2016-10-27 Esight Corp. Methods and devices for optical aberration correction
US20160328882A1 (en) * 2015-05-04 2016-11-10 Google Inc. Pass-through display of captured imagery
US20160364904A1 (en) * 2015-06-12 2016-12-15 Google Inc. Electronic display stabilization for head mounted display
US20170243324A1 (en) * 2016-02-22 2017-08-24 Google Inc. Separate time-warping for a scene and an object for display of virtual reality content

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
J. M. P. VAN WAVEREN: "The asynchronous time warp for virtual reality on consumer hardware", PROCEEDINGS OF THE 22ND ACM CONFERENCE ON VIRTUAL REALITY SOFTWARE AND TECHNOLOGY , VRST '16, 1 January 2016 (2016-01-01), New York, New York, USA, pages 37 - 46, XP055344703, ISBN: 978-1-4503-4491-3, DOI: 10.1145/2993369.2993375 *
ORLOSKY JASON ET AL: "VisMerge: Light Adaptive Vision Augmentation via Spectral and Temporal Fusion of Non-visible Light", 2017 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR), IEEE, 9 October 2017 (2017-10-09), pages 22 - 31, XP033262490, DOI: 10.1109/ISMAR.2017.19 *

Also Published As

Publication number Publication date
EP3522150A2 (en) 2019-08-07

Similar Documents

Publication Publication Date Title
US9948863B2 (en) Self-timer preview image presentation method and apparatus, and terminal
EP3082019A3 (en) Display apparatus and method for controlling the same
EP3023969A3 (en) Display and method and electronic device
EP2701390A3 (en) Apparatus for adjusting displayed picture, display apparatus and display method
EP2645703A3 (en) Image processing apparatus and method of camera device
AU2017260192A1 (en) Surgical stereoscopic visualization system with movable head mounted display
EP2393300A3 (en) Image display apparatus and method for operating the same
EP2618557A3 (en) Digital image processing apparatus and method of controlling the same
EP3514766A3 (en) Electronic device, method for displaying an augmented reality scene and non-transitory computer-readable medium
EP3764316A3 (en) Improvements in and relating to the display of images
EP3826309A3 (en) Method and apparatus for processing video
EP2434458A3 (en) Image processing program, apparatus, system, and method for mixed reality
EP2728853A3 (en) Method and device for controlling a camera
GB0210288D0 (en) Ocular display apparatus for assessment and measurement of and for treatment of ocular disorders, and methods therefor
EP3361724A3 (en) System and method for generating a virtual viewpoint
EP3378702A3 (en) Viewing device for vehicle
EP2793170A3 (en) Objects in screen images
EP2629503A3 (en) Apparatus and method for transmitting a frame image of a camera
EP1562368A3 (en) Digital camera
EP2574040A3 (en) Image photography apparatus
EP2149868A3 (en) Apparatus for processing image signal, program and apparatus for displaying image signal
EP3572916A3 (en) Apparatus, system, and method for accelerating positional tracking of head-mounted displays
EP2654288A3 (en) Image processing apparatus and method of camera
EP2423881A3 (en) Image processing apparatus and method
EP2391136A3 (en) Picture reproducing method and picture reproducing apparatus

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIC1 Information provided on ipc code assigned before grant

Ipc: G06T 19/00 20110101ALI20191125BHEP

Ipc: G09G 5/12 20060101ALI20191125BHEP

Ipc: G09G 5/397 20060101AFI20191125BHEP

Ipc: G02B 27/01 20060101ALN20191125BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200311

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210215

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20221125