CN113170136B - 重投影帧的运动平滑 - Google Patents

重投影帧的运动平滑 Download PDF

Info

Publication number
CN113170136B
CN113170136B CN201980067305.5A CN201980067305A CN113170136B CN 113170136 B CN113170136 B CN 113170136B CN 201980067305 A CN201980067305 A CN 201980067305A CN 113170136 B CN113170136 B CN 113170136B
Authority
CN
China
Prior art keywords
pixel data
frame
motion vectors
array
gpu
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201980067305.5A
Other languages
English (en)
Chinese (zh)
Other versions
CN113170136A (zh
Inventor
A·弗拉霍斯
A·雷拜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Valve Corp
Original Assignee
Valve Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Valve Corp filed Critical Valve Corp
Publication of CN113170136A publication Critical patent/CN113170136A/zh
Application granted granted Critical
Publication of CN113170136B publication Critical patent/CN113170136B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/02—Improving the quality of display appearance
    • G09G2320/0261—Improving the quality of display appearance in the context of movement of objects on the screen or movement of the observer relative to the screen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/10—Special adaptations of display systems for operation with variable images
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/10—Special adaptations of display systems for operation with variable images
    • G09G2320/106—Determination of movement vectors or equivalent parameters within the image
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0457—Improvement of perceived resolution by subpixel rendering
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/14—Solving problems related to the presentation of information to be displayed
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/16—Determination of a pixel data signal depending on the signal applied in the previous frame
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Human Computer Interaction (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Transforming Electric Information Into Light Information (AREA)
CN201980067305.5A 2018-10-09 2019-09-27 重投影帧的运动平滑 Active CN113170136B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/155,686 US10733783B2 (en) 2018-10-09 2018-10-09 Motion smoothing for re-projected frames
US16/155,686 2018-10-09
PCT/US2019/053528 WO2020076526A1 (en) 2018-10-09 2019-09-27 Motion smoothing for re-projected frames

Publications (2)

Publication Number Publication Date
CN113170136A CN113170136A (zh) 2021-07-23
CN113170136B true CN113170136B (zh) 2024-04-23

Family

ID=70052297

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201980067305.5A Active CN113170136B (zh) 2018-10-09 2019-09-27 重投影帧的运动平滑

Country Status (6)

Country Link
US (1) US10733783B2 (cg-RX-API-DMAC7.html)
EP (1) EP3857885A4 (cg-RX-API-DMAC7.html)
JP (1) JP7395577B2 (cg-RX-API-DMAC7.html)
KR (1) KR102641272B1 (cg-RX-API-DMAC7.html)
CN (1) CN113170136B (cg-RX-API-DMAC7.html)
WO (1) WO2020076526A1 (cg-RX-API-DMAC7.html)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10962780B2 (en) * 2015-10-26 2021-03-30 Microsoft Technology Licensing, Llc Remote rendering for virtual images
US10152822B2 (en) * 2017-04-01 2018-12-11 Intel Corporation Motion biased foveated renderer
US10319064B2 (en) 2017-04-10 2019-06-11 Intel Corporation Graphics anti-aliasing resolve with stencil mask
CN115842907A (zh) * 2018-03-27 2023-03-24 京东方科技集团股份有限公司 渲染方法、计算机产品及显示装置
US10861215B2 (en) * 2018-04-30 2020-12-08 Qualcomm Incorporated Asynchronous time and space warp with determination of region of interest
US10520739B1 (en) * 2018-07-11 2019-12-31 Valve Corporation Dynamic panel masking
US11381825B2 (en) * 2018-11-27 2022-07-05 Advanced Micro Devices, Inc. Variable rate rendering based on motion estimation
US11030822B2 (en) * 2019-05-15 2021-06-08 Microsoft Technology Licensing, Llc Content indicators in a 3D environment authoring application
US11287947B2 (en) 2019-05-15 2022-03-29 Microsoft Technology Licensing, Llc Contextual input in a three-dimensional environment
US11164395B2 (en) 2019-05-15 2021-11-02 Microsoft Technology Licensing, Llc Structure switching in a three-dimensional environment
US11039061B2 (en) 2019-05-15 2021-06-15 Microsoft Technology Licensing, Llc Content assistance in a three-dimensional environment
US11948242B2 (en) 2019-08-02 2024-04-02 Fmr Llc Intelligent smoothing of 3D alternative reality applications for secondary 2D viewing
US11138804B2 (en) 2019-08-02 2021-10-05 Fmr Llc Intelligent smoothing of 3D alternative reality applications for secondary 2D viewing
US12148120B2 (en) * 2019-12-18 2024-11-19 Ati Technologies Ulc Frame reprojection for virtual reality and augmented reality
US11363247B2 (en) 2020-02-14 2022-06-14 Valve Corporation Motion smoothing in a distributed system
US11651547B2 (en) * 2020-12-10 2023-05-16 Nvidia Corporation Adaptive temporal image filtering for rendering realistic illumination
US12069230B2 (en) * 2020-12-23 2024-08-20 Meta Platforms Technologies, Llc Temporal foveated rendering
CN113066001B (zh) * 2021-02-26 2024-10-22 华为技术有限公司 一种图像处理方法及相关设备
US12200398B2 (en) 2021-03-16 2025-01-14 Samsung Electronics Co., Ltd. Pixel blending for synthesizing video frames with occlusion and watermark handling
US12075018B2 (en) * 2021-04-21 2024-08-27 Microsoft Technology Licensing, Llc Distributed virtual reality
US11587208B2 (en) * 2021-05-26 2023-02-21 Qualcomm Incorporated High quality UI elements with frame extrapolation
US12236608B2 (en) * 2021-09-22 2025-02-25 Samsung Electronics Co., Ltd. Judder detection for dynamic frame rate conversion
EP4224289A1 (en) * 2022-02-08 2023-08-09 Nokia Technologies Oy Mediated reality
CN118647966A (zh) * 2022-02-14 2024-09-13 高通股份有限公司 显示掩模层生成和运行时调整
CN118057460A (zh) * 2022-11-21 2024-05-21 荣耀终端有限公司 图像处理方法及装置
CN118506702A (zh) * 2023-02-15 2024-08-16 台达电子工业股份有限公司 移动物件检测装置及方法
WO2025192345A1 (ja) * 2024-03-12 2025-09-18 株式会社ソニー・インタラクティブエンタテインメント 画像生成装置、画像生成方法および画像生成プログラム
IL311824A (en) * 2024-03-28 2025-10-01 Semiconductor Devices Ltd Correction of non-uniformity for infra-red images
KR102888530B1 (ko) * 2025-03-20 2025-11-21 서울대학교산학협력단 디스플레이 소자 어레이 제조방법 및 디스플레이 패널

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015081888A1 (en) * 2013-12-06 2015-06-11 Mediatek Inc. Method and apparatus for motion boundary processing
CN107003512A (zh) * 2014-12-30 2017-08-01 索尼互动娱乐股份有限公司 用于虚拟现实的头戴式显示器中的扫描显示系统
CN108140359A (zh) * 2015-08-19 2018-06-08 威尔乌集团 用于检测和/或校正显示器中的像素亮度和/或色度响应变化的系统和方法
CN108139807A (zh) * 2015-11-20 2018-06-08 谷歌有限责任公司 使用像素速度的电子显示稳定

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07222155A (ja) * 1994-01-31 1995-08-18 Kyocera Corp 画像符号化装置
US6466185B2 (en) 1998-04-20 2002-10-15 Alan Sullivan Multi-planar volumetric display system and method of operation using psychological vision cues
JP4408779B2 (ja) * 2004-09-15 2010-02-03 キヤノン株式会社 画像処理装置
US7404645B2 (en) * 2005-06-20 2008-07-29 Digital Display Innovations, Llc Image and light source modulation for a digital display system
EP2130381A2 (en) 2007-01-23 2009-12-09 Euclid Discoveries, LLC Computer method and apparatus for processing image data
JP2008252591A (ja) * 2007-03-30 2008-10-16 Toshiba Corp 補間フレーム生成装置、補間フレーム生成方法及び放送受信装置
US8208551B2 (en) * 2007-10-31 2012-06-26 Broadcom Corporation Method and system for hierarchically layered adaptive median motion vector smoothing
US20120021828A1 (en) 2010-02-24 2012-01-26 Valve Corporation Graphical user interface for modification of animation data using preset animation samples
JP5677002B2 (ja) * 2010-09-28 2015-02-25 キヤノン株式会社 映像制御装置、及び映像制御方法
US8624891B2 (en) 2011-01-17 2014-01-07 Disney Enterprises, Inc. Iterative reprojection of images
US9641853B2 (en) * 2013-04-15 2017-05-02 Futurewei Technologies, Inc. Method and apparatus of depth prediction mode selection
US9277129B2 (en) 2013-06-07 2016-03-01 Apple Inc. Robust image feature based video stabilization and smoothing
WO2016137149A1 (ko) * 2015-02-24 2016-09-01 엘지전자(주) 폴리곤 유닛 기반 영상 처리 방법 및 이를 위한 장치
US10037620B2 (en) * 2015-05-29 2018-07-31 Nvidia Corporation Piecewise linear irregular rasterization
GB2539042B (en) * 2015-06-05 2019-08-21 Imagination Tech Ltd Tessellation method using displacement factors
US9978180B2 (en) 2016-01-25 2018-05-22 Microsoft Technology Licensing, Llc Frame projection for augmented reality environments
US10129523B2 (en) * 2016-06-22 2018-11-13 Microsoft Technology Licensing, Llc Depth-aware reprojection
US10297086B2 (en) * 2016-07-04 2019-05-21 Liquid Cinema Inc. System and method for processing digital video
US20190045213A1 (en) * 2017-08-03 2019-02-07 Intel Corporation Reference frame reprojection for improved video coding
US10861215B2 (en) * 2018-04-30 2020-12-08 Qualcomm Incorporated Asynchronous time and space warp with determination of region of interest

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015081888A1 (en) * 2013-12-06 2015-06-11 Mediatek Inc. Method and apparatus for motion boundary processing
CN107003512A (zh) * 2014-12-30 2017-08-01 索尼互动娱乐股份有限公司 用于虚拟现实的头戴式显示器中的扫描显示系统
CN108140359A (zh) * 2015-08-19 2018-06-08 威尔乌集团 用于检测和/或校正显示器中的像素亮度和/或色度响应变化的系统和方法
CN108139807A (zh) * 2015-11-20 2018-06-08 谷歌有限责任公司 使用像素速度的电子显示稳定

Also Published As

Publication number Publication date
JP2022504475A (ja) 2022-01-13
KR20210071015A (ko) 2021-06-15
EP3857885A4 (en) 2022-06-22
US20200111195A1 (en) 2020-04-09
EP3857885A1 (en) 2021-08-04
CN113170136A (zh) 2021-07-23
WO2020076526A1 (en) 2020-04-16
JP7395577B2 (ja) 2023-12-11
KR102641272B1 (ko) 2024-02-27
US10733783B2 (en) 2020-08-04

Similar Documents

Publication Publication Date Title
CN113170136B (zh) 重投影帧的运动平滑
JP7612700B2 (ja) 分散システムにおける動き平滑化
CN113272715B (zh) 基于眼睛跟踪的动态渲染时间确定
KR102714790B1 (ko) 애플리케이션 렌더링 성능에 근거한 예측 및 스로틀링 조정
KR102794876B1 (ko) 동적 패널 마스킹
US11343486B2 (en) Counterrotation of display panels and/or virtual cameras in a HMD

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant