KR102641272B1 - 재투영된 프레임을 위한 모션 스무딩 - Google Patents

재투영된 프레임을 위한 모션 스무딩 Download PDF

Info

Publication number
KR102641272B1
KR102641272B1 KR1020217012646A KR20217012646A KR102641272B1 KR 102641272 B1 KR102641272 B1 KR 102641272B1 KR 1020217012646 A KR1020217012646 A KR 1020217012646A KR 20217012646 A KR20217012646 A KR 20217012646A KR 102641272 B1 KR102641272 B1 KR 102641272B1
Authority
KR
South Korea
Prior art keywords
pixel data
frame
motion vectors
array
gpu
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020217012646A
Other languages
English (en)
Korean (ko)
Other versions
KR20210071015A (ko
Inventor
알렉스 블라초스
아론 레이바이
Original Assignee
밸브 코포레이션
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 밸브 코포레이션 filed Critical 밸브 코포레이션
Publication of KR20210071015A publication Critical patent/KR20210071015A/ko
Application granted granted Critical
Publication of KR102641272B1 publication Critical patent/KR102641272B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H04N5/145—Movement estimation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/02—Improving the quality of display appearance
    • G09G2320/0261—Improving the quality of display appearance in the context of movement of objects on the screen or movement of the observer relative to the screen
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/10—Special adaptations of display systems for operation with variable images
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/10—Special adaptations of display systems for operation with variable images
    • G09G2320/106—Determination of movement vectors or equivalent parameters within the image
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0407—Resolution change, inclusive of the use of different resolutions for different screen areas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0457—Improvement of perceived resolution by subpixel rendering
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/14—Solving problems related to the presentation of information to be displayed
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/16—Determination of a pixel data signal depending on the signal applied in the previous frame
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Human Computer Interaction (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Transforming Electric Information Into Light Information (AREA)
KR1020217012646A 2018-10-09 2019-09-27 재투영된 프레임을 위한 모션 스무딩 Active KR102641272B1 (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/155,686 US10733783B2 (en) 2018-10-09 2018-10-09 Motion smoothing for re-projected frames
US16/155,686 2018-10-09
PCT/US2019/053528 WO2020076526A1 (en) 2018-10-09 2019-09-27 Motion smoothing for re-projected frames

Publications (2)

Publication Number Publication Date
KR20210071015A KR20210071015A (ko) 2021-06-15
KR102641272B1 true KR102641272B1 (ko) 2024-02-27

Family

ID=70052297

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020217012646A Active KR102641272B1 (ko) 2018-10-09 2019-09-27 재투영된 프레임을 위한 모션 스무딩

Country Status (6)

Country Link
US (1) US10733783B2 (cg-RX-API-DMAC7.html)
EP (1) EP3857885A4 (cg-RX-API-DMAC7.html)
JP (1) JP7395577B2 (cg-RX-API-DMAC7.html)
KR (1) KR102641272B1 (cg-RX-API-DMAC7.html)
CN (1) CN113170136B (cg-RX-API-DMAC7.html)
WO (1) WO2020076526A1 (cg-RX-API-DMAC7.html)

Families Citing this family (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10962780B2 (en) * 2015-10-26 2021-03-30 Microsoft Technology Licensing, Llc Remote rendering for virtual images
US10152822B2 (en) * 2017-04-01 2018-12-11 Intel Corporation Motion biased foveated renderer
US10319064B2 (en) 2017-04-10 2019-06-11 Intel Corporation Graphics anti-aliasing resolve with stencil mask
CN115842907A (zh) * 2018-03-27 2023-03-24 京东方科技集团股份有限公司 渲染方法、计算机产品及显示装置
US10861215B2 (en) * 2018-04-30 2020-12-08 Qualcomm Incorporated Asynchronous time and space warp with determination of region of interest
US10520739B1 (en) * 2018-07-11 2019-12-31 Valve Corporation Dynamic panel masking
US11381825B2 (en) * 2018-11-27 2022-07-05 Advanced Micro Devices, Inc. Variable rate rendering based on motion estimation
US11030822B2 (en) * 2019-05-15 2021-06-08 Microsoft Technology Licensing, Llc Content indicators in a 3D environment authoring application
US11287947B2 (en) 2019-05-15 2022-03-29 Microsoft Technology Licensing, Llc Contextual input in a three-dimensional environment
US11164395B2 (en) 2019-05-15 2021-11-02 Microsoft Technology Licensing, Llc Structure switching in a three-dimensional environment
US11039061B2 (en) 2019-05-15 2021-06-15 Microsoft Technology Licensing, Llc Content assistance in a three-dimensional environment
US11948242B2 (en) 2019-08-02 2024-04-02 Fmr Llc Intelligent smoothing of 3D alternative reality applications for secondary 2D viewing
US11138804B2 (en) 2019-08-02 2021-10-05 Fmr Llc Intelligent smoothing of 3D alternative reality applications for secondary 2D viewing
US12148120B2 (en) * 2019-12-18 2024-11-19 Ati Technologies Ulc Frame reprojection for virtual reality and augmented reality
US11363247B2 (en) 2020-02-14 2022-06-14 Valve Corporation Motion smoothing in a distributed system
US11651547B2 (en) * 2020-12-10 2023-05-16 Nvidia Corporation Adaptive temporal image filtering for rendering realistic illumination
US12069230B2 (en) * 2020-12-23 2024-08-20 Meta Platforms Technologies, Llc Temporal foveated rendering
CN113066001B (zh) * 2021-02-26 2024-10-22 华为技术有限公司 一种图像处理方法及相关设备
US12200398B2 (en) 2021-03-16 2025-01-14 Samsung Electronics Co., Ltd. Pixel blending for synthesizing video frames with occlusion and watermark handling
US12075018B2 (en) * 2021-04-21 2024-08-27 Microsoft Technology Licensing, Llc Distributed virtual reality
US11587208B2 (en) * 2021-05-26 2023-02-21 Qualcomm Incorporated High quality UI elements with frame extrapolation
US12236608B2 (en) * 2021-09-22 2025-02-25 Samsung Electronics Co., Ltd. Judder detection for dynamic frame rate conversion
EP4224289A1 (en) * 2022-02-08 2023-08-09 Nokia Technologies Oy Mediated reality
CN118647966A (zh) * 2022-02-14 2024-09-13 高通股份有限公司 显示掩模层生成和运行时调整
CN118057460A (zh) * 2022-11-21 2024-05-21 荣耀终端有限公司 图像处理方法及装置
CN118506702A (zh) * 2023-02-15 2024-08-16 台达电子工业股份有限公司 移动物件检测装置及方法
WO2025192345A1 (ja) * 2024-03-12 2025-09-18 株式会社ソニー・インタラクティブエンタテインメント 画像生成装置、画像生成方法および画像生成プログラム
IL311824A (en) * 2024-03-28 2025-10-01 Semiconductor Devices Ltd Correction of non-uniformity for infra-red images
KR102888530B1 (ko) * 2025-03-20 2025-11-21 서울대학교산학협력단 디스플레이 소자 어레이 제조방법 및 디스플레이 패널

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017004516A (ja) * 2015-06-05 2017-01-05 イマジネーション テクノロジーズ リミティッド 変位係数を用いたテッセレーション方法
US20170213388A1 (en) * 2016-01-25 2017-07-27 Jeffrey Neil Margolis Frame Projection For Augmented Reality Environments

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07222155A (ja) * 1994-01-31 1995-08-18 Kyocera Corp 画像符号化装置
US6466185B2 (en) 1998-04-20 2002-10-15 Alan Sullivan Multi-planar volumetric display system and method of operation using psychological vision cues
JP4408779B2 (ja) * 2004-09-15 2010-02-03 キヤノン株式会社 画像処理装置
US7404645B2 (en) * 2005-06-20 2008-07-29 Digital Display Innovations, Llc Image and light source modulation for a digital display system
EP2130381A2 (en) 2007-01-23 2009-12-09 Euclid Discoveries, LLC Computer method and apparatus for processing image data
JP2008252591A (ja) * 2007-03-30 2008-10-16 Toshiba Corp 補間フレーム生成装置、補間フレーム生成方法及び放送受信装置
US8208551B2 (en) * 2007-10-31 2012-06-26 Broadcom Corporation Method and system for hierarchically layered adaptive median motion vector smoothing
US20120021828A1 (en) 2010-02-24 2012-01-26 Valve Corporation Graphical user interface for modification of animation data using preset animation samples
JP5677002B2 (ja) * 2010-09-28 2015-02-25 キヤノン株式会社 映像制御装置、及び映像制御方法
US8624891B2 (en) 2011-01-17 2014-01-07 Disney Enterprises, Inc. Iterative reprojection of images
US9641853B2 (en) * 2013-04-15 2017-05-02 Futurewei Technologies, Inc. Method and apparatus of depth prediction mode selection
US9277129B2 (en) 2013-06-07 2016-03-01 Apple Inc. Robust image feature based video stabilization and smoothing
US11303900B2 (en) * 2013-12-06 2022-04-12 Mediatek Inc. Method and apparatus for motion boundary processing
US9824498B2 (en) * 2014-12-30 2017-11-21 Sony Interactive Entertainment Inc. Scanning display system in head-mounted display for virtual reality
WO2016137149A1 (ko) * 2015-02-24 2016-09-01 엘지전자(주) 폴리곤 유닛 기반 영상 처리 방법 및 이를 위한 장치
US10037620B2 (en) * 2015-05-29 2018-07-31 Nvidia Corporation Piecewise linear irregular rasterization
HK1249270A1 (zh) * 2015-08-19 2018-10-26 威尔乌集团 用於检测和/或校正显示器中的像素亮度和/或色度响应变化的系统和方法
US10026212B2 (en) * 2015-11-20 2018-07-17 Google Llc Electronic display stabilization using pixel velocities
US10129523B2 (en) * 2016-06-22 2018-11-13 Microsoft Technology Licensing, Llc Depth-aware reprojection
US10297086B2 (en) * 2016-07-04 2019-05-21 Liquid Cinema Inc. System and method for processing digital video
US20190045213A1 (en) * 2017-08-03 2019-02-07 Intel Corporation Reference frame reprojection for improved video coding
US10861215B2 (en) * 2018-04-30 2020-12-08 Qualcomm Incorporated Asynchronous time and space warp with determination of region of interest

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017004516A (ja) * 2015-06-05 2017-01-05 イマジネーション テクノロジーズ リミティッド 変位係数を用いたテッセレーション方法
US20170213388A1 (en) * 2016-01-25 2017-07-27 Jeffrey Neil Margolis Frame Projection For Augmented Reality Environments

Also Published As

Publication number Publication date
JP2022504475A (ja) 2022-01-13
KR20210071015A (ko) 2021-06-15
EP3857885A4 (en) 2022-06-22
US20200111195A1 (en) 2020-04-09
EP3857885A1 (en) 2021-08-04
CN113170136B (zh) 2024-04-23
CN113170136A (zh) 2021-07-23
WO2020076526A1 (en) 2020-04-16
JP7395577B2 (ja) 2023-12-11
US10733783B2 (en) 2020-08-04

Similar Documents

Publication Publication Date Title
KR102641272B1 (ko) 재투영된 프레임을 위한 모션 스무딩
JP7612700B2 (ja) 分散システムにおける動き平滑化
KR102843577B1 (ko) 시선 추적에 기초한 동적 렌더링 시간 타겟팅
KR102714790B1 (ko) 애플리케이션 렌더링 성능에 근거한 예측 및 스로틀링 조정
JP7608341B2 (ja) Ｈｍｄ内のディスプレイパネルおよび／または仮想カメラの反対への回転
KR20220116150A (ko) 머리 착용 디스플레이(hmd)와 호스트 컴퓨터 사이의 렌더링 분할
KR102794876B1 (ko) 동적 패널 마스킹
EP3379494B1 (en) Image processing device, information processing device, and image processing method
JP2026502351A (ja) 分散システムにおけるデュアルディテール符号化

Legal Events

Date Code Title Description
PA0105 International application

Patent event date: 20210427

Patent event code: PA01051R01D

Comment text: International Patent Application

PG1501 Laying open of application
A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20220908

Comment text: Request for Examination of Application

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

Comment text: Notification of reason for refusal

Patent event date: 20230616

Patent event code: PE09021S01D

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

Patent event code: PE07011S01D

Comment text: Decision to Grant Registration

Patent event date: 20240108

GRNT Written decision to grant
PR0701 Registration of establishment

Comment text: Registration of Establishment

Patent event date: 20240222

Patent event code: PR07011E01D

PR1002 Payment of registration fee

Payment date: 20240222

End annual number: 3

Start annual number: 1

PG1601 Publication of registration