CA3090301A1 - Methods and systems for producing content in multiple reality environments - Google Patents

Methods and systems for producing content in multiple reality environments Download PDF

Info

Publication number
CA3090301A1
CA3090301A1 CA3090301A CA3090301A CA3090301A1 CA 3090301 A1 CA3090301 A1 CA 3090301A1 CA 3090301 A CA3090301 A CA 3090301A CA 3090301 A CA3090301 A CA 3090301A CA 3090301 A1 CA3090301 A1 CA 3090301A1
Authority
CA
Canada
Prior art keywords
depth
stream
geometry
video
color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CA3090301A
Other languages
English (en)
French (fr)
Inventor
James George
Alexander PORTER
Timothy SCAFFIDI
Neil PURVEY
Patricia SHIU
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Simile Inc
Original Assignee
Simile Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Simile Inc filed Critical Simile Inc
Publication of CA3090301A1 publication Critical patent/CA3090301A1/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/73Deblurring; Sharpening
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/04Texture mapping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/08Volume rendering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/10Geometric effects
    • G06T15/40Hidden part removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/036Insert-editing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/111Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106Processing image signals
    • H04N13/15Processing image signals for colour aspects of image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/194Transmission of image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/243Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/246Calibration of cameras
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/271Image signal generators wherein the generated image signals comprise depth maps or disparity maps
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2213/00Details of stereoscopic systems
    • H04N2213/003Aspects relating to the "2D+depth" image format

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Signal Processing (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Generation (AREA)
  • Processing Or Creating Images (AREA)
CA3090301A 2018-03-08 2019-03-08 Methods and systems for producing content in multiple reality environments Pending CA3090301A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862640285P 2018-03-08 2018-03-08
US62/640,285 2018-03-08
PCT/US2019/021281 WO2019173672A1 (en) 2018-03-08 2019-03-08 Methods and systems for producing content in multiple reality environments

Publications (1)

Publication Number Publication Date
CA3090301A1 true CA3090301A1 (en) 2019-09-12

Family

ID=67846311

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3090301A Pending CA3090301A1 (en) 2018-03-08 2019-03-08 Methods and systems for producing content in multiple reality environments

Country Status (5)

Country Link
US (3) US11288864B2 (de)
EP (1) EP3762901A4 (de)
CA (1) CA3090301A1 (de)
IL (1) IL277221A (de)
WO (1) WO2019173672A1 (de)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115115780A (zh) * 2022-06-29 2022-09-27 聚好看科技股份有限公司 基于多视角rgbd相机的三维重建方法及系统

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10387730B1 (en) 2017-04-20 2019-08-20 Snap Inc. Augmented reality typography personalization system
US11348269B1 (en) * 2017-07-27 2022-05-31 AI Incorporated Method and apparatus for combining data to construct a floor plan
EP3762901A4 (de) * 2018-03-08 2021-11-17 Simile Inc. Verfahren und systeme zur herstellung von inhalt in umgebungen mit mehreren realitäten
US11972529B2 (en) * 2019-02-01 2024-04-30 Snap Inc. Augmented reality system
JP7247327B2 (ja) * 2019-04-01 2023-03-28 グーグル エルエルシー ダイナミックデプス画像を撮影して編集するための手法
JP7269515B2 (ja) * 2019-05-29 2023-05-09 日本電信電話株式会社 映像生成装置、映像生成方法、及び映像生成プログラム
JP2022551064A (ja) * 2019-10-02 2022-12-07 インターデジタル ブイシー ホールディングス フランス,エスアーエス 容積ビデオを符号化、送信、及び復号化するための方法及び装置
US20220414983A1 (en) * 2019-11-19 2022-12-29 Sony Group Corporation Information processing device, information processing method, and program
US11734873B2 (en) * 2019-12-13 2023-08-22 Sony Group Corporation Real-time volumetric visualization of 2-D images
JP7388451B2 (ja) * 2019-12-26 2023-11-29 日本電気株式会社 情報処理装置、制御方法及びプログラム
CN113256745A (zh) 2020-02-13 2021-08-13 英特尔公司 用于基于重要度值来编码的方法和装置
CN115298708A (zh) * 2020-03-30 2022-11-04 上海科技大学 多视角神经人体渲染
US12076868B2 (en) * 2020-04-08 2024-09-03 Robim Technologies Inc. Of Canada Cloud based computer-implemented visually programming method and system for robotic motions in construction
US12008740B2 (en) * 2020-08-12 2024-06-11 Niantic, Inc. Feature matching using features extracted from perspective corrected image
CN112598778B (zh) * 2020-08-28 2023-11-14 国网陕西省电力公司西咸新区供电公司 一种基于改进的纹理贴图算法的vr三维重建方法
EP4016464A1 (de) * 2020-11-26 2022-06-22 Ricoh Company, Ltd. Vorrichtung, system, verfahren und trägermittel
EP4033451A1 (de) * 2021-01-20 2022-07-27 Siemens Healthcare GmbH Interaktive bildbearbeitung mit hilfe vorzeichenbehafteter abstandsfelder
US11688126B2 (en) * 2021-02-09 2023-06-27 Canon Medical Systems Corporation Image rendering apparatus and method
US12039645B1 (en) * 2021-03-04 2024-07-16 Apple Inc. Edge contribution techniques for vector graphics rendering
CN113129391B (zh) * 2021-04-27 2023-01-31 西安邮电大学 基于多曝光图像特征分布权重的多曝光融合方法
US11501530B1 (en) 2021-09-08 2022-11-15 International Business Machines Corporation Spatio-temporal relation based MR content positioning
US20220012860A1 (en) * 2021-09-23 2022-01-13 Intel Corporation Methods and apparatus to synthesize six degree-of-freedom views from sparse rgb-depth inputs
US20240019979A1 (en) * 2022-07-15 2024-01-18 Lenovo (Singapore) Pte. Ltd. Conversion of 3d virtual actions into 2d actions
CN115270032B (zh) * 2022-08-10 2023-04-25 上海图客科技有限公司 一种基于WebGL的动态高清文字显示方法及系统
EP4345759A1 (de) * 2022-09-30 2024-04-03 Koninklijke Philips N.V. Erzeugung einer angepassten netzdarstellung für eine szene
CN115439616B (zh) * 2022-11-07 2023-02-14 成都索贝数码科技股份有限公司 基于多对象图像α叠加的异构对象表征方法
WO2024177293A1 (ko) * 2023-02-21 2024-08-29 삼성전자주식회사 디스플레이를 통해 표시되는 가상 공간 상에 외부 공간에 대응하는 이미지를 표시하기 위한 전자 장치, 방법, 및 컴퓨터 판독 가능 저장 매체
CN117156258B (zh) * 2023-10-24 2024-01-23 北京优贝在线网络科技有限公司 一种基于全景直播的多视角自切换系统

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8542252B2 (en) 2009-05-29 2013-09-24 Microsoft Corporation Target digitization, extraction, and tracking
US9607424B2 (en) 2012-06-26 2017-03-28 Lytro, Inc. Depth-assigned content for depth-enhanced pictures
US9171403B2 (en) * 2014-02-13 2015-10-27 Microsoft Technology Licensing, Llc Contour completion for augmenting surface reconstructions
TWI529662B (zh) * 2014-12-15 2016-04-11 財團法人工業技術研究院 基於深度資訊的模型建構方法、圖形處理裝置與紀錄媒體
US9928645B2 (en) * 2015-04-17 2018-03-27 Microsoft Technology Licensing, Llc Raster-based mesh decimation
US10366534B2 (en) 2015-06-10 2019-07-30 Microsoft Technology Licensing, Llc Selective surface mesh regeneration for 3-dimensional renderings
GB2543778B (en) * 2015-10-27 2018-08-08 Imagination Tech Ltd Systems and methods for processing images of objects
US10580143B2 (en) * 2015-11-04 2020-03-03 Intel Corporation High-fidelity 3D reconstruction using facial features lookup and skeletal poses in voxel models
CN109791442B (zh) * 2016-10-05 2023-04-18 奇跃公司 表面建模系统和方法
WO2018136262A1 (en) * 2017-01-20 2018-07-26 Aquifi, Inc. Systems and methods for defect detection
US10565802B2 (en) * 2017-08-31 2020-02-18 Disney Enterprises, Inc. Collaborative multi-modal mixed-reality system and methods leveraging reconfigurable tangible user interfaces for the production of immersive, cinematic, and interactive content
US10565719B2 (en) * 2017-10-13 2020-02-18 Microsoft Technology Licensing, Llc Floor detection in virtual and augmented reality devices using stereo images
EP3762901A4 (de) * 2018-03-08 2021-11-17 Simile Inc. Verfahren und systeme zur herstellung von inhalt in umgebungen mit mehreren realitäten

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115115780A (zh) * 2022-06-29 2022-09-27 聚好看科技股份有限公司 基于多视角rgbd相机的三维重建方法及系统

Also Published As

Publication number Publication date
EP3762901A4 (de) 2021-11-17
US20220215628A1 (en) 2022-07-07
IL277221A (en) 2020-10-29
US20230377269A1 (en) 2023-11-23
EP3762901A1 (de) 2021-01-13
US11721071B2 (en) 2023-08-08
US11288864B2 (en) 2022-03-29
US20210375044A1 (en) 2021-12-02
WO2019173672A1 (en) 2019-09-12

Similar Documents

Publication Publication Date Title
US11721071B2 (en) Methods and systems for producing content in multiple reality environments
CN111656407B (zh) 对动态三维模型的视图进行融合、纹理化和绘制
KR102560187B1 (ko) 3차원("3d") 장면의 2차원("2d") 캡처 이미지를 기반으로 하는 가상 현실 콘텐츠를 렌더링하기 위한 방법 및 시스템
Yemez et al. 3D reconstruction of real objects with high resolution shape and texture
US11232628B1 (en) Method for processing image data to provide for soft shadow effects using shadow depth information
KR102581134B1 (ko) 광 강도 이미지를 생성하기 위한 장치 및 방법
US11227428B2 (en) Modification of a live-action video recording using volumetric scene reconstruction to replace a designated region
US9224233B2 (en) Blending 3D model textures by image projection
Unger et al. Spatially varying image based lighting using HDR-video
CN115715464A (zh) 用于遮挡处理技术的方法和装置
Lee et al. Real time 3D avatar for interactive mixed reality
Kunert et al. An efficient diminished reality approach using real-time surface reconstruction
KR20210129685A (ko) 광 강도 이미지를 생성하기 위한 장치 및 방법
Božič et al. Neural assets: Volumetric object capture and rendering for interactive environments
Chang et al. Interactive videos: plausible video editing using sparse structure points
Hillman et al. Issues in adapting research algorithms to stereoscopic visual effects
CN113781618B (zh) 三维模型的轻量化方法、装置、电子设备和存储介质
US11887274B2 (en) Method for interpolating pixel data from image data having depth information
Keinert et al. Light field processing for media applications
Gera Casual Scene Capture and Editing for AR/VR Applications
Francois Semantic, interactive manipulation of visual data
Kravetz Polyhedral Hull Online Compositing System: Texturing and Reflections
CN115760555A (zh) 球幕图像的合成方法及装置
Frick et al. LDV generation from multi-view hybrid image and depth video
Haccius et al. A Novel Scene Representation for Digital Media

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20240308