JP7386812B2 - 照明推定 - Google Patents

照明推定 Download PDF

Info

Publication number
JP7386812B2
JP7386812B2 JP2020564910A JP2020564910A JP7386812B2 JP 7386812 B2 JP7386812 B2 JP 7386812B2 JP 2020564910 A JP2020564910 A JP 2020564910A JP 2020564910 A JP2020564910 A JP 2020564910A JP 7386812 B2 JP7386812 B2 JP 7386812B2
Authority
JP
Japan
Prior art keywords
scene
image
pixels
determining
input image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2020564910A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021524628A (ja
JPWO2019226366A5 (https=
JP2021524628A5 (https=
Inventor
ドン，ユエ
チェン，グオジュン
トン，シン
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Publication of JP2021524628A publication Critical patent/JP2021524628A/ja
Publication of JPWO2019226366A5 publication Critical patent/JPWO2019226366A5/ja
Publication of JP2021524628A5 publication Critical patent/JP2021524628A5/ja
Application granted granted Critical
Publication of JP7386812B2 publication Critical patent/JP7386812B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/50—Lighting effects
    • G06T15/60—Shadow generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/50—Lighting effects
    • G06T15/506—Illumination models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/50—Lighting effects
    • G06T15/80—Shading
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/507—Depth or shape recovery from shading
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Generation (AREA)
  • Image Processing (AREA)
JP2020564910A 2018-05-24 2019-05-13 照明推定 Active JP7386812B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201810510214.1 2018-05-24
CN201810510214.1A CN110533707B (zh) 2018-05-24 2018-05-24 照明估计
PCT/US2019/031907 WO2019226366A1 (en) 2018-05-24 2019-05-13 Lighting estimation

Publications (4)

Publication Number Publication Date
JP2021524628A JP2021524628A (ja) 2021-09-13
JPWO2019226366A5 JPWO2019226366A5 (https=) 2022-04-22
JP2021524628A5 JP2021524628A5 (https=) 2022-04-22
JP7386812B2 true JP7386812B2 (ja) 2023-11-27

Family

ID=66655477

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2020564910A Active JP7386812B2 (ja) 2018-05-24 2019-05-13 照明推定

Country Status (6)

Country Link
US (1) US11151780B2 (https=)
EP (2) EP3803803B1 (https=)
JP (1) JP7386812B2 (https=)
KR (1) KR102612808B1 (https=)
CN (1) CN110533707B (https=)
WO (1) WO2019226366A1 (https=)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7003994B2 (ja) * 2017-08-08 2022-01-21 ソニーグループ株式会社 画像処理装置および方法
CN112183551A (zh) * 2019-07-02 2021-01-05 佳能株式会社 光照颜色预测方法、图像处理方法、装置及存储介质
JP7300517B2 (ja) * 2019-11-21 2023-06-29 株式会社ソニー・インタラクティブエンタテインメント 入射光情報取得方法、入射光情報取得システム、および情報処理装置
US20230063215A1 (en) * 2020-01-23 2023-03-02 Sony Group Corporation Information processing apparatus, information processing method, and program
CN114846521B (zh) * 2020-05-20 2025-09-09 谷歌有限责任公司 从各种肖像学习照明
US12469227B2 (en) 2020-06-22 2025-11-11 Google Llc Depth-based relighting in augmented reality
CN111932641B (zh) 2020-09-27 2021-05-14 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备及存储介质
KR20230011045A (ko) 2021-07-13 2023-01-20 삼성전자주식회사 빛 추정 방법 및 장치
KR102459771B1 (ko) * 2021-11-15 2022-10-28 한국전자기술연구원 이미지 딥러닝 기반 객체의 실시간 메쉬화 장치 및 방법
KR20230092514A (ko) * 2021-12-17 2023-06-26 삼성전자주식회사 렌더링 방법 및 장치
CN114399572B (zh) * 2021-12-23 2025-07-29 网易（杭州）网络有限公司 动态阴影生成方法、装置、电子设备及存储介质
KR102559691B1 (ko) * 2022-04-04 2023-07-26 주식회사 리빌더에이아이 뉴럴 렌더링 기반 형상 색상 통합 3d 메시 복원 방법 및 장치
KR20240054111A (ko) * 2022-10-18 2024-04-25 삼성전자주식회사 전자 디바이스, 전자 디바이스의 동작 방법 및 이미지 편집 시스템
US12400395B2 (en) * 2023-03-15 2025-08-26 Nvidia Corporation Rendering and composition of neural 3D objects with non-neural assets in content generation systems and applications
KR102848513B1 (ko) 2023-11-15 2025-08-22 한국과학기술연구원 3d 공간 조명 정보 유추를 포함하는 역렌더링을 이용한 2d 이미지와 미디어 합성 장치 및 방법
US20250209670A1 (en) * 2023-12-20 2025-06-26 L'oreal Deep learning systems, devices, and methods for predicting high-dynamic range environment panoramas
CN117496359B (zh) * 2023-12-29 2024-03-22 浙江大学山东(临沂)现代农业研究院 基于三维点云的植物种植布局监测方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009163610A (ja) 2008-01-09 2009-07-23 Canon Inc 画像処理装置及び画像処理方法
JP2013156670A (ja) 2012-01-26 2013-08-15 Fujitsu Ltd 表示方法及び表示装置
JP2014164497A (ja) 2013-02-25 2014-09-08 Canon Inc 画像処理装置、画像処理方法及びプログラム
JP2015185176A (ja) 2014-03-25 2015-10-22 メタイオ ゲゼルシャフト ミット ベシュレンクテル ハフツングｍｅｔａｉｏ ＧｍｂＨ 現実環境の視野におけるバーチャルオブジェクトを表現方法及びシステム

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7262770B2 (en) * 2002-03-21 2007-08-28 Microsoft Corporation Graphics image rendering with radiance self-transfer for low-frequency lighting environments
US6791542B2 (en) 2002-06-17 2004-09-14 Mitsubishi Electric Research Laboratories, Inc. Modeling 3D objects with opacity hulls
KR101885090B1 (ko) * 2010-12-09 2018-08-03 삼성전자주식회사 영상 처리 장치, 조명 처리 장치 및 그 방법
US9857470B2 (en) 2012-12-28 2018-01-02 Microsoft Technology Licensing, Llc Using photometric stereo for 3D environment modeling
US9818224B1 (en) * 2013-06-20 2017-11-14 Amazon Technologies, Inc. Augmented reality images based on color and depth information
EP2829842B1 (en) * 2013-07-22 2022-12-21 Hexagon Technology Center GmbH Method, system and computer programme product for determination of an absolute volume of a stock pile using a structure from motion algorithm
US9412040B2 (en) * 2013-12-04 2016-08-09 Mitsubishi Electric Research Laboratories, Inc. Method for extracting planes from 3D point cloud sensor data
US20150262412A1 (en) * 2014-03-17 2015-09-17 Qualcomm Incorporated Augmented reality lighting with dynamic geometry
US20160140761A1 (en) * 2014-11-19 2016-05-19 Microsoft Technology Licensing, Llc. Using depth information for drawing in augmented reality scenes
US10008032B2 (en) * 2015-03-03 2018-06-26 Imagination Technologies Limited Graphics processing using directional representations of lighting at probe positions within a scene
US10354449B2 (en) 2015-06-12 2019-07-16 Hand Held Products, Inc. Augmented reality lighting effects
CN105825544B (zh) * 2015-11-25 2019-08-20 维沃移动通信有限公司 一种图像处理方法及移动终端
US20180308281A1 (en) 2016-04-01 2018-10-25 draw, Inc. 3-d graphic generation, artificial intelligence verification and learning system, program, and method
CN107341853B (zh) * 2017-07-13 2021-02-12 河北中科恒运软件科技股份有限公司 超大虚拟场景和动态摄屏的虚实融合方法及系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009163610A (ja) 2008-01-09 2009-07-23 Canon Inc 画像処理装置及び画像処理方法
JP2013156670A (ja) 2012-01-26 2013-08-15 Fujitsu Ltd 表示方法及び表示装置
JP2014164497A (ja) 2013-02-25 2014-09-08 Canon Inc 画像処理装置、画像処理方法及びプログラム
JP2015185176A (ja) 2014-03-25 2015-10-22 メタイオ ゲゼルシャフト ミット ベシュレンクテル ハフツングｍｅｔａｉｏ ＧｍｂＨ 現実環境の視野におけるバーチャルオブジェクトを表現方法及びシステム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
平原 匡哲，ＲＧＢ－Ｄカメラを用いた確率的探索による光源推定，映像情報メディア学会技術報告 Ｖｏｌ．３７ Ｎｏ．３６ ITE Technical Report，日本，（一社）映像情報メディア学会 The Institute of Image Information and Television Engineers，2013年08月12日，第37巻，p.9-12

Also Published As

Publication number Publication date
US20210027526A1 (en) 2021-01-28
WO2019226366A1 (en) 2019-11-28
EP3803803A1 (en) 2021-04-14
EP4607466A3 (en) 2025-10-29
US11151780B2 (en) 2021-10-19
KR20210013150A (ko) 2021-02-03
EP4607466A2 (en) 2025-08-27
EP3803803B1 (en) 2025-07-16
JP2021524628A (ja) 2021-09-13
CN110533707B (zh) 2023-04-14
CN110533707A (zh) 2019-12-03
KR102612808B1 (ko) 2023-12-11

Similar Documents

Publication Publication Date Title
JP7386812B2 (ja) 照明推定
US20240265561A1 (en) Mesh reconstruction using data-driven priors
US10776688B2 (en) Multi-frame video interpolation using optical flow
US12394140B2 (en) Sub-pixel data simulation system
CN120107454A (zh) 使用生成模型的四维对象和场景模型合成
US11954830B2 (en) High dynamic range support for legacy applications
CN113139992A (zh) 多分辨率体素网格化
US20220284663A1 (en) Method and apparatus with image processing and reconstructed image generation
US12406422B2 (en) 3D digital avatar generation from a single or few portrait images
US20240161391A1 (en) Relightable neural radiance field model
CN115439628A (zh) 通过拓扑采样进行接合点形状和外观优化
CN111161398B (zh) 一种图像生成方法、装置、设备及存储介质
EP4207089A1 (en) Image processing method and apparatus
CN117372607A (zh) 一种三维模型的生成方法、装置及电子设备
CN115797543A (zh) 单个图像逆向渲染
CN118279465A (zh) 图像处理方法、装置和存储介质
JP7827262B2 (ja) ３ｄ姿勢予測装置及び方法
CN120635100A (zh) 基于高斯泼溅辐射场模型的实时多实例分割方法及装置
CN116188349A (zh) 图像处理方法、装置、电子设备及存储介质
GB2536060B (en) Virtual trying-on experience
CN118505872A (zh) 一种图像渲染方法、图像渲染装置及计算机可读存储介质
US20210209798A1 (en) Method and apparatus for processing sparse points
CN115619924A (zh) 用于光估计的方法与装置
CN112652056A (zh) 一种3d信息展示方法及装置

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220412

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20220412

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20230410

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20230418

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230713

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20231017

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20231114

R150 Certificate of patent or registration of utility model

Ref document number: 7386812

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150