JP7437327B2 - 異種入力の補間のための方法およびシステム - Google Patents

異種入力の補間のための方法およびシステム Download PDF

Info

Publication number
JP7437327B2
JP7437327B2 JP2020571504A JP2020571504A JP7437327B2 JP 7437327 B2 JP7437327 B2 JP 7437327B2 JP 2020571504 A JP2020571504 A JP 2020571504A JP 2020571504 A JP2020571504 A JP 2020571504A JP 7437327 B2 JP7437327 B2 JP 7437327B2
Authority
JP
Japan
Prior art keywords
input
user
angular
component
linear
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2020571504A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021529380A (ja
JP2021529380A5 (https=
Inventor
ジェフリー ウェディグ，
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of JP2021529380A publication Critical patent/JP2021529380A/ja
Publication of JP2021529380A5 publication Critical patent/JP2021529380A5/ja
Application granted granted Critical
Publication of JP7437327B2 publication Critical patent/JP7437327B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)
JP2020571504A 2018-07-02 2019-06-19 異種入力の補間のための方法およびシステム Active JP7437327B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201862693237P 2018-07-02 2018-07-02
US62/693,237 2018-07-02
PCT/US2019/037952 WO2020009800A1 (en) 2018-07-02 2019-06-19 Methods and systems for interpolation of disparate inputs

Publications (3)

Publication Number Publication Date
JP2021529380A JP2021529380A (ja) 2021-10-28
JP2021529380A5 JP2021529380A5 (https=) 2022-06-27
JP7437327B2 true JP7437327B2 (ja) 2024-02-22

Family

ID=69055205

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2020571504A Active JP7437327B2 (ja) 2018-07-02 2019-06-19 異種入力の補間のための方法およびシステム

Country Status (5)

Country Link
US (1) US11669726B2 (https=)
EP (1) EP3818530A4 (https=)
JP (1) JP7437327B2 (https=)
CN (1) CN112602090A (https=)
WO (1) WO2020009800A1 (https=)

Families Citing this family (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11113887B2 (en) * 2018-01-08 2021-09-07 Verizon Patent And Licensing Inc Generating three-dimensional content from two-dimensional images
CN112602090A (zh) 2018-07-02 2021-04-02 奇跃公司 用于插值不同输入的方法和系统
US12154295B2 (en) 2018-07-02 2024-11-26 Magic Leap, Inc. Methods and systems for interpolation of disparate inputs
US10984587B2 (en) * 2018-07-13 2021-04-20 Nvidia Corporation Virtual photogrammetry
US11496723B1 (en) * 2018-09-28 2022-11-08 Apple Inc. Automatically capturing a moment
US10679397B1 (en) 2018-12-13 2020-06-09 Universal City Studios Llc Object tracking animated figure systems and methods
US11132606B2 (en) * 2019-03-15 2021-09-28 Sony Interactive Entertainment Inc. Reinforcement learning to train a character using disparate target animation data
KR102243040B1 (ko) * 2019-04-10 2021-04-21 한양대학교 산학협력단 전자 장치, 아바타 얼굴 표정 표시 시스템 및 제어 방법
WO2021094537A1 (en) * 2019-11-15 2021-05-20 Ariel Ai Ltd 3d body model generation
US11687778B2 (en) 2020-01-06 2023-06-27 The Research Foundation For The State University Of New York Fakecatcher: detection of synthetic portrait videos using biological signals
CN111223171B (zh) * 2020-01-14 2025-06-27 腾讯科技（深圳）有限公司 图像处理方法、装置、电子设备及存储介质
WO2021226128A1 (en) * 2020-05-04 2021-11-11 Rct Studio Inc. Methods and systems for 3d character pose prediction based on phase-functioned neural networks
US11113862B1 (en) * 2020-05-11 2021-09-07 Sony Interactive Entertainment Inc. Simulating motion of computer simulation characters to account for simulated injuries to the characters using current motion model
GB2598452B (en) 2020-06-22 2024-01-10 Snap Inc 3D object model reconstruction from 2D images
KR20220018760A (ko) * 2020-08-07 2022-02-15 삼성전자주식회사 단말에 3d 캐릭터 이미지를 제공하는 엣지 데이터 네트워크 및 그 동작 방법
US11430217B2 (en) * 2020-12-11 2022-08-30 Microsoft Technology Licensing, Llc Object of interest colorization
US11830121B1 (en) 2021-01-26 2023-11-28 Electronic Arts Inc. Neural animation layering for synthesizing martial arts movements
US12148102B2 (en) * 2021-08-25 2024-11-19 Sony Interactive Entertainment Inc. Dynamic augmentation of stimuli based on profile of user
US20230065631A1 (en) * 2021-08-25 2023-03-02 The Boeing Company Extended reality systems, apparatus, and methods for musculoskeletal ergonomic improvement
CN113902876B (zh) * 2021-12-06 2022-02-11 腾讯科技（深圳）有限公司 骨骼蒙皮方法、装置、计算机设备和存储介质
US12374014B2 (en) 2021-12-07 2025-07-29 Electronic Arts Inc. Predicting facial expressions using character motion states
US20230182005A1 (en) * 2021-12-13 2023-06-15 Board Of Regents, The University Of Texas System Controlling multicomputer interaction with deep learning and artificial intelligence
CN114241396A (zh) * 2021-12-31 2022-03-25 深圳市智岩科技有限公司 灯效控制方法、系统、装置、电子设备及存储介质
US20230245454A1 (en) * 2022-01-31 2023-08-03 Aktify, Inc. Presenting audio/video responses based on intent derived from features of audio/video interactions
US12205214B2 (en) * 2022-02-23 2025-01-21 Electronic Arts Inc. Joint twist generation for animation
US12403400B2 (en) 2022-03-31 2025-09-02 Electronic Arts Inc. Learning character motion alignment with periodic autoencoders
CN114647650A (zh) * 2022-05-18 2022-06-21 苏州琞能能源科技有限公司 数据存储方法、装置、电子设备及存储介质
CN116176658A (zh) * 2023-02-28 2023-05-30 南京交通职业技术学院 一种基于多源信息智能融合的地铁列车测速方法
JP2025068411A (ja) * 2023-10-16 2025-04-28 オムロン株式会社 学習装置、操作情報変換装置及び学習方法
CN117240602B (zh) * 2023-11-09 2024-01-19 北京中海通科技有限公司 一种身份认证平台安全防护方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010528716A (ja) 2007-05-29 2010-08-26 シーエフピーエイチ， エル．エル．シー． 手のモーションコントロールを有するゲーム
JP2015531098A (ja) 2012-06-21 2015-10-29 マイクロソフト コーポレーション デプスカメラを使用するアバター構築
US20180025257A1 (en) 2016-01-25 2018-01-25 Google Inc. Generating images using neural networks
JP2018504663A (ja) 2014-12-05 2018-02-15 マイフィジーク リミテッド ボディの画像化
US20180089553A1 (en) 2016-09-27 2018-03-29 Disney Enterprises, Inc. Learning to schedule control fragments for physics-based character simulation and robots using deep q-learning

Family Cites Families (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6222525B1 (en) 1992-03-05 2001-04-24 Brad A. Armstrong Image controllers with sheet connected sensors
US5670988A (en) 1995-09-05 1997-09-23 Interlink Electronics, Inc. Trigger operated electronic device
US9525696B2 (en) * 2000-09-25 2016-12-20 Blue Coat Systems, Inc. Systems and methods for processing data flows
GB2403363A (en) 2003-06-25 2004-12-29 Hewlett Packard Development Co Tags for automated image processing
US11428937B2 (en) 2005-10-07 2022-08-30 Percept Technologies Enhanced optical and perceptual digital eyewear
US8696113B2 (en) 2005-10-07 2014-04-15 Percept Technologies Inc. Enhanced optical and perceptual digital eyewear
US20070081123A1 (en) 2005-10-07 2007-04-12 Lewis Scott W Digital eyewear
WO2009072264A1 (ja) * 2007-12-03 2009-06-11 Panasonic Corporation 画像処理装置、撮影装置、再生装置、集積回路及び画像処理方法
US9304319B2 (en) 2010-11-18 2016-04-05 Microsoft Technology Licensing, Llc Automatic focus improvement for augmented reality displays
US10156722B2 (en) 2010-12-24 2018-12-18 Magic Leap, Inc. Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
NZ706893A (en) 2010-12-24 2017-02-24 Magic Leap Inc An ergonomic head mounted display device and optical system
CA3035118C (en) 2011-05-06 2022-01-04 Magic Leap, Inc. Massive simultaneous remote digital presence world
US11074495B2 (en) * 2013-02-28 2021-07-27 Z Advanced Computing, Inc. (Zac) System and method for extremely efficient image and pattern recognition and artificial intelligence platform
EP2760363A4 (en) 2011-09-29 2015-06-24 Magic Leap Inc TACTILE GLOVE FOR HUMAN COMPUTER INTERACTION
RU2017115669A (ru) 2011-10-28 2019-01-28 Мэджик Лип, Инк. Система и способ для дополненной и виртуальной реальности
BR112014024941A2 (pt) 2012-04-05 2017-09-19 Magic Leap Inc dispositivo de imagem de campo de visão amplo com capacidade de focalização ativa
US9671566B2 (en) 2012-06-11 2017-06-06 Magic Leap, Inc. Planar waveguide apparatus with diffraction element(s) and system employing same
JP2015534108A (ja) 2012-09-11 2015-11-26 マジック リープ， インコーポレイテッド 人間工学的な頭部搭載型ディスプレイデバイスおよび光学システム
IL301489B2 (en) 2013-01-15 2024-08-01 Magic Leap Inc A system for scanning electromagnetic imaging radiation
KR20230044041A (ko) 2013-03-11 2023-03-31 매직 립, 인코포레이티드 증강 및 가상 현실을 위한 시스템 및 방법
CN105229719B (zh) 2013-03-15 2018-04-27 奇跃公司 显示系统和方法
US9874749B2 (en) 2013-11-27 2018-01-23 Magic Leap, Inc. Virtual and augmented reality systems and methods
US10262462B2 (en) 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
EP3058418B1 (en) 2013-10-16 2023-10-04 Magic Leap, Inc. Virtual or augmented reality headsets having adjustable interpupillary distance
KR102651578B1 (ko) 2013-11-27 2024-03-25 매직 립, 인코포레이티드 가상 및 증강 현실 시스템들 및 방법들
US9857591B2 (en) 2014-05-30 2018-01-02 Magic Leap, Inc. Methods and system for creating focal planes in virtual and augmented reality
KR102177133B1 (ko) 2014-01-31 2020-11-10 매직 립, 인코포레이티드 멀티-포컬 디스플레이 시스템 및 방법
NZ722903A (en) 2014-01-31 2020-05-29 Magic Leap Inc Multi-focal display system and method
US10203762B2 (en) 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
EP3140779A4 (en) 2014-05-09 2017-11-29 Google LLC Systems and methods for using eye signals with secure mobile communications
NZ764905A (en) 2014-05-30 2022-05-27 Magic Leap Inc Methods and systems for generating virtual content display with a virtual or augmented reality apparatus
NZ773822A (en) 2015-03-16 2022-07-29 Magic Leap Inc Methods and systems for diagnosing and treating health ailments
US9443192B1 (en) * 2015-08-30 2016-09-13 Jasmin Cosic Universal artificial intelligence engine for autonomous computing devices and software applications
US10535174B1 (en) * 2017-09-14 2020-01-14 Electronic Arts Inc. Particle-based inverse kinematic rendering system
US11276219B2 (en) 2018-04-16 2022-03-15 Magic Leap, Inc. Systems and methods for cross-application authoring, transfer, and evaluation of rigging control systems for virtual characters
CN112602090A (zh) 2018-07-02 2021-04-02 奇跃公司 用于插值不同输入的方法和系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010528716A (ja) 2007-05-29 2010-08-26 シーエフピーエイチ， エル．エル．シー． 手のモーションコントロールを有するゲーム
JP2015531098A (ja) 2012-06-21 2015-10-29 マイクロソフト コーポレーション デプスカメラを使用するアバター構築
JP2018504663A (ja) 2014-12-05 2018-02-15 マイフィジーク リミテッド ボディの画像化
US20180025257A1 (en) 2016-01-25 2018-01-25 Google Inc. Generating images using neural networks
US20180089553A1 (en) 2016-09-27 2018-03-29 Disney Enterprises, Inc. Learning to schedule control fragments for physics-based character simulation and robots using deep q-learning

Also Published As

Publication number Publication date
CN112602090A (zh) 2021-04-02
EP3818530A1 (en) 2021-05-12
US20200005138A1 (en) 2020-01-02
WO2020009800A1 (en) 2020-01-09
JP2021529380A (ja) 2021-10-28
EP3818530A4 (en) 2022-03-30
US11669726B2 (en) 2023-06-06

Similar Documents

Publication Publication Date Title
JP7437327B2 (ja) 異種入力の補間のための方法およびシステム
US12033263B2 (en) Using three-dimensional scans of a physical subject to determine positions and/or orientations of skeletal joints in the rigging for a virtual character
JP7459050B2 (ja) 仮想キャラクタの姿勢空間変形のための姿勢空間次元低減
US11763510B2 (en) Avatar animation using markov decision process policies
JP7342008B2 (ja) 仮想アバタのためのメッシュの合致
US11557076B2 (en) Computer generated hair groom transfer tool
JP7355748B2 (ja) 眼球位置合わせを使用した頭部走査整合
JP2023165936A (ja) 仮想アバタをアニメーション化するための骨格システム
US12229861B2 (en) Systems and methods for cross-application authoring, transfer, and evaluation of rigging control systems for virtual characters
US12154295B2 (en) Methods and systems for interpolation of disparate inputs

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220617

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20220617

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20230712

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20230725

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231024

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240111

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240209

R150 Certificate of patent or registration of utility model

Ref document number: 7437327

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150