JP2022534708A - 動的に反応する仮想キャラクターのためのマルチモーダルモデル - Google Patents

動的に反応する仮想キャラクターのためのマルチモーダルモデル Download PDF

Info

Publication number
JP2022534708A
JP2022534708A JP2021569969A JP2021569969A JP2022534708A JP 2022534708 A JP2022534708 A JP 2022534708A JP 2021569969 A JP2021569969 A JP 2021569969A JP 2021569969 A JP2021569969 A JP 2021569969A JP 2022534708 A JP2022534708 A JP 2022534708A
Authority
JP
Japan
Prior art keywords
virtual character
information
user
identified
environment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2021569969A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022534708A5 (https=
JPWO2020247590A5 (https=
Inventor
マッキンタイヤ－カーウィン，アルマンド
ホーリガン，ライアン
アイゼンバーグ，ジョシュ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Artie Inc
Original Assignee
Artie Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Artie Inc filed Critical Artie Inc
Publication of JP2022534708A publication Critical patent/JP2022534708A/ja
Publication of JP2022534708A5 publication Critical patent/JP2022534708A5/ja
Publication of JPWO2020247590A5 publication Critical patent/JPWO2020247590A5/ja
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
    • A63F13/65—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/70—Game security or game management aspects
    • A63F13/79—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/205—Three-dimensional [3D] animation driven by audio data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems
    • G10L13/02—Methods for producing synthetic speech; Speech synthesisers
    • G10L13/027—Concept to speech synthesisers; Generation of natural phrases from machine-based concepts
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1822—Parsing for meaning understanding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/10—Multimedia information
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/50—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by details of game servers
    • A63F2300/55—Details of game data or player data management
    • A63F2300/5546—Details of game data or player data management using player registration data, e.g. identification, account, preferences, game history
    • A63F2300/5553—Details of game data or player data management using player registration data, e.g. identification, account, preferences, game history user representation in the game field, e.g. avatar
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0381—Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2213/00—Indexing scheme for animation
    • G06T2213/08—Animation software package
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/02—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computer Graphics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
JP2021569969A 2019-06-06 2020-06-04 動的に反応する仮想キャラクターのためのマルチモーダルモデル Pending JP2022534708A (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962858234P 2019-06-06 2019-06-06
US62/858,234 2019-06-06
PCT/US2020/036068 WO2020247590A1 (en) 2019-06-06 2020-06-04 Multi-modal model for dynamically responsive virtual characters

Publications (3)

Publication Number Publication Date
JP2022534708A true JP2022534708A (ja) 2022-08-03
JP2022534708A5 JP2022534708A5 (https=) 2023-06-13
JPWO2020247590A5 JPWO2020247590A5 (https=) 2023-06-13

Family

ID=73652134

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2021569969A Pending JP2022534708A (ja) 2019-06-06 2020-06-04 動的に反応する仮想キャラクターのためのマルチモーダルモデル

Country Status (8)

Country Link
US (2) US11501480B2 (https=)
EP (1) EP3980865A4 (https=)
JP (1) JP2022534708A (https=)
KR (1) KR20220039702A (https=)
CN (1) CN114303116A (https=)
AU (1) AU2020287622A1 (https=)
CA (1) CA3137927A1 (https=)
WO (1) WO2020247590A1 (https=)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024038699A1 (ja) * 2022-08-19 2024-02-22 ソニーセミコンダクタソリューションズ株式会社 表情加工装置、表情加工方法および表情加工プログラム
JP7632925B1 (ja) 2024-02-22 2025-02-19 デジタルヒューマン株式会社 情報処理システム、情報処理方法及びプログラム
WO2025089532A1 (en) * 2023-10-23 2025-05-01 Samsung Electronics Co., Ltd. Electronic device and method for managing iot devices in a metaverse environment

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10242503B2 (en) 2017-01-09 2019-03-26 Snap Inc. Surface aware lens
US11030813B2 (en) 2018-08-30 2021-06-08 Snap Inc. Video clip object tracking
JP7142315B2 (ja) * 2018-09-27 2022-09-27 パナソニックＩｐマネジメント株式会社 説明支援装置および説明支援方法
US11176737B2 (en) 2018-11-27 2021-11-16 Snap Inc. Textured mesh building
CN113330484B (zh) 2018-12-20 2025-08-05 斯纳普公司 虚拟表面修改
US11189098B2 (en) 2019-06-28 2021-11-30 Snap Inc. 3D object camera customization system
KR20210014909A (ko) * 2019-07-31 2021-02-10 삼성전자주식회사 대상의 언어 수준을 식별하는 전자 장치 및 방법
US11232646B2 (en) * 2019-09-06 2022-01-25 Snap Inc. Context-based virtual object rendering
US11227442B1 (en) 2019-12-19 2022-01-18 Snap Inc. 3D captions with semantic graphical elements
US11093691B1 (en) * 2020-02-14 2021-08-17 Capital One Services, Llc System and method for establishing an interactive communication session
US20210375023A1 (en) * 2020-06-01 2021-12-02 Nvidia Corporation Content animation using one or more neural networks
US11763366B1 (en) * 2020-06-04 2023-09-19 Walgreen Co. Automatic initialization of customer assistance based on computer vision analysis
WO2022046674A1 (en) * 2020-08-24 2022-03-03 Sterling Labs Llc Devices and methods for motion planning of computer characters
US11756251B2 (en) * 2020-09-03 2023-09-12 Sony Interactive Entertainment Inc. Facial animation control by automatic generation of facial action units using text and speech
WO2022056151A1 (en) * 2020-09-09 2022-03-17 Colin Brady A system to convert expression input into a complex full body animation, in real time or from recordings, analyzed over time
CN115426553B (zh) * 2021-05-12 2025-01-14 海信集团控股股份有限公司 一种智能音箱及其显示方法
US12296266B2 (en) 2021-07-12 2025-05-13 Emotelogic Llc Digital character with dynamic interactive behavior
CN114201042B (zh) * 2021-11-09 2023-09-15 北京电子工程总体研究所 分布式综合集成研讨厅装置、系统、构建方法及交互方法
KR102701578B1 (ko) * 2021-12-17 2024-09-02 한국전자기술연구원 메타버스 플랫폼에서 신체활동이 어려운 환자의 활동 및 고인의 추억을 기억하기 위한 방법 및 시스템
US12346994B2 (en) 2022-01-11 2025-07-01 Meetkai, Inc Method and system for virtual intelligence user interaction
US12400634B2 (en) * 2022-04-21 2025-08-26 Google Llc Dynamically adapting given assistant output based on a given persona assigned to an automated assistant
CN114782594A (zh) * 2022-04-29 2022-07-22 北京慧夜科技有限公司 一种动画生成方法和系统
CN114995636B (zh) * 2022-05-09 2025-10-17 阿里巴巴（中国）有限公司 多模态交互方法以及装置
KR20230164954A (ko) * 2022-05-26 2023-12-05 한국전자기술연구원 대화형 가상 아바타의 구현 방법
KR102860506B1 (ko) * 2022-12-06 2025-09-16 그루브웍스 주식회사 Ai 기반 인터랙티브 아바타톡 제공 장치 및 방법
WO2024145667A1 (en) * 2022-12-30 2024-07-04 Theai, Inc. Archetype-based generation of artificial intelligence characters
US12002470B1 (en) * 2022-12-31 2024-06-04 Theai, Inc. Multi-source based knowledge data for artificial intelligence characters
WO2024170658A1 (en) * 2023-02-17 2024-08-22 Sony Semiconductor Solutions Corporation Device, method, and computer program to control an avatar
US20240303891A1 (en) * 2023-03-10 2024-09-12 Artie, Inc. Multi-modal model for dynamically responsive virtual characters
US12589309B2 (en) 2023-08-10 2026-03-31 Sony Interactive Entertainment Inc. Tailoring in-game dialogue to player attributes
KR102644550B1 (ko) * 2023-09-27 2024-03-07 셀렉트스타 주식회사 자연어처리모델을 이용한 캐릭터 영상통화 제공방법, 이를 수행하는 컴퓨팅시스템, 및 이를 구현하기 위한 컴퓨터-판독가능 기록매체
US20250124662A1 (en) * 2023-10-17 2025-04-17 Kyndryl, Inc. Preventing harassment on metaverse environments
JP2025077645A (ja) * 2023-11-07 2025-05-19 株式会社リコー 情報処理装置、情報処理方法、プログラム、情報処理システム
US20250182366A1 (en) * 2023-11-30 2025-06-05 Nvidia Corporation Interactive bot animations for interactive systems and applications
CN118135068B (zh) * 2024-05-07 2024-07-23 深圳威尔视觉科技有限公司 基于虚拟数字人的云互动方法、装置及计算机设备
US12271986B1 (en) * 2024-12-17 2025-04-08 Peakspan Capital Management, Llc Systems and methods for generating an autonomous bot that replicates speech characteristics, visual expressions, and actions of a professional
US12403402B1 (en) * 2025-03-14 2025-09-02 Bitpart AI, Inc. Multi-agent planning system for controlling non-player agents in a game
CN121349460A (zh) * 2025-12-18 2026-01-16 杭州秋果计划科技有限公司 一种基于Web前端的交互方法、系统和电子设备

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015526168A (ja) * 2012-07-26 2015-09-10 クアルコム，インコーポレイテッド 拡張現実を制御するための方法および装置
US20170256262A1 (en) * 2016-03-02 2017-09-07 Wipro Limited System and Method for Speech-to-Text Conversion

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6570555B1 (en) * 1998-12-30 2003-05-27 Fuji Xerox Co., Ltd. Method and apparatus for embodied conversational characters with multimodal input/output in an interface device
US6964023B2 (en) * 2001-02-05 2005-11-08 International Business Machines Corporation System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input
US20070015121A1 (en) * 2005-06-02 2007-01-18 University Of Southern California Interactive Foreign Language Teaching
US20070111795A1 (en) * 2005-11-15 2007-05-17 Joon-Hyuk Choi Virtual entity on a network
US8224652B2 (en) 2008-09-26 2012-07-17 Microsoft Corporation Speech and text driven HMM-based body animation synthesis
US9796095B1 (en) * 2012-08-15 2017-10-24 Hanson Robokind And Intelligent Bots, Llc System and method for controlling intelligent animated characters
US20140212854A1 (en) * 2013-01-31 2014-07-31 Sri International Multi-modal modeling of temporal interaction sequences
US9378576B2 (en) * 2013-06-07 2016-06-28 Faceshift Ag Online modeling for real-time facial animation
CN107431635B (zh) * 2015-03-27 2021-10-08 英特尔公司 化身面部表情和/或语音驱动的动画化
WO2017137947A1 (en) * 2016-02-10 2017-08-17 Vats Nitin Producing realistic talking face with expression using images text and voice
US10810780B2 (en) * 2017-07-28 2020-10-20 Baobab Studios Inc. Systems and methods for real-time complex character animations and interactivity
CN107765852A (zh) * 2017-10-11 2018-03-06 北京光年无限科技有限公司 基于虚拟人的多模态交互处理方法及系统
CN107797663A (zh) * 2017-10-26 2018-03-13 北京光年无限科技有限公司 基于虚拟人的多模态交互处理方法及系统
EP3752957A4 (en) * 2018-02-15 2021-11-17 DMAI, Inc. SYSTEM AND PROCEDURE FOR SPEECH UNDERSTANDING VIA INTEGRATED AUDIO AND VIDEO-BASED VOICE RECOGNITION
US11062494B2 (en) * 2018-03-06 2021-07-13 Didimo, Inc. Electronic messaging utilizing animatable 3D models
CN108646918A (zh) * 2018-05-10 2018-10-12 北京光年无限科技有限公司 基于虚拟人的视觉交互方法及系统
JP2022500795A (ja) * 2018-07-04 2022-01-04 ウェブ アシスタンツ ゲーエムベーハー アバターアニメーション
US11315325B2 (en) * 2018-10-09 2022-04-26 Magic Leap, Inc. Systems and methods for artificial intelligence-based virtual and augmented reality

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015526168A (ja) * 2012-07-26 2015-09-10 クアルコム，インコーポレイテッド 拡張現実を制御するための方法および装置
US20170256262A1 (en) * 2016-03-02 2017-09-07 Wipro Limited System and Method for Speech-to-Text Conversion

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024038699A1 (ja) * 2022-08-19 2024-02-22 ソニーセミコンダクタソリューションズ株式会社 表情加工装置、表情加工方法および表情加工プログラム
WO2025089532A1 (en) * 2023-10-23 2025-05-01 Samsung Electronics Co., Ltd. Electronic device and method for managing iot devices in a metaverse environment
US12554318B2 (en) 2023-10-23 2026-02-17 Samsung Electronics Co., Ltd. Electronic device and method for managing IoT devices in a metaverse environment
JP7632925B1 (ja) 2024-02-22 2025-02-19 デジタルヒューマン株式会社 情報処理システム、情報処理方法及びプログラム
JP2025128870A (ja) * 2024-02-22 2025-09-03 デジタルヒューマン株式会社 情報処理システム、情報処理方法及びプログラム

Also Published As

Publication number Publication date
EP3980865A1 (en) 2022-04-13
WO2020247590A1 (en) 2020-12-10
CA3137927A1 (en) 2020-12-10
US11501480B2 (en) 2022-11-15
KR20220039702A (ko) 2022-03-29
US20230145369A1 (en) 2023-05-11
CN114303116A (zh) 2022-04-08
EP3980865A4 (en) 2023-05-17
AU2020287622A1 (en) 2021-11-18
US20220148248A1 (en) 2022-05-12

Similar Documents

Publication Publication Date Title
US20230145369A1 (en) Multi-modal model for dynamically responsive virtual characters
Park et al. A metaverse: Taxonomy, components, applications, and open challenges
JP6902683B2 (ja) 仮想ロボットのインタラクション方法、装置、記憶媒体及び電子機器
Stappen et al. The multimodal sentiment analysis in car reviews (muse-car) dataset: Collection, insights and improvements
US12488792B2 (en) Real-time video conference chat filtering using machine learning models
Seymour et al. Actors, avatars and agents: Potentials and implications of natural face technology for the creation of realistic visual presence
US20240303891A1 (en) Multi-modal model for dynamically responsive virtual characters
JP7254772B2 (ja) ロボットインタラクションのための方法及びデバイス
US9796095B1 (en) System and method for controlling intelligent animated characters
CN112204565B (zh) 用于基于视觉背景无关语法模型推断场景的系统和方法
CN112204564A (zh) 经由基于集成音频和视觉的语音识别进行语音理解的系统和方法
CN112074899A (zh) 基于多模态传感输入的人机对话的智能发起的系统和方法
CN106663219A (zh) 处理与机器人的对话的方法和系统
JP2009077380A (ja) 画像修正方法、画像修正システム、及び画像修正プログラム
CN115461793A (zh) 交互式多模态书籍阅读的系统和方法
Nam et al. Watch buddy: Evaluating the impact of an expressive virtual agent on video consumption experience in augmented reality
US20250126329A1 (en) Interactive Video
US20240379107A1 (en) Real-time ai screening and auto-moderation of audio comments in a livestream
Pelzl et al. Designing a multimodal emotional interface in the context of negotiation
Gonzalez et al. Passing an enhanced Turing test–interacting with lifelike computer representations of specific individuals
HK40065108A (en) Multi-modal model for dynamically responsive virtual characters
Iqbal et al. A GPT-based Practical Architecture for Conversational Human Digital Twins.
US20260082016A1 (en) Private audio transmission in virtual meetings using artificial intelligence (ai)
宮脇亮輔 et al. A Data Collection Protocol, Tool, and Analysis of Multimodal Data at Different Speech Voice Levels for Avatar Facial Animation
Dobre Using machine learning to generate engaging behaviours in immersive virtual environments

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20230605

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230605

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240718

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240726

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20250124