JP7635779B2 - 学習システム及びデータ収集装置 - Google Patents

学習システム及びデータ収集装置 Download PDF

Info

Publication number
JP7635779B2
JP7635779B2 JP2022512040A JP2022512040A JP7635779B2 JP 7635779 B2 JP7635779 B2 JP 7635779B2 JP 2022512040 A JP2022512040 A JP 2022512040A JP 2022512040 A JP2022512040 A JP 2022512040A JP 7635779 B2 JP7635779 B2 JP 7635779B2
Authority
JP
Japan
Prior art keywords
learning
data
unit
emotion
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022512040A
Other languages
English (en)
Japanese (ja)
Other versions
JPWO2021200503A1 (https=
Inventor
アンドリュー シン
由幸 小林
健二 鈴木
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Sony Group Corp
Original Assignee
Sony Corp
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp, Sony Group Corp filed Critical Sony Corp
Publication of JPWO2021200503A1 publication Critical patent/JPWO2021200503A1/ja
Application granted granted Critical
Publication of JP7635779B2 publication Critical patent/JP7635779B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/091—Active learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/10—Speech classification or search using distance or distortion measures between unknown speech and reference templates

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Medical Informatics (AREA)
  • Dermatology (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Computer Graphics (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
JP2022512040A 2020-03-31 2021-03-24 学習システム及びデータ収集装置 Active JP7635779B2 (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
JP2020065069 2020-03-31
JP2020065069 2020-03-31
JP2020120049 2020-07-13
JP2020120049 2020-07-13
PCT/JP2021/012368 WO2021200503A1 (ja) 2020-03-31 2021-03-24 学習システム及びデータ収集装置

Publications (2)

Publication Number Publication Date
JPWO2021200503A1 JPWO2021200503A1 (https=) 2021-10-07
JP7635779B2 true JP7635779B2 (ja) 2025-02-26

Family

ID=77928815

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022512040A Active JP7635779B2 (ja) 2020-03-31 2021-03-24 学習システム及びデータ収集装置

Country Status (3)

Country Link
US (1) US12541997B2 (https=)
JP (1) JP7635779B2 (https=)
WO (1) WO2021200503A1 (https=)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6800453B1 (ja) * 2020-05-07 2020-12-16 株式会社 情報システムエンジニアリング 情報処理装置及び情報処理方法
KR20240033217A (ko) * 2021-07-15 2024-03-12 소니그룹주식회사 신호 처리 장치 및 방법
EP4137801B1 (en) * 2021-08-17 2025-09-24 Hitachi High-Tech Analytical Science Finland Oy Monitoring reliability of analysis of elemental composition of a sample
WO2023055267A1 (en) * 2021-09-29 2023-04-06 Telefonaktiebolaget Lm Ericsson (Publ) Efficient transmission of decoding information
KR20230089215A (ko) * 2021-12-13 2023-06-20 삼성전자주식회사 획득된 정보에 기반하여 화면을 구성하기 위한 전자 장치 및 방법
WO2023119577A1 (ja) * 2021-12-23 2023-06-29 楽天グループ株式会社 情報処理システム、情報処理方法及びプログラム
US20240221100A1 (en) * 2021-12-23 2024-07-04 Rakuten Group, Inc. Information processing system, information processing method and program
JP2023106888A (ja) * 2022-01-21 2023-08-02 オムロン株式会社 情報処理装置および情報処理方法
US12327430B2 (en) * 2022-06-24 2025-06-10 Microsoft Technology Licensing, Llc Simulated capacitance measurements for facial expression recognition training
US12450806B2 (en) * 2022-07-26 2025-10-21 Verizon Patent And Licensing Inc. System and method for generating emotionally-aware virtual facial expressions
US12347135B2 (en) * 2022-11-14 2025-07-01 Adobe Inc. Generating gesture reenactment video from video motion graphs using machine learning
US20240193920A1 (en) * 2022-12-13 2024-06-13 Korea Electronics Technology Institute Method for predicting user personality by mapping multimodal information on personality expression space
US20240371397A1 (en) * 2023-05-03 2024-11-07 KAI Conversations Limited System for processing text, image and audio signals using artificial intelligence and method thereof
KR20250031866A (ko) * 2023-08-29 2025-03-07 포항공과대학교 산학협력단 이미지 변화 데이터 생성 방법 및 장치
WO2026028784A1 (ja) * 2024-07-29 2026-02-05 富士フイルム株式会社 情報処理装置、情報処理装置の作動方法及び作動プログラム

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008269065A (ja) 2007-04-17 2008-11-06 Nippon Telegr & Teleph Corp <Ntt> ユーザ支援方法、ユーザ支援装置およびユーザ支援プログラム
JP2009111938A (ja) 2007-11-01 2009-05-21 Nippon Telegr & Teleph Corp <Ntt> 情報編集装置、情報編集方法、情報編集プログラムおよびそのプログラムを記録した記録媒体
JP2010093584A (ja) 2008-10-08 2010-04-22 Nippon Telegr & Teleph Corp <Ntt> 視聴印象推定方法及び装置及びプログラム及びコンピュータ読み取り可能な記録媒体
JP2012160082A (ja) 2011-02-01 2012-08-23 Bond:Kk 入力支援装置、入力支援方法及びプログラム
WO2018030206A1 (ja) 2016-08-10 2018-02-15 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ カメラワーク生成方法及び映像処理装置
WO2019215778A1 (ja) 2018-05-07 2019-11-14 日本電気株式会社 データ提供システムおよびデータ収集システム

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1185719A (ja) * 1997-09-03 1999-03-30 Matsushita Electric Ind Co Ltd パラメータ推定装置
JP6900918B2 (ja) 2017-03-14 2021-07-07 オムロン株式会社 学習装置及び学習方法
US10726078B2 (en) * 2017-05-09 2020-07-28 Oath Inc. Method and system for dynamic score floor modeling and application thereof
WO2019050508A1 (en) * 2017-09-06 2019-03-14 Hitachi Data Systems Corporation EMOTION DETECTION ACTIVATED VIDEO CUSHIONING
CN113168439A (zh) * 2019-02-22 2021-07-23 居米奥公司 为算法决定提供结果解释
US11393144B2 (en) * 2019-04-11 2022-07-19 City University Of Hong Kong System and method for rendering an image

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008269065A (ja) 2007-04-17 2008-11-06 Nippon Telegr & Teleph Corp <Ntt> ユーザ支援方法、ユーザ支援装置およびユーザ支援プログラム
JP2009111938A (ja) 2007-11-01 2009-05-21 Nippon Telegr & Teleph Corp <Ntt> 情報編集装置、情報編集方法、情報編集プログラムおよびそのプログラムを記録した記録媒体
JP2010093584A (ja) 2008-10-08 2010-04-22 Nippon Telegr & Teleph Corp <Ntt> 視聴印象推定方法及び装置及びプログラム及びコンピュータ読み取り可能な記録媒体
JP2012160082A (ja) 2011-02-01 2012-08-23 Bond:Kk 入力支援装置、入力支援方法及びプログラム
WO2018030206A1 (ja) 2016-08-10 2018-02-15 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ カメラワーク生成方法及び映像処理装置
WO2019215778A1 (ja) 2018-05-07 2019-11-14 日本電気株式会社 データ提供システムおよびデータ収集システム

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
横井直明 ほか，"ＡＩの予測結果に対する納得度を高める予測根拠解釈支援技術の提案"，電子情報通信学会技術研究報告，一般社団法人電子情報通信学会，2019年03月10日，Vol. 118，No. 513，p. 61-66
猪貝光祥，"ディープラーニング技術を用いた高速な画像認識ソリューション"，月刊自動認識，2020年03月10日，Vol. 33，No .3，p. 33-38，ISSN: 0915-1060

Also Published As

Publication number Publication date
US12541997B2 (en) 2026-02-03
JPWO2021200503A1 (https=) 2021-10-07
WO2021200503A1 (ja) 2021-10-07
US20230360437A1 (en) 2023-11-09

Similar Documents

Publication Publication Date Title
JP7635779B2 (ja) 学習システム及びデータ収集装置
US20190332952A1 (en) Learning device, image pickup apparatus, image processing device, learning method, non-transient computer-readable recording medium for recording learning program, display control method and inference model manufacturing method
WO2019085585A1 (zh) 设备控制处理方法及装置
US20220335246A1 (en) System And Method For Video Processing
CN103079034A (zh) 一种感知拍摄方法及系统
KR20100055946A (ko) 동영상 썸네일 생성 방법 및 장치
TWI857242B (zh) 光流資訊預測方法、裝置、電子設備和儲存媒體
CN120182488A (zh) 一种基于用户行为的沉浸式展厅智能导览展示方法及系统
CN105960801A (zh) 增强视频会议
US20150281586A1 (en) Method and apparatus for forming a video sequence
Vacher et al. The CIRDO corpus: comprehensive audio/video database of domestic falls of elderly people
CN115035007A (zh) 基于像素级对齐生成对抗网络的人脸老化系统及建立方法
CN116016978B (zh) 在线课堂的画面导播方法、装置、电子设备及存储介质
KR101839406B1 (ko) 디스플레이장치 및 그 제어방법
US20200259999A1 (en) Intelligent photography with machine learning
CN108810398B (zh) 图像处理装置、图像处理方法以及记录介质
Chen et al. Hierarchical cross-modal talking face generationwith dynamic pixel-wise loss
CN120434504A (zh) 一种基于跨模态蒸馏的声视协同调焦方法、装置及设备
CN120412048A (zh) 一种基于视觉场景分析算法模型的分析方法和ai眼镜
CN104780341B (zh) 一种信息处理方法以及信息处理装置
US20240348885A1 (en) System and method for question answering
JP2017041857A (ja) 画像処理装置、その制御方法、プログラム及び撮像装置
WO2024062971A1 (ja) 情報処理装置、情報処理方法および情報処理プログラム
WO2022014143A1 (ja) 撮像システム
US11523047B2 (en) Imaging device, imaging method, and program

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240202

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20240827

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20241009

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250114

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250127

R150 Certificate of patent or registration of utility model

Ref document number: 7635779

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150