JP7635234B2 - 連合混合モデル - Google Patents

連合混合モデル Download PDF

Info

Publication number
JP7635234B2
JP7635234B2 JP2022534677A JP2022534677A JP7635234B2 JP 7635234 B2 JP7635234 B2 JP 7635234B2 JP 2022534677 A JP2022534677 A JP 2022534677A JP 2022534677 A JP2022534677 A JP 2022534677A JP 7635234 B2 JP7635234 B2 JP 7635234B2
Authority
JP
Japan
Prior art keywords
machine learning
learning model
processing device
parameters
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2022534677A
Other languages
English (en)
Japanese (ja)
Other versions
JP2023505973A (ja
JP2023505973A5 (https=
JPWO2021119601A5 (https=
Inventor
マティアス・ライサー
マックス・ウェリング
エフストラティオス・ゲイヴス
クリストス・ルイゾス
Original Assignee
クアルコム テクノロジーズ インコーポレイテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by クアルコム テクノロジーズ インコーポレイテッド filed Critical クアルコム テクノロジーズ インコーポレイテッド
Publication of JP2023505973A publication Critical patent/JP2023505973A/ja
Publication of JP2023505973A5 publication Critical patent/JP2023505973A5/ja
Publication of JPWO2021119601A5 publication Critical patent/JPWO2021119601A5/ja
Application granted granted Critical
Publication of JP7635234B2 publication Critical patent/JP7635234B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Machine Translation (AREA)
JP2022534677A 2019-12-13 2020-12-14 連合混合モデル Active JP7635234B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
GR20190100556 2019-12-13
GR20190100556 2019-12-13
PCT/US2020/064889 WO2021119601A1 (en) 2019-12-13 2020-12-14 Federated mixture models

Publications (4)

Publication Number Publication Date
JP2023505973A JP2023505973A (ja) 2023-02-14
JP2023505973A5 JP2023505973A5 (https=) 2023-12-08
JPWO2021119601A5 JPWO2021119601A5 (https=) 2023-12-08
JP7635234B2 true JP7635234B2 (ja) 2025-02-25

Family

ID=74175956

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2022534677A Active JP7635234B2 (ja) 2019-12-13 2020-12-14 連合混合モデル

Country Status (7)

Country Link
US (1) US20230036702A1 (https=)
EP (1) EP4073714A1 (https=)
JP (1) JP7635234B2 (https=)
KR (1) KR20220112766A (https=)
CN (1) CN114787824B (https=)
BR (1) BR112022011012A2 (https=)
WO (1) WO2021119601A1 (https=)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210312336A1 (en) * 2020-04-03 2021-10-07 International Business Machines Corporation Federated learning of machine learning model features
US11842260B2 (en) * 2020-09-25 2023-12-12 International Business Machines Corporation Incremental and decentralized model pruning in federated machine learning
US11790039B2 (en) * 2020-10-29 2023-10-17 EMC IP Holding Company LLC Compression switching for federated learning
CN113516249B (zh) * 2021-06-18 2023-04-07 重庆大学 基于半异步的联邦学习方法、系统、服务器及介质
US11777812B2 (en) * 2021-06-25 2023-10-03 Qualcomm Technologies, Inc. Zone-based federated learning
CN113435537B (zh) * 2021-07-16 2022-08-26 同盾控股有限公司 基于Soft GBDT的跨特征联邦学习方法、预测方法
US11443245B1 (en) * 2021-07-22 2022-09-13 Alipay Labs (singapore) Pte. Ltd. Method and system for federated adversarial domain adaptation
KR20240058131A (ko) * 2021-08-31 2024-05-03 도쿄엘렉트론가부시키가이샤 정보 처리 방법, 정보 처리 장치, 및 정보 처리 시스템
JP7566707B2 (ja) * 2021-09-15 2024-10-15 株式会社東芝 学習システムおよび方法
US20230110602A1 (en) * 2021-10-13 2023-04-13 International Business Machines Corporation Federated learning model lineage
US20230110975A1 (en) * 2021-10-13 2023-04-13 International Business Machines Corporation Recommending model contributions based on federated learning lineage
US20230117768A1 (en) * 2021-10-15 2023-04-20 Kiarash SHALOUDEGI Methods and systems for updating optimization parameters of a parameterized optimization algorithm in federated learning
CN114004363B (zh) * 2021-10-27 2024-05-31 支付宝(杭州)信息技术有限公司 联合更新模型的方法、装置及系统
EP4238291B1 (en) * 2021-11-16 2025-08-27 Huawei Technologies Co., Ltd. Management entity, network element, and methods for supporting anomaly detection for communication networks
CH719269B1 (de) * 2021-12-17 2024-04-15 Palantir Technologies Inc Verfahren zur Identifizierung eines Zielobjekts und Modell- und Sensororchestrierungssystem
CN116418686A (zh) * 2021-12-31 2023-07-11 华为技术有限公司 模型的数据处理方法及装置
CN114781540B (zh) * 2022-05-09 2025-02-07 国网智能电网研究院有限公司 基于电力物联网的全局模型生成方法、装置、设备及介质
EP4524830A4 (en) * 2022-05-30 2025-07-16 Huawei Tech Co Ltd MODEL TRAINING SYSTEM, MODEL TRAINING METHOD, TRAINING DEVICE, AND TRAINING NODE
EP4296909A1 (de) * 2022-06-22 2023-12-27 Siemens Aktiengesellschaft Individuelle testmodelle für generalisierte maschinelle lernmodelle
KR102573880B1 (ko) * 2022-07-21 2023-09-06 고려대학교 산학협력단 다중-너비 인공신경망에 기반한 연합 학습 시스템 및 연합 학습 방법
KR102684383B1 (ko) * 2022-12-22 2024-07-12 서울과학기술대학교 산학협력단 블록체인 기반 부분 모델 동기화 방법
US20240242505A1 (en) * 2023-01-14 2024-07-18 Radiusai, Inc. Visual analytics
KR20240114646A (ko) * 2023-01-17 2024-07-24 숭실대학교산학협력단 파라미터 부분 공유에 기초한 개인화 연합 학습 방법 및 장치
CN116597672B (zh) * 2023-06-14 2024-02-13 南京云创大数据科技股份有限公司 基于多智能体近端策略优化算法的区域信号灯控制方法
CN116975683B (zh) * 2023-06-27 2025-10-10 中国科学技术大学 一种基于用户分簇和模型分层的个性化联邦学习方法
US20250077948A1 (en) * 2023-08-29 2025-03-06 Dell Products L.P. In-Field Radio Frequency Impairment Compensation
KR20250036337A (ko) * 2023-09-07 2025-03-14 감바랩스(주) 음성과 음향 인식을 위한 인공 신경망 모델 학습 장치 및 추론 시스템
US12602214B2 (en) * 2023-09-20 2026-04-14 Accenture Global Solutions Limited Dynamic evaluation and improvement of energy efficiency of computer code
CN117009095B (zh) * 2023-10-07 2024-01-02 湘江实验室 一种隐私数据处理模型生成方法、装置、终端设备及介质
CN117408330B (zh) * 2023-12-14 2024-03-15 合肥高维数据技术有限公司 面向非独立同分布数据的联邦知识蒸馏方法及装置
CN117575291B (zh) * 2024-01-15 2024-05-10 湖南科技大学 基于边缘参数熵的联邦学习的数据协同管理方法
KR20260021147A (ko) * 2024-08-05 2026-02-13 삼성전자주식회사 전자 장치 및 그 제어 방법

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017519282A (ja) 2014-05-12 2017-07-13 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ 分散モデル学習
WO2018085643A1 (en) 2016-11-04 2018-05-11 Google Llc Mixture of experts neural networks

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150324690A1 (en) * 2014-05-08 2015-11-12 Microsoft Corporation Deep Learning Training System
US10354184B1 (en) * 2014-06-24 2019-07-16 Amazon Technologies, Inc. Joint modeling of user behavior
US10402469B2 (en) * 2015-10-16 2019-09-03 Google Llc Systems and methods of distributed optimization
CN110268423A (zh) * 2016-08-19 2019-09-20 莫维迪乌斯有限公司 用于深度学习模型的分布式训练的系统和方法
US20180089587A1 (en) * 2016-09-26 2018-03-29 Google Inc. Systems and Methods for Communication Efficient Distributed Mean Estimation
US20180285759A1 (en) * 2017-04-03 2018-10-04 Linkedin Corporation Online hyperparameter tuning in distributed machine learning
US11727301B2 (en) * 2017-07-21 2023-08-15 Sap Se Exploiting local inter-task relationships in adaptive multi-task learning
US11328210B2 (en) * 2017-12-29 2022-05-10 Micron Technology, Inc. Self-learning in distributed architecture for enhancing artificial neural network
US11836576B2 (en) * 2018-04-13 2023-12-05 International Business Machines Corporation Distributed machine learning at edge nodes
US11836643B2 (en) * 2019-03-08 2023-12-05 Nec Corporation System for secure federated learning
EP3798934A1 (en) * 2019-09-27 2021-03-31 Siemens Healthcare GmbH Method and system for scalable and decentralized incremental machine learning which protects data privacy
US12164677B2 (en) * 2022-12-08 2024-12-10 Capital One Services, Llc Methods and systems for federated learning utilizing customer synthetic data models

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017519282A (ja) 2014-05-12 2017-07-13 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ 分散モデル学習
WO2018085643A1 (en) 2016-11-04 2018-05-11 Google Llc Mixture of experts neural networks

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Li Huang et al.，"Patient Clustering Improves Efficiency of Federated Machine Learning to predict mortality and hospital stay time using distributed Electronic Medical Records"，arXiv.org [online]，arXiv:1903.09296v1，米国，Cornell University，2019年03月，[検索日 2024.09.17], インターネット:<URL: https://arxiv.org/abs/1903.09296v1>

Also Published As

Publication number Publication date
JP2023505973A (ja) 2023-02-14
WO2021119601A1 (en) 2021-06-17
BR112022011012A2 (pt) 2022-08-16
US20230036702A1 (en) 2023-02-02
CN114787824A (zh) 2022-07-22
CN114787824B (zh) 2026-04-17
KR20220112766A (ko) 2022-08-11
EP4073714A1 (en) 2022-10-19

Similar Documents

Publication Publication Date Title
JP7635234B2 (ja) 連合混合モデル
JP7706544B2 (ja) スパース性を誘導する連合機械学習
US11132604B2 (en) Nested machine learning architecture
US11144812B2 (en) Mixed machine learning architecture
US20220391778A1 (en) Online Federated Learning of Embeddings
US20190073580A1 (en) Sparse Neural Network Modeling Infrastructure
US11375176B2 (en) Few-shot viewpoint estimation
US20140279741A1 (en) Scalable online hierarchical meta-learning
Liau et al. Stochastic multi-armed bandits in constant space
Xue et al. SUGAR: Efficient subgraph-level training via resource-aware graph partitioning
US20240362468A1 (en) Hybrid generative artificial intelligence models
US11632305B2 (en) Method and system for link prediction in large multiplex networks
US20210264274A1 (en) Secret sharing with a neural cryptosystem
US20240249133A1 (en) Systems, apparatuses, methods, and non-transitory computer-readable storage devices for training artificial-intelligence models using adaptive data-sampling
EP4487261A1 (en) Federated learning with training metadata
JP6900576B2 (ja) 移動状況認識モデル学習装置、移動状況認識装置、方法、及びプログラム
US20250200358A1 (en) Hybrid generative artificial intelligence models
EP4689997A1 (en) Speculative decoding in autoregressive generative artificial intelligence models
Nguyen et al. Performance analysis of data parallelism technique in machine learning for human activity recognition using LSTM
US20260044747A1 (en) Hyperparameter optimization using partitioned machine learning models
US20240346203A1 (en) Method and system for synthetic event series generation
Munir et al. Learning fast and slow: Towards inclusive federated learning
Shen et al. Hierarchical and heterogeneous federated learning via a learning-on-model paradigm
Archambeau et al. Incremental variational inference for latent dirichlet allocation
US20250077624A1 (en) Directed graph generation with diffusion kernels

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231130

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20231130

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20241007

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20241121

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250114

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250212

R150 Certificate of patent or registration of utility model

Ref document number: 7635234

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150