CN114787824B - 联合混合模型 - Google Patents

联合混合模型

Info

Publication number
CN114787824B
CN114787824B CN202080084734.6A CN202080084734A CN114787824B CN 114787824 B CN114787824 B CN 114787824B CN 202080084734 A CN202080084734 A CN 202080084734A CN 114787824 B CN114787824 B CN 114787824B
Authority
CN
China
Prior art keywords
machine learning
learning model
model
processing device
parameters
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080084734.6A
Other languages
English (en)
Chinese (zh)
Other versions
CN114787824A (zh
Inventor
M·雷瑟
M·威林
E·加维斯
C·路易索斯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Technologies Inc
Original Assignee
Qualcomm Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Technologies Inc filed Critical Qualcomm Technologies Inc
Publication of CN114787824A publication Critical patent/CN114787824A/zh
Application granted granted Critical
Publication of CN114787824B publication Critical patent/CN114787824B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Machine Translation (AREA)
CN202080084734.6A 2019-12-13 2020-12-14 联合混合模型 Active CN114787824B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
GR20190100556 2019-12-13
GR20190100556 2019-12-13
PCT/US2020/064889 WO2021119601A1 (en) 2019-12-13 2020-12-14 Federated mixture models

Publications (2)

Publication Number Publication Date
CN114787824A CN114787824A (zh) 2022-07-22
CN114787824B true CN114787824B (zh) 2026-04-17

Family

ID=74175956

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080084734.6A Active CN114787824B (zh) 2019-12-13 2020-12-14 联合混合模型

Country Status (7)

Country Link
US (1) US20230036702A1 (https=)
EP (1) EP4073714A1 (https=)
JP (1) JP7635234B2 (https=)
KR (1) KR20220112766A (https=)
CN (1) CN114787824B (https=)
BR (1) BR112022011012A2 (https=)
WO (1) WO2021119601A1 (https=)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210312336A1 (en) * 2020-04-03 2021-10-07 International Business Machines Corporation Federated learning of machine learning model features
US11842260B2 (en) * 2020-09-25 2023-12-12 International Business Machines Corporation Incremental and decentralized model pruning in federated machine learning
US11790039B2 (en) * 2020-10-29 2023-10-17 EMC IP Holding Company LLC Compression switching for federated learning
CN113516249B (zh) * 2021-06-18 2023-04-07 重庆大学 基于半异步的联邦学习方法、系统、服务器及介质
US11777812B2 (en) * 2021-06-25 2023-10-03 Qualcomm Technologies, Inc. Zone-based federated learning
CN113435537B (zh) * 2021-07-16 2022-08-26 同盾控股有限公司 基于Soft GBDT的跨特征联邦学习方法、预测方法
US11443245B1 (en) * 2021-07-22 2022-09-13 Alipay Labs (singapore) Pte. Ltd. Method and system for federated adversarial domain adaptation
KR20240058131A (ko) * 2021-08-31 2024-05-03 도쿄엘렉트론가부시키가이샤 정보 처리 방법, 정보 처리 장치, 및 정보 처리 시스템
JP7566707B2 (ja) * 2021-09-15 2024-10-15 株式会社東芝 学習システムおよび方法
US20230110602A1 (en) * 2021-10-13 2023-04-13 International Business Machines Corporation Federated learning model lineage
US20230110975A1 (en) * 2021-10-13 2023-04-13 International Business Machines Corporation Recommending model contributions based on federated learning lineage
US20230117768A1 (en) * 2021-10-15 2023-04-20 Kiarash SHALOUDEGI Methods and systems for updating optimization parameters of a parameterized optimization algorithm in federated learning
CN114004363B (zh) * 2021-10-27 2024-05-31 支付宝(杭州)信息技术有限公司 联合更新模型的方法、装置及系统
EP4238291B1 (en) * 2021-11-16 2025-08-27 Huawei Technologies Co., Ltd. Management entity, network element, and methods for supporting anomaly detection for communication networks
CH719269B1 (de) * 2021-12-17 2024-04-15 Palantir Technologies Inc Verfahren zur Identifizierung eines Zielobjekts und Modell- und Sensororchestrierungssystem
CN116418686A (zh) * 2021-12-31 2023-07-11 华为技术有限公司 模型的数据处理方法及装置
CN114781540B (zh) * 2022-05-09 2025-02-07 国网智能电网研究院有限公司 基于电力物联网的全局模型生成方法、装置、设备及介质
EP4524830A4 (en) * 2022-05-30 2025-07-16 Huawei Tech Co Ltd MODEL TRAINING SYSTEM, MODEL TRAINING METHOD, TRAINING DEVICE, AND TRAINING NODE
EP4296909A1 (de) * 2022-06-22 2023-12-27 Siemens Aktiengesellschaft Individuelle testmodelle für generalisierte maschinelle lernmodelle
KR102573880B1 (ko) * 2022-07-21 2023-09-06 고려대학교 산학협력단 다중-너비 인공신경망에 기반한 연합 학습 시스템 및 연합 학습 방법
KR102684383B1 (ko) * 2022-12-22 2024-07-12 서울과학기술대학교 산학협력단 블록체인 기반 부분 모델 동기화 방법
US20240242505A1 (en) * 2023-01-14 2024-07-18 Radiusai, Inc. Visual analytics
KR20240114646A (ko) * 2023-01-17 2024-07-24 숭실대학교산학협력단 파라미터 부분 공유에 기초한 개인화 연합 학습 방법 및 장치
CN116597672B (zh) * 2023-06-14 2024-02-13 南京云创大数据科技股份有限公司 基于多智能体近端策略优化算法的区域信号灯控制方法
CN116975683B (zh) * 2023-06-27 2025-10-10 中国科学技术大学 一种基于用户分簇和模型分层的个性化联邦学习方法
US20250077948A1 (en) * 2023-08-29 2025-03-06 Dell Products L.P. In-Field Radio Frequency Impairment Compensation
KR20250036337A (ko) * 2023-09-07 2025-03-14 감바랩스(주) 음성과 음향 인식을 위한 인공 신경망 모델 학습 장치 및 추론 시스템
US12602214B2 (en) * 2023-09-20 2026-04-14 Accenture Global Solutions Limited Dynamic evaluation and improvement of energy efficiency of computer code
CN117009095B (zh) * 2023-10-07 2024-01-02 湘江实验室 一种隐私数据处理模型生成方法、装置、终端设备及介质
CN117408330B (zh) * 2023-12-14 2024-03-15 合肥高维数据技术有限公司 面向非独立同分布数据的联邦知识蒸馏方法及装置
CN117575291B (zh) * 2024-01-15 2024-05-10 湖南科技大学 基于边缘参数熵的联邦学习的数据协同管理方法
KR20260021147A (ko) * 2024-08-05 2026-02-13 삼성전자주식회사 전자 장치 및 그 제어 방법

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110268423A (zh) * 2016-08-19 2019-09-20 莫维迪乌斯有限公司 用于深度学习模型的分布式训练的系统和方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150324690A1 (en) * 2014-05-08 2015-11-12 Microsoft Corporation Deep Learning Training System
US20150324686A1 (en) * 2014-05-12 2015-11-12 Qualcomm Incorporated Distributed model learning
US10354184B1 (en) * 2014-06-24 2019-07-16 Amazon Technologies, Inc. Joint modeling of user behavior
US10402469B2 (en) * 2015-10-16 2019-09-03 Google Llc Systems and methods of distributed optimization
US20180089587A1 (en) * 2016-09-26 2018-03-29 Google Inc. Systems and Methods for Communication Efficient Distributed Mean Estimation
WO2018085643A1 (en) * 2016-11-04 2018-05-11 Google Llc Mixture of experts neural networks
US20180285759A1 (en) * 2017-04-03 2018-10-04 Linkedin Corporation Online hyperparameter tuning in distributed machine learning
US11727301B2 (en) * 2017-07-21 2023-08-15 Sap Se Exploiting local inter-task relationships in adaptive multi-task learning
US11328210B2 (en) * 2017-12-29 2022-05-10 Micron Technology, Inc. Self-learning in distributed architecture for enhancing artificial neural network
US11836576B2 (en) * 2018-04-13 2023-12-05 International Business Machines Corporation Distributed machine learning at edge nodes
US11836643B2 (en) * 2019-03-08 2023-12-05 Nec Corporation System for secure federated learning
EP3798934A1 (en) * 2019-09-27 2021-03-31 Siemens Healthcare GmbH Method and system for scalable and decentralized incremental machine learning which protects data privacy
US12164677B2 (en) * 2022-12-08 2024-12-10 Capital One Services, Llc Methods and systems for federated learning utilizing customer synthetic data models

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110268423A (zh) * 2016-08-19 2019-09-20 莫维迪乌斯有限公司 用于深度学习模型的分布式训练的系统和方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Sjöberg, A ; Gustavsson, E ; Koppisetty, AC ; Jirstrand, M.Federated Learning of Deep Neural Decision Forests.MACHINE LEARNING, OPTIMIZATION, AND DATA SCIENCE.2019,第11943卷第3-8页. *

Also Published As

Publication number Publication date
JP2023505973A (ja) 2023-02-14
WO2021119601A1 (en) 2021-06-17
BR112022011012A2 (pt) 2022-08-16
US20230036702A1 (en) 2023-02-02
CN114787824A (zh) 2022-07-22
KR20220112766A (ko) 2022-08-11
EP4073714A1 (en) 2022-10-19
JP7635234B2 (ja) 2025-02-25

Similar Documents

Publication Publication Date Title
CN114787824B (zh) 联合混合模型
US11941527B2 (en) Population based training of neural networks
US12229280B2 (en) Privacy preserving cooperative learning in untrusted environments
US20240135191A1 (en) Method, apparatus, and system for generating neural network model, device, medium, and program product
US12518168B2 (en) Training and application method apparatus system and storage medium of neural network model
CN115552429A (zh) 使用非iid数据的横向联邦学习方法和系统
US20230169350A1 (en) Sparsity-inducing federated machine learning
US20210326757A1 (en) Federated Learning with Only Positive Labels
US20230316090A1 (en) Federated learning with training metadata
Xue et al. SUGAR: Efficient subgraph-level training via resource-aware graph partitioning
TWI900601B (zh) 用於硬體加速器之神經架構縮放之電腦實施方法、系統及非暫時性電腦可讀儲存媒體
KR20240158174A (ko) 연합 학습을 위한 기계 학습 모델의 파라미터 튜닝 시스템 및 방법
US20220108194A1 (en) Private split client-server inferencing
Yadav et al. Edge computing in centralized data server deployment for network QoS and latency improvement for virtualization environment
CN119895444A (zh) 用于联邦学习的模型去相关和子空间划分
EP4689997A1 (en) Speculative decoding in autoregressive generative artificial intelligence models
US12524653B2 (en) Method and system for efficient learning on large multiplex networks
CN117015784B (zh) 用于使用广播式残差学习处理音频特征的方法和装置
CN119907981A (zh) 使用分区机器学习模型的超参数优化
Berta et al. Dimension reduction methods for collaborative mobile gossip learning
US20250190865A1 (en) Decentralized federated learning using a random walk over a communication graph
Li et al. FNCS: Federated learning strategy based on cosine similarity under resource constraints
WO2025189371A1 (en) Multiple token generation in autoregressive generative artificial intelligence models
CN118862972A (zh) 用于处理自然语言处理任务的方法、计算装置和设备
JP2025510498A (ja) 混合エキスパートとモデルアンサンブリングとを組み合わせた機械学習モデルアーキテクチャ

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant