CA3195441A1 - Systemes et procedes de fourniture fonction de perte modifiee dans un apprentissage federe/divise - Google Patents
Systemes et procedes de fourniture fonction de perte modifiee dans un apprentissage federe/diviseInfo
- Publication number
- CA3195441A1 CA3195441A1 CA3195441A CA3195441A CA3195441A1 CA 3195441 A1 CA3195441 A1 CA 3195441A1 CA 3195441 A CA3195441 A CA 3195441A CA 3195441 A CA3195441 A CA 3195441A CA 3195441 A1 CA3195441 A1 CA 3195441A1
- Authority
- CA
- Canada
- Prior art keywords
- client
- server
- client system
- model
- server system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006870 function Effects 0.000 title claims abstract description 164
- 238000000034 method Methods 0.000 title claims abstract description 151
- 238000012549 training Methods 0.000 claims abstract description 56
- 238000013135 deep learning Methods 0.000 claims abstract description 35
- 230000001902 propagating effect Effects 0.000 claims abstract description 23
- 238000012935 Averaging Methods 0.000 claims description 64
- 230000004913 activation Effects 0.000 claims description 14
- 238000001994 activation Methods 0.000 claims description 14
- 238000009826 distribution Methods 0.000 claims description 10
- 230000000873 masking effect Effects 0.000 claims description 3
- 239000010410 layer Substances 0.000 description 196
- 230000008569 process Effects 0.000 description 55
- 238000013459 approach Methods 0.000 description 38
- 238000004422 calculation algorithm Methods 0.000 description 37
- 238000013528 artificial neural network Methods 0.000 description 24
- 238000012545 processing Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 11
- 230000008901 benefit Effects 0.000 description 9
- 230000015654 memory Effects 0.000 description 9
- 239000000203 mixture Substances 0.000 description 6
- 239000013598 vector Substances 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 239000003086 colorant Substances 0.000 description 4
- 230000000977 initiatory effect Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 235000009499 Vanilla fragrans Nutrition 0.000 description 2
- 244000263375 Vanilla tahitensis Species 0.000 description 2
- 235000012036 Vanilla tahitensis Nutrition 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003058 natural language processing Methods 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 239000000243 solution Substances 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 244000035744 Hura crepitans Species 0.000 description 1
- 235000010678 Paulownia tomentosa Nutrition 0.000 description 1
- 240000002834 Paulownia tomentosa Species 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011985 exploratory data analysis Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000003999 initiator Substances 0.000 description 1
- 238000002347 injection Methods 0.000 description 1
- 239000007924 injection Substances 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- APTZNLHMIGJTEW-UHFFFAOYSA-N pyraflufen-ethyl Chemical compound C1=C(Cl)C(OCC(=O)OCC)=CC(C=2C(=C(OC(F)F)N(C)N=2)Cl)=C1F APTZNLHMIGJTEW-UHFFFAOYSA-N 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000002356 single layer Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Storage Device Security (AREA)
Abstract
L'invention concerne un procédé qui comprend l'entraînement, au niveau d'un client, d'une partie d'un réseau d'apprentissage profond jusqu'à une couche divisée du client. En fonction d'une sortie de la couche divisée, le procédé comprend la réalisation, au niveau d'un serveur, de l'entraînement du réseau d'apprentissage profond par la propagation directe de la sortie reçue au niveau d'une couche divisée du serveur vers une dernière couche du serveur. Le serveur calcule une fonction de perte pondérée pour le client au niveau de la dernière couche et stocke la fonction de perte calculée. Après le stockage de la fonction de perte respective de chaque client respectif d'une pluralité de clients, le serveur calcule la moyenne de la pluralité de fonctions de perte de client pondérées respectives et effectue la rétropropagation des gradients en fonction de la valeur de perte moyenne de la dernière couche du serveur dans la couche divisée du serveur et transmet uniquement les gradients de couche divisée de serveur aux clients respectifs.
Applications Claiming Priority (7)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202063090904P | 2020-10-13 | 2020-10-13 | |
| US63/090,904 | 2020-10-13 | ||
| US202163226135P | 2021-07-27 | 2021-07-27 | |
| US63/226,135 | 2021-07-27 | ||
| US17/499,153 US11431688B2 (en) | 2019-12-13 | 2021-10-12 | Systems and methods for providing a modified loss function in federated-split learning |
| PCT/US2021/054518 WO2022081539A1 (fr) | 2020-10-13 | 2021-10-12 | Systèmes et procédés de fourniture fonction de perte modifiée dans un apprentissage fédéré/divisé |
| US17/499,153 | 2021-10-12 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA3195441A1 true CA3195441A1 (fr) | 2022-04-21 |
Family
ID=81208549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3195441A Pending CA3195441A1 (fr) | 2020-10-13 | 2021-10-12 | Systemes et procedes de fourniture fonction de perte modifiee dans un apprentissage federe/divise |
Country Status (2)
| Country | Link |
|---|---|
| CA (1) | CA3195441A1 (fr) |
| WO (1) | WO2022081539A1 (fr) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115545172B (zh) * | 2022-11-29 | 2023-02-07 | 支付宝(杭州)信息技术有限公司 | 兼顾隐私保护和公平性的图神经网络的训练方法及装置 |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| USRE22222E (en) | 1934-07-20 | 1942-11-10 | Follo web machine | |
| US2130101A (en) | 1934-09-07 | 1938-09-13 | Ritzerfeld Wilhelm | Apparatus for feeding envelopes and method relating thereto |
| US2130103A (en) | 1935-04-10 | 1938-09-13 | Eagle Picher Lead Company | Storage battery plate and composition therefor |
| US2130100A (en) | 1937-09-30 | 1938-09-13 | Reuben G Doege | Trailer hitch |
| US9984337B2 (en) * | 2014-10-08 | 2018-05-29 | Nec Corporation | Parallelized machine learning with distributed lockless training |
| US10755172B2 (en) | 2016-06-22 | 2020-08-25 | Massachusetts Institute Of Technology | Secure training of multi-party deep neural network |
| US10924460B2 (en) | 2019-12-13 | 2021-02-16 | TripleBlind, Inc. | Systems and methods for dividing filters in neural networks for private data computations |
-
2021
- 2021-10-12 CA CA3195441A patent/CA3195441A1/fr active Pending
- 2021-10-12 WO PCT/US2021/054518 patent/WO2022081539A1/fr unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022081539A1 (fr) | 2022-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11431688B2 (en) | Systems and methods for providing a modified loss function in federated-split learning | |
| US12278806B2 (en) | Operating system for blockchain IoT devices | |
| TWI770022B (zh) | 電腦實施之控制方法、系統及控制系統 | |
| US12277548B2 (en) | System and method of multi-round token distribution using a blockchain network | |
| US11991156B2 (en) | Systems and methods for secure averaging of models for federated learning and blind learning using secure multi-party computation | |
| AU2019389028A1 (en) | Multi-hop security amplification | |
| Hamza et al. | A social qualitative trust framework for Fog computing | |
| CA3195441A1 (fr) | Systemes et procedes de fourniture fonction de perte modifiee dans un apprentissage federe/divise | |
| CN116167868A (zh) | 基于隐私计算的风险识别方法、装置、设备以及存储介质 | |
| EP3379408B1 (fr) | Fonctions aléatoires actualisables | |
| CN119046955A (zh) | 一种图联邦训练方法、装置、设备、存储介质及产品 | |
| EP4229559A1 (fr) | Systèmes et procédés de fourniture fonction de perte modifiée dans un apprentissage fédéré/divisé | |
| Piotrowski et al. | Towards a secure peer-to-peer federated learning framework | |
| Rahmani et al. | Secure two-party computation via measurement-based quantum computing | |
| US20250182861A1 (en) | Computation system and computation method | |
| Barbosa | Secure two-party computation via measurement-based quantum computing | |
| Hong et al. | A designated private set based trapdoor authentication scheme for privacy preserving trust management in decentralized systems | |
| Kirubakaran et al. | Enhanced VANET Communication: Fractional Order Water Flow Optimization and Secure Communication via Spatial Bayesian Neural Network | |
| Hota et al. | Advanced federated learning security: NTRU and blockchain synergy | |
| HK40017117B (en) | System and method of multi-round token distribution using a blockchain network | |
| HK40017117A (en) | System and method of multi-round token distribution using a blockchain network |