CA3168515A1 - Systeme et methode d'entrainement de bas niveau de reseaux neuronaux - Google Patents

Systeme et methode d'entrainement de bas niveau de reseaux neuronaux

Info

Publication number
CA3168515A1
CA3168515A1 CA3168515A CA3168515A CA3168515A1 CA 3168515 A1 CA3168515 A1 CA 3168515A1 CA 3168515 A CA3168515 A CA 3168515A CA 3168515 A CA3168515 A CA 3168515A CA 3168515 A1 CA3168515 A1 CA 3168515A1
Authority
CA
Canada
Prior art keywords
neural network
network model
training
nodes
low rank
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CA3168515A
Other languages
English (en)
Inventor
Siddhartha Rao Kamalakara
Bharat Venkitesh
Aidan N. Gomez
Acyr Flavio Neto Locatelli
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Cohere Inc
Original Assignee
Cohere Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cohere Inc filed Critical Cohere Inc
Publication of CA3168515A1 publication Critical patent/CA3168515A1/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Feedback Control In General (AREA)
CA3168515A 2021-07-23 2022-07-21 Systeme et methode d'entrainement de bas niveau de reseaux neuronaux Pending CA3168515A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202163203454P 2021-07-23 2021-07-23
US63/203,454 2021-07-23

Publications (1)

Publication Number Publication Date
CA3168515A1 true CA3168515A1 (fr) 2023-01-23

Family

ID=84540531

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3168515A Pending CA3168515A1 (fr) 2021-07-23 2022-07-21 Systeme et methode d'entrainement de bas niveau de reseaux neuronaux

Country Status (3)

Country Link
US (1) US20230057387A1 (fr)
CA (1) CA3168515A1 (fr)
GB (1) GB2614112A (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117318037B (zh) * 2023-09-27 2024-09-06 国网湖北省电力有限公司电力科学研究院 一种含规模分布式新能源接入的配电网状态分析方法
CN117372702B (zh) * 2023-12-08 2024-02-06 江西师范大学 自监督深度学习与模型方法相结合的云层去除方法及装置
CN118520975A (zh) * 2024-07-22 2024-08-20 智慧眼科技股份有限公司 一种大语言模型训练方法、装置、电子设备及存储介质
CN118569324A (zh) * 2024-08-01 2024-08-30 浪潮软件科技有限公司 一种大语言模型加速方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9262724B2 (en) * 2012-07-13 2016-02-16 International Business Machines Corporation Low-rank matrix factorization for deep belief network training with high-dimensional output targets
US9400955B2 (en) * 2013-12-13 2016-07-26 Amazon Technologies, Inc. Reducing dynamic range of low-rank decomposition matrices
US10360497B2 (en) * 2014-07-16 2019-07-23 Qualcomm Incorporated Decomposing convolution operation in neural networks
JP6706326B2 (ja) * 2016-02-03 2020-06-03 グーグル エルエルシー リカレントニューラルネットワークモデルの圧縮
WO2020190772A1 (fr) * 2019-03-15 2020-09-24 Futurewei Technologies, Inc. Compression et optimisation de modèle de réseau de neurones artificiels

Also Published As

Publication number Publication date
US20230057387A1 (en) 2023-02-23
GB2614112A (en) 2023-06-28
GB202210740D0 (en) 2022-09-07

Similar Documents

Publication Publication Date Title
CA3168515A1 (fr) Systeme et methode d'entrainement de bas niveau de reseaux neuronaux
Hubara et al. Accelerated sparse neural training: A provable and efficient method to find n: m transposable masks
Swaminathan et al. Sparse low rank factorization for deep neural network compression
Lu et al. Learning compact recurrent neural networks
Zhang et al. Platon: Pruning large transformer models with upper confidence bound of weight importance
Mueller et al. Siamese recurrent architectures for learning sentence similarity
US8700552B2 (en) Exploiting sparseness in training deep neural networks
Rae et al. Fast parametric learning with activation memorization
Pandey et al. Attention gated tensor neural network architectures for speech emotion recognition
CN111723914A (zh) 一种基于卷积核预测的神经网络架构搜索方法
Jakkala Deep Gaussian processes: A survey
Liang et al. Homodistil: Homotopic task-agnostic distillation of pre-trained transformers
Kamalakara et al. Exploring low rank training of deep neural networks
Santacroce et al. What matters in the structured pruning of generative language models?
Berman et al. Multifactor sequential disentanglement via structured koopman autoencoders
CN117951274A (zh) 一种基于融合向量和关键词检索的rag知识问答方法和装置
Hu et al. One pass imagenet
May Kernel approximation methods for speech recognition
Zhang et al. Pruned-yolo: Learning efficient object detector using model pruning
Ahn et al. Multi-Corpus Speech Emotion Recognition for Unseen Corpus Using Corpus-Wise Weights in Classification Loss.
Chen et al. Survey: Exploiting data redundancy for optimization of deep learning
Upreti Convolutional neural network (cnn). a comprehensive overview
Park et al. An effective 3D text recurrent voting generator for metaverse
Zheng et al. A novel and efficient model pruning method for deep convolutional neural networks by evaluating the direct and indirect effects of filters
Hutchinson et al. A sparse plus low-rank exponential language model for limited resource scenarios