JP2017525038A - ニューラルネットワークにおける畳込み演算の分解 - Google Patents

ニューラルネットワークにおける畳込み演算の分解 Download PDF

Info

Publication number
JP2017525038A
JP2017525038A JP2017502110A JP2017502110A JP2017525038A JP 2017525038 A JP2017525038 A JP 2017525038A JP 2017502110 A JP2017502110 A JP 2017502110A JP 2017502110 A JP2017502110 A JP 2017502110A JP 2017525038 A JP2017525038 A JP 2017525038A
Authority
JP
Japan
Prior art keywords
filter
neural network
low rank
regularization term
processor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2017502110A
Other languages
English (en)
Japanese (ja)
Other versions
JP2017525038A5 (enExample
Inventor
アンナプレディー、ベンカタ・スリーカンタ・レッディ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of JP2017525038A publication Critical patent/JP2017525038A/ja
Publication of JP2017525038A5 publication Critical patent/JP2017525038A5/ja
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Image Analysis (AREA)
  • Electrically Operated Instructional Devices (AREA)
JP2017502110A 2014-07-16 2015-07-13 ニューラルネットワークにおける畳込み演算の分解 Pending JP2017525038A (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201462025406P 2014-07-16 2014-07-16
US62/025,406 2014-07-16
US14/526,046 US10402720B2 (en) 2014-07-16 2014-10-28 Decomposing convolution operation in neural networks
US14/526,046 2014-10-28
PCT/US2015/040221 WO2016010930A1 (en) 2014-07-16 2015-07-13 Decomposing convolution operation in neural networks

Publications (2)

Publication Number Publication Date
JP2017525038A true JP2017525038A (ja) 2017-08-31
JP2017525038A5 JP2017525038A5 (enExample) 2018-07-26

Family

ID=55074837

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2017502110A Pending JP2017525038A (ja) 2014-07-16 2015-07-13 ニューラルネットワークにおける畳込み演算の分解

Country Status (8)

Country Link
US (2) US10402720B2 (enExample)
EP (2) EP3170127A1 (enExample)
JP (1) JP2017525038A (enExample)
KR (1) KR20170031695A (enExample)
CN (2) CN106537421A (enExample)
AU (1) AU2015289877A1 (enExample)
BR (1) BR112017000229A2 (enExample)
WO (2) WO2016010922A1 (enExample)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019125351A (ja) * 2017-11-28 2019-07-25 南京地平綫机器人技術有限公司 畳み込みニューラルネットワークの畳み込み層における演算を実行する方法及び装置
WO2019146189A1 (ja) * 2018-01-29 2019-08-01 日本電気株式会社 ニューラルネットワークのランク最適化装置および最適化方法
JP2019212121A (ja) * 2018-06-06 2019-12-12 株式会社Ｎｔｔドコモ 機械学習システム
WO2020235011A1 (ja) * 2019-05-21 2020-11-26 日本電信電話株式会社 学習装置、学習方法及び学習プログラム
JP7041244B1 (ja) 2020-09-18 2022-03-23 ネイバー コーポレーション 表現的ボトルネック現象が最小化された人工ニューラルネットワークに基づく電子装置およびその動作方法
JP2023083997A (ja) * 2021-12-06 2023-06-16 株式会社デンソー モデル生成方法、モデル生成プログラム、モデル生成装置、データ処理装置

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10402720B2 (en) 2014-07-16 2019-09-03 Qualcomm Incorporated Decomposing convolution operation in neural networks
US10262259B2 (en) * 2015-05-08 2019-04-16 Qualcomm Incorporated Bit width selection for fixed point neural networks
KR102565273B1 (ko) * 2016-01-26 2023-08-09 삼성전자주식회사 뉴럴 네트워크에 기초한 인식 장치 및 뉴럴 네트워크의 학습 방법
US10713562B2 (en) * 2016-06-18 2020-07-14 International Business Machines Corporation Neuromorphic memory circuit
CN106326985A (zh) * 2016-08-18 2017-01-11 北京旷视科技有限公司 神经网络训练方法和装置及数据处理方法和装置
US11238337B2 (en) * 2016-08-22 2022-02-01 Applied Brain Research Inc. Methods and systems for implementing dynamic neural networks
EP3306535B1 (en) * 2016-10-10 2019-12-04 Alcatel Lucent Runtime optimization of convolutional neural networks
KR102879261B1 (ko) * 2016-12-22 2025-10-31 삼성전자주식회사 컨볼루션 신경망 처리 방법 및 장치
US11301750B2 (en) 2017-03-31 2022-04-12 Ecole Polytechnique Federale De Lausanne (Epfl) Simplification of neural models that include arborized projections
DE102017205713A1 (de) * 2017-04-04 2018-10-04 Siemens Aktiengesellschaft Verfahren und Steuereinrichtung zum Steuern eines technischen Systems
US11037330B2 (en) 2017-04-08 2021-06-15 Intel Corporation Low rank matrix compression
CN107248144B (zh) * 2017-04-27 2019-12-10 东南大学 一种基于压缩型卷积神经网络的图像去噪方法
US11093832B2 (en) 2017-10-19 2021-08-17 International Business Machines Corporation Pruning redundant neurons and kernels of deep convolutional neural networks
US11238346B2 (en) * 2018-04-25 2022-02-01 Qualcomm Incorproated Learning a truncation rank of singular value decomposed matrices representing weight tensors in neural networks
US11922314B1 (en) * 2018-11-30 2024-03-05 Ansys, Inc. Systems and methods for building dynamic reduced order physical models
CN109948787B (zh) * 2019-02-26 2021-01-08 山东师范大学 用于神经网络卷积层的运算装置、芯片及方法
US11580399B2 (en) 2019-04-30 2023-02-14 Samsung Electronics Co., Ltd. System and method for convolutional layer structure for neural networks
KR102774162B1 (ko) * 2019-05-16 2025-03-04 삼성전자주식회사 전자 장치 및 이의 제어 방법
CN112215329B (zh) * 2019-07-09 2023-09-29 杭州海康威视数字技术股份有限公司 基于神经网络的卷积计算方法及装置
CN112784207B (zh) * 2019-11-01 2024-02-02 中科寒武纪科技股份有限公司 运算方法及相关产品
US11010691B1 (en) * 2020-03-16 2021-05-18 Sas Institute Inc. Distributable event prediction and machine learning recognition system
US12450472B2 (en) * 2020-06-22 2025-10-21 Qualcomm Incorporated Charge-pump-based current-mode neuron for machine learning
US20230057387A1 (en) * 2021-07-23 2023-02-23 Cohere Inc. System and Method for Low Rank Training of Neural Networks

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5781700A (en) 1996-02-05 1998-07-14 Ford Global Technologies, Inc. Trained Neural network air/fuel control system
US6351740B1 (en) 1997-12-01 2002-02-26 The Board Of Trustees Of The Leland Stanford Junior University Method and system for training dynamic nonlinear adaptive filters which have embedded memory
US6269351B1 (en) 1999-03-31 2001-07-31 Dryken Technologies, Inc. Method and system for training an artificial neural network
US6754380B1 (en) * 2003-02-14 2004-06-22 The University Of Chicago Method of training massive training artificial neural networks (MTANN) for the detection of abnormalities in medical images
US7702502B2 (en) 2005-02-23 2010-04-20 Digital Intelligence, L.L.C. Apparatus for signal decomposition, analysis and reconstruction
ITRM20050192A1 (it) 2005-04-20 2006-10-21 Consiglio Nazionale Ricerche Sistema per la rilevazione e la classificazione di eventi durante azioni in movimento.
US7945061B1 (en) 2006-06-07 2011-05-17 Bae Systems Information And Electronic Systems Integration Inc. Scalable architecture for subspace signal tracking
WO2010003041A2 (en) 2008-07-03 2010-01-07 Nec Laboratories America, Inc. Mitotic figure detector and counter system and method for detecting and counting mitotic figures
CN101667425A (zh) 2009-09-22 2010-03-10 山东大学 一种对卷积混叠语音信号进行盲源分离的方法
BRPI0904540B1 (pt) 2009-11-27 2021-01-26 Samsung Eletrônica Da Amazônia Ltda método para animar rostos/cabeças/personagens virtuais via processamento de voz
US8874432B2 (en) 2010-04-28 2014-10-28 Nec Laboratories America, Inc. Systems and methods for semi-supervised relationship extraction
US8583586B2 (en) 2011-01-21 2013-11-12 International Business Machines Corporation Mining temporal patterns in longitudinal event data using discrete event matrices and sparse coding
US9262724B2 (en) 2012-07-13 2016-02-16 International Business Machines Corporation Low-rank matrix factorization for deep belief network training with high-dimensional output targets
CN102820653B (zh) 2012-09-12 2014-07-30 湖南大学 一种电能质量综合控制器模糊-神经网络双闭环控制方法
US20140156575A1 (en) 2012-11-30 2014-06-05 Nuance Communications, Inc. Method and Apparatus of Processing Data Using Deep Belief Networks Employing Low-Rank Matrix Factorization
CN103325382A (zh) 2013-06-07 2013-09-25 大连民族学院 一种自动识别中国少数民族传统乐器音频数据的方法
US9728184B2 (en) 2013-06-18 2017-08-08 Microsoft Technology Licensing, Llc Restructuring deep neural network acoustic models
US9400955B2 (en) 2013-12-13 2016-07-26 Amazon Technologies, Inc. Reducing dynamic range of low-rank decomposition matrices
US10402720B2 (en) 2014-07-16 2019-09-03 Qualcomm Incorporated Decomposing convolution operation in neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ROBERTO RIGAMONTI，外３名: "Learning Separable Filters", ［ONLINE］, JPN6019026668, 3 October 2013 (2013-10-03), pages 2754 - 2761, ISSN: 0004223751 *
浜田宏一，外２名: "超解像自由視点画像合成における超解像演算の高速化実装", 電子情報通信学会技術研究報告 ＩＥ２０１１−７７−ＩＥ２０１１−８７ 画像工学, vol. 第111巻，第284号, JPN6019026669, 4 November 2011 (2011-11-04), JP, pages 1 - 6, ISSN: 0004223752 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019125351A (ja) * 2017-11-28 2019-07-25 南京地平綫机器人技術有限公司 畳み込みニューラルネットワークの畳み込み層における演算を実行する方法及び装置
US11822616B2 (en) 2017-11-28 2023-11-21 Nanjing Horizon Robotics Technology Co., Ltd. Method and apparatus for performing operation of convolutional layers in convolutional neural network
WO2019146189A1 (ja) * 2018-01-29 2019-08-01 日本電気株式会社 ニューラルネットワークのランク最適化装置および最適化方法
JPWO2019146189A1 (ja) * 2018-01-29 2020-12-17 日本電気株式会社 ニューラルネットワークのランク最適化装置および最適化方法
US12165054B2 (en) 2018-01-29 2024-12-10 Nec Corporation Neural network rank optimization device and optimization method
JP2019212121A (ja) * 2018-06-06 2019-12-12 株式会社Ｎｔｔドコモ 機械学習システム
JP7021010B2 (ja) 2018-06-06 2022-02-16 株式会社Ｎｔｔドコモ 機械学習システム
WO2020235011A1 (ja) * 2019-05-21 2020-11-26 日本電信電話株式会社 学習装置、学習方法及び学習プログラム
JP7041244B1 (ja) 2020-09-18 2022-03-23 ネイバー コーポレーション 表現的ボトルネック現象が最小化された人工ニューラルネットワークに基づく電子装置およびその動作方法
JP2022051484A (ja) * 2020-09-18 2022-03-31 ネイバー コーポレーション 表現的ボトルネック現象が最小化された人工ニューラルネットワークに基づく電子装置およびその動作方法
JP2023083997A (ja) * 2021-12-06 2023-06-16 株式会社デンソー モデル生成方法、モデル生成プログラム、モデル生成装置、データ処理装置
JP7600972B2 (ja) 2021-12-06 2024-12-17 株式会社デンソー モデル生成方法、モデル生成プログラム、モデル生成装置、データ処理装置

Also Published As

Publication number Publication date
EP3170126A1 (en) 2017-05-24
KR20170031695A (ko) 2017-03-21
US10360497B2 (en) 2019-07-23
AU2015289877A1 (en) 2017-01-05
WO2016010930A1 (en) 2016-01-21
BR112017000229A2 (pt) 2017-10-31
US20160019455A1 (en) 2016-01-21
WO2016010922A1 (en) 2016-01-21
US20160019456A1 (en) 2016-01-21
CN106663222A (zh) 2017-05-10
US10402720B2 (en) 2019-09-03
CN106537421A (zh) 2017-03-22
EP3170127A1 (en) 2017-05-24

Similar Documents

Publication Publication Date Title
JP2017525038A (ja) ニューラルネットワークにおける畳込み演算の分解
US10339447B2 (en) Configuring sparse neuronal networks
CN107077637B (zh) 神经网络中的差分编码
US9558442B2 (en) Monitoring neural networks with shadow networks
US9330355B2 (en) Computed synapses for neuromorphic systems
JP6092477B2 (ja) ニューラルダイナミクスを修正するための自動化された方法
US20150242745A1 (en) Event-based inference and learning for stochastic spiking bayesian networks
JP2017513127A (ja) スパイキング深層信念ネットワーク（ｄｂｎ）におけるトレーニング、認識、および生成
US20150212861A1 (en) Value synchronization across neural processors
JP2017515205A (ja) Ｃｏｌｄニューロンスパイクタイミングバックプロバゲーション
JP2017519268A (ja) スパイキングニューラルネットワークにおけるグローバルスカラ値によって可塑性を調節すること
JP6219509B2 (ja) シナプス遅延を動的に割り当てることおおよび検査すること
US9342782B2 (en) Stochastic delay plasticity
WO2015127124A2 (en) Imbalanced cross-inhibitory mechanism for spatial target selection

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20170329

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20170328

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20180618

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20180618

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20190606

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20190716

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20200303