CN111344720A - 融合稀疏内核以近似卷积神经网络的完整内核 - Google Patents

融合稀疏内核以近似卷积神经网络的完整内核 Download PDF

Info

Publication number
CN111344720A
CN111344720A CN201880072812.3A CN201880072812A CN111344720A CN 111344720 A CN111344720 A CN 111344720A CN 201880072812 A CN201880072812 A CN 201880072812A CN 111344720 A CN111344720 A CN 111344720A
Authority
CN
China
Prior art keywords
kernel
complementary
sparse
fusion
computer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201880072812.3A
Other languages
English (en)
Chinese (zh)
Inventor
R·陈
范权福
M·皮斯托亚
铃村丰太郎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of CN111344720A publication Critical patent/CN111344720A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2134—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis
    • G06F18/21345—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on separation criteria, e.g. independent component analysis enforcing sparsity or involving a domain transformation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
  • Complex Calculations (AREA)
CN201880072812.3A 2017-12-14 2018-12-13 融合稀疏内核以近似卷积神经网络的完整内核 Pending CN111344720A (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US15/841,480 US10740659B2 (en) 2017-12-14 2017-12-14 Fusing sparse kernels to approximate a full kernel of a convolutional neural network
US15/841,480 2017-12-14
PCT/IB2018/059993 WO2019116291A1 (en) 2017-12-14 2018-12-13 Fusing sparse kernels to approximate a full kernel of a convolutional neural network

Publications (1)

Publication Number Publication Date
CN111344720A true CN111344720A (zh) 2020-06-26

Family

ID=66814568

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880072812.3A Pending CN111344720A (zh) 2017-12-14 2018-12-13 融合稀疏内核以近似卷积神经网络的完整内核

Country Status (6)

Country Link
US (1) US10740659B2 (enExample)
JP (1) JP7179850B2 (enExample)
CN (1) CN111344720A (enExample)
DE (1) DE112018006377T5 (enExample)
GB (1) GB2583623A (enExample)
WO (1) WO2019116291A1 (enExample)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102548718B1 (ko) * 2017-06-07 2023-06-28 삼성전자주식회사 전자 장치 및 그 제어 방법
US12393829B2 (en) * 2019-07-25 2025-08-19 Samsung Electronics Co., Ltd. Methods and systems with convolutional neural network (CNN) performance
US11144290B2 (en) 2019-09-13 2021-10-12 Huawei Technologies Co., Ltd. Method and apparatus for enabling autonomous acceleration of dataflow AI applications
US12265911B2 (en) * 2020-02-06 2025-04-01 Google Llc Neural network layers with a controlled degree of spatial invariance
US20210256385A1 (en) * 2020-02-14 2021-08-19 Northeastern University Computer-implemented methods and systems for dnn weight pruning for real-time execution on mobile devices
US11379951B2 (en) * 2020-03-25 2022-07-05 Nintendo Co., Ltd. Systems and methods for machine learned image conversion
US11494875B2 (en) 2020-03-25 2022-11-08 Nintendo Co., Ltd. Systems and methods for machine learned image conversion
CN113344199B (zh) * 2021-06-17 2024-05-03 阿波罗智联(北京)科技有限公司 用于训练可分离卷积网络的方法、路侧设备及云控平台
US20230004800A1 (en) * 2021-07-04 2023-01-05 Numenta, Inc. Complementary sparsity in processing tensors
US20250138820A1 (en) * 2023-10-26 2025-05-01 Etched.ai, Inc. Model-specific asic compilation using fused kernel replacement

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077612A (zh) * 2014-07-15 2014-10-01 中国科学院合肥物质科学研究院 一种基于多特征稀疏表示技术的害虫图像识别方法
CN105046193A (zh) * 2015-06-05 2015-11-11 上海大学 一种基于融合稀疏表示矩阵的人体动作识别方法
US20170243345A1 (en) * 2016-02-19 2017-08-24 International Business Machines Corporation Structure-preserving composite model for skin lesion segmentation

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9424513B2 (en) * 2011-11-09 2016-08-23 Qualcomm Incorporated Methods and apparatus for neural component memory transfer of a referenced pattern by including neurons to output a pattern substantially the same as the referenced pattern
US9652817B2 (en) 2015-03-12 2017-05-16 Samsung Electronics Co., Ltd. Automated compute kernel fusion, resizing, and interleave
US9741107B2 (en) * 2015-06-05 2017-08-22 Sony Corporation Full reference image quality assessment based on convolutional neural network
US9972063B2 (en) 2015-07-30 2018-05-15 International Business Machines Corporation Pipelined approach to fused kernels for optimization of machine learning workloads on graphical processing units
US10380479B2 (en) 2015-10-08 2019-08-13 International Business Machines Corporation Acceleration of convolutional neural network training using stochastic perforation
US9904874B2 (en) 2015-11-05 2018-02-27 Microsoft Technology Licensing, Llc Hardware-efficient deep convolutional neural networks
CN108701210B (zh) 2016-02-02 2021-08-17 北京市商汤科技开发有限公司 用于cnn网络适配和对象在线追踪的方法和系统
US10832136B2 (en) 2016-05-18 2020-11-10 Nec Corporation Passive pruning of filters in a convolutional neural network
CN107330463B (zh) 2017-06-29 2020-12-08 南京信息工程大学 基于cnn多特征联合和多核稀疏表示的车型识别方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104077612A (zh) * 2014-07-15 2014-10-01 中国科学院合肥物质科学研究院 一种基于多特征稀疏表示技术的害虫图像识别方法
CN105046193A (zh) * 2015-06-05 2015-11-11 上海大学 一种基于融合稀疏表示矩阵的人体动作识别方法
US20170243345A1 (en) * 2016-02-19 2017-08-24 International Business Machines Corporation Structure-preserving composite model for skin lesion segmentation

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
CHRISTOPH FEICHTENHOFER 等: "Convolutional Two-Stream Network Fusion for Video Action Recognition", ARXIV *
KE-KUN HUANG 等: "Learning Kernel Extended Dictionary for Face Recognition", IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS *
LINGFENG WANG 等: "Visual Tracking Via Kernel Sparse Representation With Multikernel Fusion", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY *
XUN YANG 等: "Enhancing Person Re-identification in a Self-trained Subspace", ARXIV *

Also Published As

Publication number Publication date
US20190188526A1 (en) 2019-06-20
GB2583623A (en) 2020-11-04
WO2019116291A1 (en) 2019-06-20
JP7179850B2 (ja) 2022-11-29
US10740659B2 (en) 2020-08-11
JP2021507345A (ja) 2021-02-22
DE112018006377T5 (de) 2020-08-20
GB202010475D0 (en) 2020-08-19

Similar Documents

Publication Publication Date Title
CN111344720A (zh) 融合稀疏内核以近似卷积神经网络的完整内核
US11836610B2 (en) Concurrent training of functional subnetworks of a neural network
JP7451614B2 (ja) オンチップの計算ネットワーク
CN110503125B (zh) 利用感受野中的运动进行动作检测
US12169782B2 (en) Dynamic precision scaling at epoch granularity in neural networks
US12061990B2 (en) Static block scheduling in massively parallel software defined hardware systems
WO2020113355A1 (en) A content adaptive attention model for neural network-based image and video encoders
TWI515668B (zh) 用於一狀態機中偵測之方法及系統
US20190114533A1 (en) Machine learning runtime library for neural network acceleration
CN112990427A (zh) 域自适应的神经网络实现的装置和方法
JP2022510782A (ja) 機械学習モデルの自動生成
CN114008635B (zh) 神经网络逐层调试
US20200110999A1 (en) Thermodynamic ram technology stack
US11144291B1 (en) Loop-oriented neural network compilation
JP7372011B2 (ja) 深層学習の大規模なモデル・サポート
US20210182670A1 (en) Method and apparatus with training verification of neural network between different frameworks
WO2023185209A1 (zh) 模型剪枝
EP3980943A1 (en) Automatic machine learning policy network for parametric binary neural networks
US20220044107A1 (en) Optimized sensor fusion in deep learning accelerator with integrated random access memory
CN111914989A (zh) 神经网络系统及其学习方法、以及迁移学习方法
JP7507549B2 (ja) ニューロモーフィック装置、及びニューロモーフィック装置でマルチビットニューロモーフィック演算を処理する方法
CN116663640A (zh) 用于剪枝的方法和设备
CN117581245A (zh) 用于机器学习的共享数据的加速处理设备和方法
US11663465B2 (en) Method of managing task performance in an artificial neural network, and system executing an artificial neural network
CN116508027A (zh) 利用振荡神经网络的特征识别

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200626