TWI698807B - 以類別為基礎修剪之人工類神經網路 - Google Patents

以類別為基礎修剪之人工類神經網路 Download PDF

Info

Publication number
TWI698807B
TWI698807B TW106136613A TW106136613A TWI698807B TW I698807 B TWI698807 B TW I698807B TW 106136613 A TW106136613 A TW 106136613A TW 106136613 A TW106136613 A TW 106136613A TW I698807 B TWI698807 B TW I698807B
Authority
TW
Taiwan
Prior art keywords
artificial neural
neural network
neurons
activation frequency
categories
Prior art date
Application number
TW106136613A
Other languages
English (en)
Chinese (zh)
Other versions
TW201824093A (zh
Inventor
羅賓 賽柏德
陳建單
漢納 畢卓文多堤爾
馬丁 樂容維斯特
Original Assignee
瑞典商安訊士有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 瑞典商安訊士有限公司 filed Critical 瑞典商安訊士有限公司
Publication of TW201824093A publication Critical patent/TW201824093A/zh
Application granted granted Critical
Publication of TWI698807B publication Critical patent/TWI698807B/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/40—Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/061—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using biological neurons, e.g. biological neurons connected to an integrated circuit
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Neurology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
TW106136613A 2016-12-21 2017-10-25 以類別為基礎修剪之人工類神經網路 TWI698807B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP16205831.7 2016-12-21
??16205831.7 2016-12-21
EP16205831.7A EP3340129B1 (en) 2016-12-21 2016-12-21 Artificial neural network class-based pruning

Publications (2)

Publication Number Publication Date
TW201824093A TW201824093A (zh) 2018-07-01
TWI698807B true TWI698807B (zh) 2020-07-11

Family

ID=57583087

Family Applications (1)

Application Number Title Priority Date Filing Date
TW106136613A TWI698807B (zh) 2016-12-21 2017-10-25 以類別為基礎修剪之人工類神經網路

Country Status (6)

Country Link
US (1) US10552737B2 (enExample)
EP (1) EP3340129B1 (enExample)
JP (1) JP6755849B2 (enExample)
KR (1) KR102110486B1 (enExample)
CN (1) CN108229667B (enExample)
TW (1) TWI698807B (enExample)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108268947A (zh) * 2016-12-30 2018-07-10 富士通株式会社 用于提高神经网络的处理速度的装置和方法及其应用
US10892050B2 (en) * 2018-04-13 2021-01-12 International Business Machines Corporation Deep image classification of medical images
KR101989793B1 (ko) 2018-10-22 2019-06-17 한국기술교육대학교 산학협력단 컨볼루션 신경망을 위한 가속기 인식 가지 치기 방법 및 기록 매체
KR102796861B1 (ko) 2018-12-10 2025-04-17 삼성전자주식회사 인공 신경망을 압축하기 위한 장치 및 방법
KR102865734B1 (ko) * 2019-01-02 2025-09-26 삼성전자주식회사 뉴럴 네트워크 최적화 장치 및 뉴럴 네트워크 최적화 방법
US11551093B2 (en) * 2019-01-22 2023-01-10 Adobe Inc. Resource-aware training for neural networks
KR102214837B1 (ko) * 2019-01-29 2021-02-10 주식회사 디퍼아이 컨벌루션 신경망 파라미터 최적화 방법, 컨벌루션 신경망 연산방법 및 그 장치
JP7099968B2 (ja) * 2019-01-31 2022-07-12 日立Astemo株式会社 演算装置
JP7225876B2 (ja) 2019-02-08 2023-02-21 富士通株式会社 情報処理装置、演算処理装置および情報処理装置の制御方法
DE102019202816A1 (de) * 2019-03-01 2020-09-03 Robert Bosch Gmbh Training neuronaler Netzwerke für effizientes Implementieren auf Hardware
JP7150651B2 (ja) * 2019-03-22 2022-10-11 株式会社日立ソリューションズ・テクノロジー ニューラルネットワークのモデル縮約装置
KR102368962B1 (ko) 2019-03-22 2022-03-03 국민대학교산학협력단 멤리스터 어레이 회로를 제어하기 위한 게이트 회로를 포함하는 뉴럴 네트워크 시스템
JP2022095999A (ja) * 2019-04-19 2022-06-29 国立大学法人北海道大学 ニューラル計算装置、および、ニューラル計算方法
KR102782971B1 (ko) 2019-05-08 2025-03-18 삼성전자주식회사 인공 신경망 모델을 트레이닝하는 컴퓨팅 장치, 인공 신경망 모델을 트레이닝하는 방법 및 이를 저장하는 메모리 시스템
JP6787444B1 (ja) 2019-05-23 2020-11-18 沖電気工業株式会社 ニューラルネットワーク軽量化装置、ニューラルネットワーク軽量化方法およびプログラム
CN112070221B (zh) * 2019-05-31 2024-01-16 中科寒武纪科技股份有限公司 运算方法、装置及相关产品
US11514311B2 (en) * 2019-07-03 2022-11-29 International Business Machines Corporation Automated data slicing based on an artificial neural network
WO2021040921A1 (en) * 2019-08-29 2021-03-04 Alibaba Group Holding Limited Systems and methods for providing vector-wise sparsity in a neural network
JP7111671B2 (ja) 2019-09-05 2022-08-02 株式会社東芝 学習装置、学習システム、および学習方法
DE102019128715A1 (de) * 2019-10-24 2021-04-29 Krohne Messtechnik Gmbh Verfahren zur Erzeugung eines neuronalen Netzes für ein Feldgerät zur Vorhersage von Feldgerätfehlern und ein entsprechendes System
US11816574B2 (en) * 2019-10-25 2023-11-14 Alibaba Group Holding Limited Structured pruning for machine learning model
JP7396117B2 (ja) 2020-02-27 2023-12-12 オムロン株式会社 モデル更新装置、方法、及びプログラム
JP7495833B2 (ja) * 2020-07-07 2024-06-05 株式会社日立ソリューションズ・テクノロジー Ｄｎｎモデル圧縮システム
EP3945470A1 (en) * 2020-07-31 2022-02-02 Aptiv Technologies Limited Methods and systems for reducing the complexity of a computational network
KR20220048832A (ko) 2020-10-13 2022-04-20 삼성전자주식회사 인공 신경망 프루닝 방법 및 장치
US12340313B2 (en) * 2020-11-30 2025-06-24 Moffett International Co., Limited Neural network pruning method and system via layerwise analysis
JP7700650B2 (ja) * 2021-11-25 2025-07-01 富士通株式会社 モデル削減プログラム、装置、及び方法
GB202118066D0 (en) 2021-12-14 2022-01-26 Univ Dublin Class separation aware artificial neural network pruning method
US12061632B2 (en) * 2022-03-29 2024-08-13 Treasure Data, Inc. Interactive adaptation of machine learning models for time series data
KR20240028055A (ko) * 2022-08-24 2024-03-05 동국대학교 산학협력단 사이버 공격에 강인한 인공 신경망 구조 재구성 장치 및 방법
US20250190694A1 (en) * 2023-12-07 2025-06-12 International Business Machines Corporation Limiting undesired large language model (llm) output

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101014935A (zh) * 2004-05-24 2007-08-08 艾菲诺瓦公司 决定群体的设计喜好
WO2009088419A1 (en) * 2008-01-09 2009-07-16 Sensormatic Electronics Corporation Electronic article surveillance system neural network minimizing false alarms and failures to deactivate
US20140180986A1 (en) * 2012-12-24 2014-06-26 Google Inc. System and method for addressing overfitting in a neural network
TWI522935B (zh) * 2011-08-16 2016-02-21 高通公司 用於神經時間編碼、學習及辨識之方法及裝置（三）
TWI559156B (zh) * 2010-10-27 2016-11-21 索力托自動化設計股份有限公司 識別稀有事件故障率的方法與系統

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5787408A (en) * 1996-08-23 1998-07-28 The United States Of America As Represented By The Secretary Of The Navy System and method for determining node functionality in artificial neural networks
JP2000298661A (ja) * 1999-04-15 2000-10-24 Fuji Xerox Co Ltd ニューラルネットワーク装置
JP4175296B2 (ja) * 2004-06-25 2008-11-05 キャタピラージャパン株式会社 建設機械のデータ処理装置及び建設機械のデータ処理方法
JP2009516246A (ja) * 2005-11-15 2009-04-16 ベルナデット ガーナー ニューラルネットワークのトレーニング方法
KR101359352B1 (ko) * 2012-06-28 2014-02-21 한국과학기술원 시각신경 회로장치 및 이를 이용한 시각신경 모방 시스템
CN103209417B (zh) * 2013-03-05 2016-01-20 北京邮电大学 基于神经网络的频谱占用状态的预测方法以及装置
US9747554B2 (en) * 2013-05-24 2017-08-29 Qualcomm Incorporated Learning device with continuous configuration capability
US9721204B2 (en) * 2013-10-28 2017-08-01 Qualcomm Incorporated Evaluation of a system including separable sub-systems over a multidimensional range
US10373054B2 (en) * 2015-04-19 2019-08-06 International Business Machines Corporation Annealed dropout training of neural networks
CN107545303B (zh) * 2016-01-20 2021-09-07 中科寒武纪科技股份有限公司 用于稀疏人工神经网络的计算装置和运算方法
US11315018B2 (en) * 2016-10-21 2022-04-26 Nvidia Corporation Systems and methods for pruning neural networks for resource efficient inference

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101014935A (zh) * 2004-05-24 2007-08-08 艾菲诺瓦公司 决定群体的设计喜好
WO2009088419A1 (en) * 2008-01-09 2009-07-16 Sensormatic Electronics Corporation Electronic article surveillance system neural network minimizing false alarms and failures to deactivate
TWI559156B (zh) * 2010-10-27 2016-11-21 索力托自動化設計股份有限公司 識別稀有事件故障率的方法與系統
TWI522935B (zh) * 2011-08-16 2016-02-21 高通公司 用於神經時間編碼、學習及辨識之方法及裝置（三）
US20140180986A1 (en) * 2012-12-24 2014-06-26 Google Inc. System and method for addressing overfitting in a neural network

Also Published As

Publication number Publication date
US20180181867A1 (en) 2018-06-28
US10552737B2 (en) 2020-02-04
JP6755849B2 (ja) 2020-09-16
CN108229667B (zh) 2019-09-10
KR102110486B1 (ko) 2020-05-13
EP3340129B1 (en) 2019-01-30
KR20180072562A (ko) 2018-06-29
JP2018129033A (ja) 2018-08-16
CN108229667A (zh) 2018-06-29
TW201824093A (zh) 2018-07-01
EP3340129A1 (en) 2018-06-27

Similar Documents

Publication Publication Date Title
TWI698807B (zh) 以類別為基礎修剪之人工類神經網路
CN111602148B (zh) 正则化神经网络架构搜索
WO2019155064A1 (en) Data compression using jointly trained encoder, decoder, and prior neural networks
KR20210032140A (ko) 뉴럴 네트워크에 대한 프루닝을 수행하는 방법 및 장치
CN110168578A (zh) 具有任务特定路径的多任务神经网络
CN116034325A (zh) 检测异常数据点的系统、设备和方法
CN109376844A (zh) 基于云平台和模型推荐的神经网络自动训练方法和装置
US11586909B2 (en) Information processing method, information processing apparatus, and computer readable storage medium
CN111598238A (zh) 深度学习模型的压缩方法及装置
CN108536784B (zh) 评论信息情感分析方法、装置、计算机存储介质和服务器
US20190138929A1 (en) System and method for automatic building of learning machines using learning machines
CN113392867A (zh) 一种图像识别方法、装置、计算机设备及存储介质
CN113961765A (zh) 基于神经网络模型的搜索方法、装置、设备和介质
CN114169460A (zh) 样本筛选方法、装置、计算机设备和存储介质
KR102413588B1 (ko) 학습 데이터에 따른 객체 인식 모델 추천 방법, 시스템 및 컴퓨터 프로그램
CN109934352B (zh) 智能模型的自动进化方法
CN119539041A (zh) 一种基于蒙特卡洛树的自适应因果发现方法及设备
KR20190108958A (ko) 유해단어 어휘목록 자동 생성과 기계학습을 이용한 청소년 유해가사 자동 분류 방법 및 장치
CN111967941B (zh) 一种构建序列推荐模型的方法和序列推荐方法
CN114819100B (zh) 目标检测模型的神经网络搜索方法、装置和电子设备
JP2019095894A (ja) 推定装置、学習装置、学習済みモデル、推定方法、学習方法、及びプログラム
CN114548401B (zh) 推荐模型的训练方法、装置、计算设备及介质
CN113869402A (zh) 基于模型应用画像的多模型融合方法及装置
CN116778264B (zh) 基于类增学习的对象分类方法、图像分类方法及相关设备
CN117577097A (zh) 模型训练方法、装置、电子设备和介质