CN111406263A - 神经网络架构搜索的方法与装置 - Google Patents
神经网络架构搜索的方法与装置 Download PDFInfo
- Publication number
- CN111406263A CN111406263A CN201880068164.4A CN201880068164A CN111406263A CN 111406263 A CN111406263 A CN 111406263A CN 201880068164 A CN201880068164 A CN 201880068164A CN 111406263 A CN111406263 A CN 111406263A
- Authority
- CN
- China
- Prior art keywords
- neural network
- network model
- iteration
- structural parameters
- calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 120
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 40
- 238000003062 neural network model Methods 0.000 claims abstract description 222
- 238000005457 optimization Methods 0.000 claims abstract description 148
- 238000004364 calculation method Methods 0.000 claims abstract description 108
- 230000008569 process Effects 0.000 claims abstract description 63
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 20
- 230000006870 function Effects 0.000 claims description 73
- 238000012545 processing Methods 0.000 claims description 13
- 238000010200 validation analysis Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 101000870363 Oryctolagus cuniculus Glutathione S-transferase Yc Proteins 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
提供一种神经网络架构搜索的方法与装置，该方法包括：获取待进行网络架构搜索的神经网络模型；确定神经网络模型的搜索空间，搜索空间定义了神经网络模型中每两个节点之间的操作层上的多种操作；为搜索空间中的每个操作层上的多种操作配置结构参数；利用基于梯度信息的优化算法，对神经网络模型进行网络架构搜索，获得优化后的结构参数，其中，网络架构搜索所使用的目标优化函数包括神经网络模型的损失函数，以及采用优化过程中每次迭代的结构参数的神经网络模型的计算量与使用神经网络模型的计算设备的计算资源之间的差异。可以在计算资源有限的场景下，有效提高神经网络模型的性能。
Description
PCT国内申请，说明书已公开。
Claims (21)
- PCT国内申请，权利要求书已公开。
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2018/117957 WO2020107264A1 (zh) | 2018-11-28 | 2018-11-28 | 神经网络架构搜索的方法与装置 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111406263A true CN111406263A (zh) | 2020-07-10 |
Family
ID=70854171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201880068164.4A Pending CN111406263A (zh) | 2018-11-28 | 2018-11-28 | 神经网络架构搜索的方法与装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111406263A (zh) |
| WO (1) | WO2020107264A1 (zh) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200304A (zh) * | 2020-09-30 | 2021-01-08 | 北京市商汤科技开发有限公司 | 神经网络搜索方法、装置、电子设备和存储介质 |
| CN112819138A (zh) * | 2021-01-26 | 2021-05-18 | 上海依图网络科技有限公司 | 一种图像神经网络结构的优化方法及装置 |
| CN113312175A (zh) * | 2021-04-27 | 2021-08-27 | 北京迈格威科技有限公司 | 一种算子确定、运行方法及装置 |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114595375A (zh) * | 2020-12-03 | 2022-06-07 | 北京搜狗科技发展有限公司 | 一种搜索方法、装置和电子设备 |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9659248B1 (en) * | 2016-01-19 | 2017-05-23 | International Business Machines Corporation | Machine learning and training a computer-implemented neural network to retrieve semantically equivalent questions using hybrid in-memory representations |
| CN107209873A (zh) * | 2015-01-29 | 2017-09-26 | 高通股份有限公司 | 用于深度卷积网络的超参数选择 |
| US20170330068A1 (en) * | 2016-05-16 | 2017-11-16 | Canon Kabushiki Kaisha | Devices, systems, and methods for feature encoding |
| CN107945204A (zh) * | 2017-10-27 | 2018-04-20 | 西安电子科技大学 | 一种基于生成对抗网络的像素级人像抠图方法 |
| US20180121807A1 (en) * | 2016-10-31 | 2018-05-03 | Oracle International Corporation | When output units must obey hard constraints |
| CN108022257A (zh) * | 2017-12-28 | 2018-05-11 | 中国科学院半导体研究所 | 适用于硬件的高速卷积神经网络目标跟踪方法和装置 |
| US20180314930A1 (en) * | 2017-04-27 | 2018-11-01 | Futurewei Technologies, Inc. | Non-convex optimization by gradient-accelerated simulated annealing |
| CN108805257A (zh) * | 2018-04-26 | 2018-11-13 | 北京大学 | 一种基于参数范数的神经网络量化方法 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107180410A (zh) * | 2017-04-11 | 2017-09-19 | 中国农业大学 | 一种图像的风格化重建方法及装置 |
| CN107463953B (zh) * | 2017-07-21 | 2019-11-19 | 上海媒智科技有限公司 | 在标签含噪情况下基于质量嵌入的图像分类方法及系统 |
-
2018
- 2018-11-28 WO PCT/CN2018/117957 patent/WO2020107264A1/zh active Application Filing
- 2018-11-28 CN CN201880068164.4A patent/CN111406263A/zh active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107209873A (zh) * | 2015-01-29 | 2017-09-26 | 高通股份有限公司 | 用于深度卷积网络的超参数选择 |
| US9659248B1 (en) * | 2016-01-19 | 2017-05-23 | International Business Machines Corporation | Machine learning and training a computer-implemented neural network to retrieve semantically equivalent questions using hybrid in-memory representations |
| US20170330068A1 (en) * | 2016-05-16 | 2017-11-16 | Canon Kabushiki Kaisha | Devices, systems, and methods for feature encoding |
| US20180121807A1 (en) * | 2016-10-31 | 2018-05-03 | Oracle International Corporation | When output units must obey hard constraints |
| US20180314930A1 (en) * | 2017-04-27 | 2018-11-01 | Futurewei Technologies, Inc. | Non-convex optimization by gradient-accelerated simulated annealing |
| CN107945204A (zh) * | 2017-10-27 | 2018-04-20 | 西安电子科技大学 | 一种基于生成对抗网络的像素级人像抠图方法 |
| CN108022257A (zh) * | 2017-12-28 | 2018-05-11 | 中国科学院半导体研究所 | 适用于硬件的高速卷积神经网络目标跟踪方法和装置 |
| CN108805257A (zh) * | 2018-04-26 | 2018-11-13 | 北京大学 | 一种基于参数范数的神经网络量化方法 |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112200304A (zh) * | 2020-09-30 | 2021-01-08 | 北京市商汤科技开发有限公司 | 神经网络搜索方法、装置、电子设备和存储介质 |
| CN112819138A (zh) * | 2021-01-26 | 2021-05-18 | 上海依图网络科技有限公司 | 一种图像神经网络结构的优化方法及装置 |
| CN113312175A (zh) * | 2021-04-27 | 2021-08-27 | 北京迈格威科技有限公司 | 一种算子确定、运行方法及装置 |
| CN113312175B (zh) * | 2021-04-27 | 2024-09-06 | 北京迈格威科技有限公司 | 一种算子确定、运行方法及装置 |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020107264A1 (zh) | 2020-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111406263A (zh) | 神经网络架构搜索的方法与装置 | |
| CN109990790B (zh) | 一种无人机路径规划方法及装置 | |
| CN108154237B (zh) | 一种数据处理系统及方法 | |
| EP3340129B1 (en) | Artificial neural network class-based pruning | |
| WO2022027937A1 (zh) | 一种神经网络压缩方法、装置、设备及存储介质 | |
| CN113128678A (zh) | 神经网络的自适应搜索方法及装置 | |
| CN114330699A (zh) | 神经网络结构搜索方法及装置 | |
| JP7009020B2 (ja) | 学習方法、学習システム、学習装置、方法、適用装置、及びコンピュータプログラム | |
| WO2019141902A1 (en) | An apparatus, a method and a computer program for running a neural network | |
| CN110428046A (zh) | 神经网络结构的获取方法及装置、存储介质 | |
| CN113505883A (zh) | 一种神经网络训练方法以及装置 | |
| CN111931901A (zh) | 一种神经网络构建方法以及装置 | |
| CN111797992A (zh) | 一种机器学习优化方法以及装置 | |
| EP3767549A1 (en) | Delivery of compressed neural networks | |
| WO2022246986A1 (zh) | 数据处理方法、装置、设备及计算机可读存储介质 | |
| CN109005060B (zh) | 一种基于层级化高度异构分布式系统的深度学习应用优化框架 | |
| CN111935005B (zh) | 数据传输方法、装置、处理设备及介质 | |
| CN116668351A (zh) | 服务质量预测方法、装置、计算机设备及存储介质 | |
| Zhang et al. | Vulcan: Automatic Query Planning for Live {ML} Analytics | |
| CN112446461A (zh) | 一种神经网络模型训练方法及装置 | |
| CN115545168A (zh) | 基于注意力机制和循环神经网络的动态QoS预测方法及系统 | |
| KR102574719B1 (ko) | 동적 예측 신뢰도 임계값 선택에 기반한 저복잡도 인공지능 모델 학습을 위한 전자 장치 및 학습 방법 | |
| CN116644783A (zh) | 模型训练方法、对象处理方法及装置、电子设备、介质 | |
| CN115983362A (zh) | 一种量化方法、推荐方法以及装置 | |
| CN114610922A (zh) | 图像处理方法及装置、存储介质及电子设备 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200710 |