WO2016145675A1 - 一种基于分段的两级深度学习模型的大数据处理方法 - Google Patents

一种基于分段的两级深度学习模型的大数据处理方法 Download PDF

Info

Publication number
WO2016145675A1
WO2016145675A1 PCT/CN2015/075472 CN2015075472W WO2016145675A1 WO 2016145675 A1 WO2016145675 A1 WO 2016145675A1 CN 2015075472 W CN2015075472 W CN 2015075472W WO 2016145675 A1 WO2016145675 A1 WO 2016145675A1
Authority
WO
WIPO (PCT)
Prior art keywords
layer
level
segment
deep learning
learning model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/075472
Other languages
English (en)
French (fr)
Chinese (zh)
Inventor
王劲林
尤佳莉
盛益强
李超鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Acoustics CAS
Shanghai 3Ntv Network Technology Co Ltd
Original Assignee
Institute of Acoustics CAS
Shanghai 3Ntv Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Acoustics CAS, Shanghai 3Ntv Network Technology Co Ltd filed Critical Institute of Acoustics CAS
Priority to JP2017548135A priority Critical patent/JP2018511870A/ja
Priority to US15/557,463 priority patent/US20180068215A1/en
Priority to EP15885058.6A priority patent/EP3270329A4/en
Publication of WO2016145675A1 publication Critical patent/WO2016145675A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0495—Quantised networks; Sparse networks; Compressed networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning

Definitions

  • the invention relates to the field of artificial intelligence and big data, in particular to a big data processing method based on a two-level deep learning model of segmentation.
  • Hinton et al. proposed a layer-by-layer initialization training method for deep confidence networks, which is the starting point of the deep learning method.
  • This method breaks the difficult situation of deep neural network training that lasts for several decades and the effect is not good.
  • deep learning algorithms have been widely used in the fields of image recognition, speech recognition, and natural language understanding. Deep learning is to simulate the hierarchical abstraction of the human brain, and map the underlying data layer by layer to obtain more abstract features. Because it can extract features automatically from big data, and get a good processing effect through massive sample training, Has received extensive attention.
  • the rapid growth of big data and the research breakthroughs of deep learning are complementary.
  • the rapid growth of big data requires a method to efficiently process massive data.
  • the training of deep learning models requires massive sample data. In short, big data can maximize the performance of deep learning.
  • the existing deep learning models still have many serious problems, such as: the model is difficult to expand, the parameter optimization is difficult, the training time is too long, and the reasoning efficiency is low.
  • Bengio in 2013, it summarizes the challenges and difficulties of current deep learning, including how to extend the scale of existing deep learning models and apply them to larger data sets; how to reduce parameter optimization difficulties How to avoid expensive reasoning and sampling; and how to solve the changing factors.
  • the object of the present invention is to overcome the above problems existing in the existing neural network deep learning model in big data applications, and propose a two-level deep learning model based on segmentation, by processing the deep learning model hierarchical segmentation, Based on the weighting of the model, the model expands the ability.
  • the present invention proposes a big data processing method based on the two-level deep learning model of segmentation, which can improve the processing speed of big data and shorten the processing time. .
  • the present invention provides a big data processing method based on a two-level deep learning model of segmentation, the method comprising:
  • Step 1) Construct and train a two-level deep learning model based on segmentation, which is divided into two levels from the vertical level: the first level and the second level; the layers of the first level are divided into M segments from the horizontal direction.
  • M is the number of modal inputs of the multi-modal input; the weight between the neuron nodes of the adjacent layers between different segments in the first stage is 0;
  • Step 2 The big data to be processed is divided into M subsets according to the type of the data, and the M segments of the first layer of the two-level deep learning model based on the segment are respectively input for processing;
  • Step 3 Output the big data processing result.
  • step 1) further includes:
  • Step 101 The depth learning model with the depth of the L layer is divided into two levels from the vertical level: the first level and the second level:
  • the input layer is the first layer
  • the output layer is the Lth layer
  • the L * layer is the dividing layer, 2 ⁇ L * ⁇ L-1; then all layers from the first layer to the L * layer are called the first level , while all of the L * +1 interlayer to the first layer a second layer is referred to L level;
  • Step 102) Divide the neuron nodes on each layer in the first level from the horizontal direction into M segments:
  • the input width of the L-layer neural network be N, that is, there are N neuron nodes in each layer, and the first-level neuron nodes are divided into M segments, each segment having a width of D m , 1 ⁇ m ⁇ M, and And in the same paragraph, the width of any two layers is the same;
  • Step 103) Divide the training sample into M subsets and input the M segments of the first layer of the deep learning model respectively;
  • Step 104) training the M-segment sub-model of the first level separately:
  • the weight between the neurons in the adjacent layer between different segments in the first stage is 0, that is, all the nodes in the m segment are S m , and any node in the l-1 layer is 2 ⁇ l ⁇ L * , and any node of the first layer of the o segment And m ⁇ o, then the node with Weight between
  • the deep neural network learning algorithm is used to train the M-segment sub-model of the first level.
  • Step 105) training each layer of the second level
  • Step 106) Perform global fine-tuning of the network parameters of each layer by using a deep neural network learning algorithm until the network parameters of the layers reach an optimal value.
  • the value of the L* method is: determining an optimal value by the cross-validation method in the value range of the L*.
  • the segment-based two-level deep learning model proposed by the present invention can effectively reduce the scale of the model and shorten The training time of the model;
  • the big data processing method proposed by the present invention supports parallel input of multi-source heterogeneous or multi-modal big data, improves the processing speed of big data, and shortens the processing time.
  • FIG. 1 is a flowchart of a big data processing method of a segment-based two-level deep learning model according to the present invention
  • FIG. 2 is a schematic diagram of a two-level deep learning model based on segmentation.
  • a big data processing method based on a two-level deep learning model of segmentation includes:
  • Step 1) Build and train a two-level deep learning model based on segmentation
  • Step 101 The depth learning model with the depth of the L layer is divided into two levels from the vertical direction: the first level and the second level:
  • the input layer is the first layer
  • the output layer is the Lth layer
  • the L * layer is the dividing layer, wherein 2 ⁇ L * ⁇ L-1, then all layers from the first layer to the L * layer are called first level, and all the first interlayer L * + 1 layer to the second layer is referred to L-th stage.
  • L* is as follows: an optimal value is determined by a cross-validation method within the value range of L*.
  • Step 102 dividing the neuron node on each layer in the first level from the horizontal direction into M segments; wherein M is the number of modal inputs of the multi-modal input;
  • the input width of the L-layer neural network is N, that is, there are N neuron nodes in each layer, and the first-level neuron nodes are divided into M segments, and the width of each segment is D m , 1 ⁇ m ⁇ M, and And in the same paragraph, the width of any two layers is the same;
  • Step 103) Divide the training sample into M subsets and input the M segments of the first layer of the deep learning model respectively;
  • Step 104) training the M-segment sub-model of the first level separately;
  • the weight between the neurons in the adjacent layer between different segments in the first stage is 0, that is, all the nodes in the m segment are S m , and any node in the l-1 layer is 2 ⁇ l ⁇ L * , and any node of the first layer of the o segment And m ⁇ o, then the node with Weight between
  • the deep neural network learning algorithm is used to train the first-stage M-segment sub-model.
  • Step 105) training each layer of the second level
  • Step 106 globally fine-tuning the network parameters of each layer by using a deep neural network learning algorithm until each layer The network parameters reach an optimal value;
  • the deep neural network learning algorithm is a BP algorithm.
  • Step 2 The big data to be processed is divided into M subsets according to the type of the data, and the M segments of the first layer of the two-level deep learning model based on the segment are respectively input for processing;
  • Step 3 Output the big data processing result.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Complex Calculations (AREA)
PCT/CN2015/075472 2015-03-13 2015-03-31 一种基于分段的两级深度学习模型的大数据处理方法 Ceased WO2016145675A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2017548135A JP2018511870A (ja) 2015-03-13 2015-03-31 セグメントに基づく二段深層学習モデル用のビッグデータの処理方法
US15/557,463 US20180068215A1 (en) 2015-03-13 2015-03-31 Big data processing method for segment-based two-grade deep learning model
EP15885058.6A EP3270329A4 (en) 2015-03-13 2015-03-31 Big data processing method for segment-based two-grade deep learning model

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510111904.6A CN106033554A (zh) 2015-03-13 2015-03-13 一种基于分段的两级深度学习模型的大数据处理方法
CN201510111904.6 2015-03-13

Publications (1)

Publication Number Publication Date
WO2016145675A1 true WO2016145675A1 (zh) 2016-09-22

Family

ID=56918381

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/075472 Ceased WO2016145675A1 (zh) 2015-03-13 2015-03-31 一种基于分段的两级深度学习模型的大数据处理方法

Country Status (5)

Country Link
US (1) US20180068215A1 (https=)
EP (1) EP3270329A4 (https=)
JP (1) JP2018511870A (https=)
CN (1) CN106033554A (https=)
WO (1) WO2016145675A1 (https=)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107316024A (zh) * 2017-06-28 2017-11-03 北京博睿视科技有限责任公司 基于深度学习的周界报警算法
CN108198625A (zh) * 2016-12-08 2018-06-22 北京推想科技有限公司 一种分析高维医疗数据的深度学习方法和装置
JP2018206199A (ja) * 2017-06-07 2018-12-27 Kddi株式会社 管理装置、管理方法、及びプログラム

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109522914A (zh) * 2017-09-19 2019-03-26 中国科学院沈阳自动化研究所 一种基于图像的模型融合的神经网络结构训练方法
CN109993299B (zh) * 2017-12-29 2024-02-27 中兴通讯股份有限公司 数据训练方法及装置、存储介质、电子装置
CN109005060B (zh) * 2018-08-02 2022-01-25 上海交通大学 一种基于层级化高度异构分布式系统的深度学习应用优化框架
CN109299782B (zh) * 2018-08-02 2021-11-12 奇安信科技集团股份有限公司 一种基于深度学习模型的数据处理方法及装置
CN109657285A (zh) * 2018-11-27 2019-04-19 中国科学院空间应用工程与技术中心 汽轮机转子瞬态应力的检测方法
CN109558909B (zh) * 2018-12-05 2020-10-23 清华大学深圳研究生院 基于数据分布的机器深度学习方法
CN110287175A (zh) * 2019-05-19 2019-09-27 中国地质调查局西安地质调查中心 一种资源环境承载能力的大数据智能测定系统
CN110889492B (zh) * 2019-11-25 2022-03-08 北京百度网讯科技有限公司 用于训练深度学习模型的方法和装置
CN112465030B (zh) * 2020-11-28 2022-06-07 河南财政金融学院 一种基于两级迁移学习的多源异构信息融合故障诊断方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103945533A (zh) * 2014-05-15 2014-07-23 济南嘉科电子技术有限公司 基于大数据的无线实时位置定位方法
CN104102929A (zh) * 2014-07-25 2014-10-15 哈尔滨工业大学 基于深度学习的高光谱遥感数据分类方法
WO2014205231A1 (en) * 2013-06-19 2014-12-24 The Regents Of The University Of Michigan Deep learning framework for generic object detection

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0438556A (ja) * 1990-06-04 1992-02-07 Takayama:Kk データ処理装置
KR910020571A (ko) * 1990-05-21 1991-12-20 다카도리 수나오 데이터 처리장치
JP2001022722A (ja) * 1999-07-05 2001-01-26 Nippon Telegr & Teleph Corp <Ntt> 質的変数で条件付けられる数法則の発見方法及び装置及び質的変数で条件付けられる数法則の発見プログラムを格納した記憶媒体
JP2005237668A (ja) * 2004-02-26 2005-09-08 Kazuya Mera コンピュータネットワークにおける感情を考慮した対話装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014205231A1 (en) * 2013-06-19 2014-12-24 The Regents Of The University Of Michigan Deep learning framework for generic object detection
CN103945533A (zh) * 2014-05-15 2014-07-23 济南嘉科电子技术有限公司 基于大数据的无线实时位置定位方法
CN104102929A (zh) * 2014-07-25 2014-10-15 哈尔滨工业大学 基于深度学习的高光谱遥感数据分类方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3270329A4 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108198625A (zh) * 2016-12-08 2018-06-22 北京推想科技有限公司 一种分析高维医疗数据的深度学习方法和装置
CN108198625B (zh) * 2016-12-08 2021-07-20 推想医疗科技股份有限公司 一种分析高维医疗数据的深度学习方法和装置
JP2018206199A (ja) * 2017-06-07 2018-12-27 Kddi株式会社 管理装置、管理方法、及びプログラム
CN107316024A (zh) * 2017-06-28 2017-11-03 北京博睿视科技有限责任公司 基于深度学习的周界报警算法

Also Published As

Publication number Publication date
CN106033554A (zh) 2016-10-19
US20180068215A1 (en) 2018-03-08
EP3270329A4 (en) 2018-04-04
EP3270329A1 (en) 2018-01-17
JP2018511870A (ja) 2018-04-26

Similar Documents

Publication Publication Date Title
WO2016145675A1 (zh) 一种基于分段的两级深度学习模型的大数据处理方法
CN108470320B (zh) 一种基于cnn的图像风格化方法及系统
WO2016145676A1 (zh) 基于满足k度稀疏约束的深度学习模型的大数据处理方法
CN106897714B (zh) 一种基于卷积神经网络的视频动作检测方法
CN107506761B (zh) 基于显著性学习卷积神经网络的脑部图像分割方法及系统
CN104778448B (zh) 一种基于结构自适应卷积神经网络的人脸识别方法
CN111192291B (zh) 一种基于级联回归与孪生网络的目标跟踪方法
CN107529650A (zh) 网络模型的构建和闭环检测方法、相应装置及计算机设备
CN113706545A (zh) 一种基于双分支神经判别降维的半监督图像分割方法
CN103778414A (zh) 基于深度神经网络的实时人脸识别方法
CN104751227B (zh) 用于语音识别的深度神经网络的构建方法及系统
CN110458844A (zh) 一种低光照场景的语义分割方法
CN108108677A (zh) 一种基于改进的cnn人脸表情识别方法
CN108171318B (zh) 一种基于模拟退火—高斯函数的卷积神经网络集成方法
CN108062753A (zh) 基于深度对抗学习的无监督域自适应脑肿瘤语义分割方法
CN105701508A (zh) 基于多级卷积神经网络的全局-局部优化模型及显著性检测算法
CN104217214A (zh) 基于可配置卷积神经网络的rgb-d人物行为识别方法
CN108596069A (zh) 基于深度3d残差网络的新生儿疼痛表情识别方法及系统
CN106934456A (zh) 一种深度卷积神经网络模型构建方法
CN106845631B (zh) 一种流执行方法及装置
CN111242155A (zh) 一种基于多模深度学习的双模态情感识别方法
CN106897254A (zh) 一种网络表示学习方法
Miao et al. Evolving convolutional neural networks by symbiotic organisms search algorithm for image classification
CN111401261A (zh) 基于gan-cnn框架的机器人手势识别方法
CN112836729A (zh) 一种图像分类模型的构建方法和图像分类方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15885058

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15557463

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2017548135

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

REEP Request for entry into the european phase

Ref document number: 2015885058

Country of ref document: EP