WO2019114618A1 - 一种深度神经网络训练方法、装置及计算机设备 - Google Patents

一种深度神经网络训练方法、装置及计算机设备 Download PDF

Info

Publication number
WO2019114618A1
WO2019114618A1 PCT/CN2018/119725 CN2018119725W WO2019114618A1 WO 2019114618 A1 WO2019114618 A1 WO 2019114618A1 CN 2018119725 W CN2018119725 W CN 2018119725W WO 2019114618 A1 WO2019114618 A1 WO 2019114618A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
task
network
training
parent node
Prior art date
Application number
PCT/CN2018/119725
Other languages
English (en)
French (fr)
Inventor
谢迪
浦世亮
Original Assignee
杭州海康威视数字技术股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州海康威视数字技术股份有限公司 filed Critical 杭州海康威视数字技术股份有限公司
Priority to EP18889433.1A priority Critical patent/EP3726435A4/en
Priority to US16/771,944 priority patent/US11514315B2/en
Publication of WO2019114618A1 publication Critical patent/WO2019114618A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks

Definitions

  • the present application relates to the field of machine learning technology, and in particular, to a deep neural network training method, apparatus, and computer device.
  • Deep learning is a machine learning method developed on the basis of artificial intelligence neural network. Deep neural network is the main model of deep learning. It interprets data by mimicking the mechanism of human brain. It is an analysis by establishing and simulating human brain. The intelligent model of learning has been widely used in applications such as speech recognition, image classification, face recognition, natural language processing, and advertisement placement.
  • the purpose of the embodiments of the present application is to provide a deep neural network training method, apparatus, and computer device to improve the computational efficiency of deep learning.
  • the specific technical solutions are as follows:
  • an embodiment of the present application provides a deep neural network training method, where the method includes:
  • each node in the tree network topology is a neural network corresponding to different tasks
  • the tree A leaf node in a network topology is a neural network that has been trained for a given task
  • the clustering analysis is performed on each node in the current network layer based on task attributes of each node in the current network layer, and extracting a common part of task attributes of multiple nodes in the same category, as the The task attributes of the parent nodes of multiple nodes, including:
  • training network parameters of each parent node including:
  • the output feature of each child node of the parent node is used as an input of the parent node, and the network parameter of the parent node is trained.
  • training network parameters of each parent node including:
  • the parent node For any parent node, based on the task attribute of the parent node, the parent node is generated by using a preset structure with a feature signal control mechanism; and the output attribute of each child node of the parent node is related to the task attribute.
  • a weighted combination of the output features trains the network parameters of the parent node.
  • determining that the deep neural network training corresponding to the tree network topology ends including:
  • the network parameters of the nodes in each network layer are sequentially trained to complete the training of each node;
  • the embodiment of the present application provides a deep neural network training device, where the device includes:
  • An acquiring module configured to acquire task attributes of each node in the current network layer for a current network layer in a tree network topology, where each node in the tree network topology is a nerve corresponding to different tasks a network, the leaf nodes in the tree network topology being a neural network that has been trained for a specified task;
  • a clustering module configured to perform cluster analysis on each node in the current network layer based on task attributes of each node in the current network layer, and extract a common part of task attributes of multiple nodes in the same category, as the The task attribute of the parent node of multiple nodes;
  • a training module configured to train network parameters of each parent node based on task attributes of each parent node
  • the determining module is configured to determine that the deep neural network training corresponding to the tree network topology ends after the training is completed for each node in each network layer.
  • the clustering module is specifically configured to:
  • the training module is specifically configured to:
  • the output feature of each child node of the parent node is used as an input of the parent node, and the network parameter of the parent node is trained.
  • the training module is specifically configured to:
  • the parent node For any parent node, based on the task attribute of the parent node, the parent node is generated by using a preset structure with a feature signal control mechanism; and the output attribute of each child node of the parent node is related to the task attribute.
  • a weighted combination of the output features trains the network parameters of the parent node.
  • the determining module is specifically configured to:
  • the network parameters of the nodes in each network layer are sequentially trained to complete the training of each node;
  • the deep neural network corresponding to the tree topology is determined.
  • the embodiment of the present application provides a computer readable storage medium for storing executable code, which is executed at runtime: a deep neural network provided by the first aspect of the embodiment of the present application Training method.
  • the embodiment of the present application provides an application program for performing the deep neural network training method provided by the first aspect of the embodiment of the present application.
  • an embodiment of the present application provides a computer device, including a processor and a computer readable storage medium, where
  • the computer readable storage medium for storing executable code
  • the processor is configured to execute the deep neural network training method provided by the first aspect of the embodiments of the present application when the executable code stored on the computer readable storage medium is executed.
  • the task attribute of each node in the current network layer that has been trained in the tree network topology structure is obtained by constructing a tree network topology structure, and the current network is based on the task attribute.
  • Each node in the layer performs cluster analysis, and the common part of the task attributes of multiple nodes in the same category is used as the task attribute of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node, After each node in each network layer completes training, it can be determined that the deep neural network corresponding to the tree topology is trained, and the deep neural network after training can achieve multiple tasks.
  • the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer.
  • the operation of the tree-like network topology layer by layer that is, a complete deep neural network can be used to implement multiple specified tasks, and nodes are clustered based on task attributes to construct a parent node of a node belonging to the same category, the parent The node can realize the common task of the child nodes, so the redundancy between the neural networks can be effectively reduced, thereby improving the computational efficiency of deep learning.
  • FIG. 1 is a schematic flow chart of a deep neural network training method according to an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a deep neural network training apparatus according to an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.
  • the embodiments of the present application provide a deep neural network training method, apparatus, and computer device.
  • the deep neural network training method provided by the embodiment of the present application is first introduced.
  • An execution body of a deep neural network training method provided by an embodiment of the present application may be a computer device that implements a plurality of specified tasks, and the execution body includes at least a core processing chip having data processing capability.
  • a manner of implementing a deep neural network training method provided by an embodiment of the present application may be at least one of software, hardware circuits, and logic circuits disposed in an execution body.
  • a deep neural network training method provided by an embodiment of the present application, the deep neural network training method may include the following steps:
  • a neural network for each specified task can be obtained, and each neural network can implement a specific task. For example, target attribute detection, target state estimation, etc., there is often a certain similarity between these tasks, so there is redundancy in each neural network that implements these tasks, that is, there are some identical between the trained neural networks.
  • the network model is used to achieve the same task. Based on the above considerations, a complete deep neural network can be designed based on the neural network used to perform different specified tasks to achieve each specified task.
  • the deep neural network may be a tree network topology, where each node may be a neural network corresponding to a different task.
  • Each node may be a neural network corresponding to a different task.
  • the intermediate node and the root node are neural networks trained on the layer by layer based on the nodes of the next layer.
  • S102 Perform cluster analysis on each node in the current network layer based on task attributes of each node in the current network layer, and extract common parts of task attributes of multiple nodes in the same category, as task attributes of parent nodes of multiple nodes. .
  • both of the specified tasks need to be detected first.
  • the two nodes corresponding to the two specified tasks can be divided into the same category, and the common parts of the two specified tasks are extracted, that is, the human body Or the detection of the face area, the detection of the human body or the face area can be used as the task attribute of the parent node of the above two nodes.
  • the task of detecting the human body or the face region can be performed first, and after detecting the human body or the face region, the task of gender recognition of the target and age estimation of the target can be performed.
  • Cluster analysis is the process of classifying data into different categories, so objects in the same category have great similarities. There are many ways to analyze clusters, such as system clustering, decomposition, joining, dynamic clustering, etc., which are not limited here.
  • the cluster analysis is to divide the nodes with similar task attributes into the same category. Therefore, the tasks may be clustered according to the similarity of the task attributes, and the nodes corresponding to the task attributes of the same category are set to have a common Parent node.
  • the similarity measure can be used for clustering. Therefore, optionally, the method of clustering each node in the network layer to obtain the parent node may include the following steps:
  • a similarity measure matrix corresponding to the task attributes of each node is generated by a preset similarity measure algorithm.
  • a plurality of nodes whose similarity is greater than a preset threshold are determined as the same category.
  • the third step is to extract the common part of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes.
  • the preset similarity measurement algorithm may be Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, standardized Euclidean distance, Mahalanobis distance, angle cosine, Hamming distance, etc., and no specific limitation is made herein.
  • the algorithm can generate a similarity measure matrix between the task attribute and the task attribute. According to the similarity measure matrix, if the similarity of the multiple task attributes is greater than the preset threshold, the multiple task attributes are similar, and then the The nodes that implement these task attributes are determined to be the same category, and the common parts of these task attributes can be calculated only once. After the execution of the common part of the task attributes, the characteristic parts of each task attribute are executed, so that each of the attributes can be effectively reduced. Redundancy between neural networks improves operational efficiency.
  • the tasks that each parent node needs to implement can be determined.
  • the traditional neural network model that implements the task can be used, but the network parameters of each parent node need to be trained to determine, that is, The output of each parent node can satisfy the requirements of the task of the neural network as its child node. Therefore, the network parameters of each parent node can be trained by the following steps:
  • the output feature of each child node of the parent node is used as an input of the parent node, and the network parameter of the parent node is trained.
  • the training process of the network parameters may be that the output features of the parent nodes of each parent node are used as inputs of the parent node, and then the network parameters are continuously adjusted so that the output features of the child nodes satisfy the specified task.
  • the first layer of the neural network tends to be the instantaneous function, and the second to last is the feature layer, that is, the output of the penultimate layer is the coded feature, so the second to last layer The output is used as input to the node at the previous level.
  • the parent node can It is composed of any structure with signal control mechanism, such as LSTM (Long Short-Term Memory) network mechanism, Attention Attention mechanism, GRU (Gated Recurrent Unit) network mechanism, etc.
  • LSTM Long Short-Term Memory
  • Attention Attention mechanism GRU (Gated Recurrent Unit) network mechanism, etc.
  • the input is weighted and combined, the network parameters of the parent node are trained, and the weights are adjusted during the training. That is, you can also train the network parameters of each parent node by the following steps:
  • the parent node For any parent node, based on the task attribute of the parent node, the parent node is generated by using a preset structure with a feature signal control mechanism; and the output attribute of each child node of the parent node is related to the task attribute.
  • a weighted combination of the output features trains the network parameters of the parent node.
  • the essence of the parent node that can be constructed by any structure with signal control mechanism is to automatically select the relevant feature signals for specific task attributes and eliminate the feature signals that are not related to the task attributes. And by weighted combination, a large weight can be assigned to the effective part of the child node, and a small weight is assigned to the invalid part, so that the desired task effect can be highlighted in the task result, and the invalid part is shielded. This can improve performance on a given task.
  • each node of a network layer in the tree topology can be obtained.
  • the deep neural network corresponding to the tree topology can be determined after the training ends.
  • the step of determining the end of the deep neural network training may specifically include:
  • the network parameters of the nodes in each network layer are sequentially trained according to the training sequence from the bottom layer to the top layer, and the training of each node is completed;
  • the execution subject can periodically detect whether there is a new neural network. If a new neural network corresponding to the specified task is newly added, the neural network can be used as the leaf node of the tree network topology according to the above process, step by step. Train to the top until the root node. In order to ensure the consistency of the update, the training process of each node in the same network layer is independent of each other and will not be interfered by other nodes.
  • the task attribute of each node in the current network layer that has been trained in the tree network topology structure is obtained by constructing a tree network topology structure, and clustering analysis is performed on each node in the current network layer based on the task attribute. And the common part of the task attributes of multiple nodes in the same category is used as the task attribute of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node, and the training is completed for each node in each network layer. After that, it can be determined that the deep neural network corresponding to the tree topology is trained, and the deep neural network after training can achieve multiple tasks.
  • the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer.
  • the operation of the tree-like network topology layer by layer that is, a complete deep neural network can be used to implement multiple specified tasks, and the parent node of the node belonging to the same category is constructed by clustering the nodes based on the task belongings.
  • the parent node can implement the common task of the child nodes, so the redundancy between the neural networks can be effectively reduced, thereby improving the computational efficiency of deep learning.
  • the embodiment of the present application provides a deep neural network training device.
  • the deep neural network training device may include:
  • the obtaining module 210 is configured to acquire task attributes of each node in the current network layer for a current network layer in a tree network topology, where each node in the tree network topology corresponds to a different task.
  • a neural network the leaf nodes in the tree network topology being neural networks that have been trained for a given task;
  • the clustering module 220 is configured to perform cluster analysis on each node in the current network layer based on task attributes of each node in the current network layer, and extract common parts of task attributes of multiple nodes in the same category. a task attribute of a parent node of a plurality of nodes;
  • the training module 230 is configured to train network parameters of each parent node based on task attributes of each parent node;
  • the determining module 240 is configured to determine that the deep neural network training corresponding to the tree network topology ends after the training is completed for each node in each network layer.
  • the clustering module 220 is specifically configured to:
  • the training module 230 is specifically configured to:
  • the output feature of each child node of the parent node is used as an input of the parent node, and the network parameter of the parent node is trained.
  • the training module 230 is specifically configured to:
  • the parent node For any parent node, based on the task attribute of the parent node, the parent node is generated by using a preset structure with a feature signal control mechanism; and the output attribute of each child node of the parent node is related to the task attribute.
  • a weighted combination of the output features trains the network parameters of the parent node.
  • the determining module 240 is specifically configured to:
  • the network parameters of the nodes in each network layer are sequentially trained to complete the training of each node;
  • the task attribute of each node in the current network layer that has been trained in the tree network topology structure is obtained by constructing a tree network topology structure, and clustering analysis is performed on each node in the current network layer based on the task attribute. And the common part of the task attributes of multiple nodes in the same category is used as the task attribute of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node, and the training is completed for each node in each network layer. After that, it can be determined that the deep neural network corresponding to the tree topology is trained, and the deep neural network after training can achieve multiple tasks.
  • the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer.
  • the operation of the tree-like network topology layer by layer that is, a complete deep neural network can be used to implement multiple specified tasks, and the parent node of the node belonging to the same category is constructed by clustering the nodes based on the task belongings.
  • the parent node can implement the common task of the child nodes, so the redundancy between the neural networks can be effectively reduced, thereby improving the computational efficiency of deep learning.
  • the embodiment of the present application provides a computer readable storage medium for storing executable code, where the executable code is used to execute at runtime:
  • the computer readable storage medium stores executable code that executes the deep neural network training method provided by the embodiment of the present application at runtime, and thus can implement: acquiring the tree network by constructing a tree network topology structure.
  • the task attributes of each node in the current network layer that have been trained in the topology, based on the task attributes, perform cluster analysis on each node in the current network layer, and use the common part of the task attributes of multiple nodes in the same category as the parent node.
  • the task attribute so that the network parameters of each parent node can be trained according to the task attributes of each parent node.
  • the deep neural network corresponding to the tree network topology structure can be determined, and the training is completed.
  • the deep neural network can achieve multiple tasks. Since the leaf nodes in the tree topology are trained for the specified task, the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer.
  • the operation of the tree-like network topology layer by layer that is, a complete deep neural network can be used to implement multiple specified tasks, and the parent node of the node belonging to the same category is constructed by clustering the nodes based on the task belongings.
  • the parent node can implement the common task of the child nodes, so the redundancy between the neural networks can be effectively reduced, thereby improving the computational efficiency of deep learning.
  • the embodiment of the present application provides an application program for performing the deep neural network training method provided by the foregoing embodiment.
  • the application performs the deep neural network training method provided by the embodiment of the present application at runtime, so that the current network that has been trained in the tree network topology structure can be obtained by constructing a tree network topology structure.
  • the task attributes of each node in the layer based on the task attributes, perform cluster analysis on each node in the current network layer, and use the common part of the task attributes of multiple nodes in the same category as the task attribute of the parent node, so that each The task attribute of the parent node trains the network parameters of each parent node.
  • the deep neural network corresponding to the tree topology can be determined, and the deep neural network after training can realize multiple task.
  • the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer.
  • the operation of the tree-like network topology layer by layer that is, a complete deep neural network can be used to implement multiple specified tasks, and the parent node of the node belonging to the same category is constructed by clustering the nodes based on the task belongings.
  • the parent node can implement the common task of the child nodes, so the redundancy between the neural networks can be effectively reduced, thereby improving the computational efficiency of deep learning.
  • the embodiment of the present application further provides a computer device, as shown in FIG. 3, including a processor 301 and a computer readable storage medium 302, wherein
  • Computer readable storage medium 302 for storing executable code
  • the processor 301 is configured to execute the executable code stored on the computer readable storage medium 302: the deep neural network training method provided by the foregoing embodiment.
  • the computer readable storage medium 302 and the processor 301 can perform data transmission by means of a wired connection or a wireless connection, and the computer device can communicate with other devices through a wired communication interface or a wireless communication interface.
  • the above computer readable storage medium may include a RAM (Random Access Memory), and may also include NVM (Non-Volatile Memory), such as at least one disk storage.
  • the computer readable storage medium may be at least one storage device located remotely from the processor.
  • the processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), or the like; or a DSP (Digital Signal Processing) or an ASIC (Application) Specific Integrated Circuit, FPGA (Field-Programmable Gate Array) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components.
  • CPU Central Processing Unit
  • NP Network Processor
  • DSP Digital Signal Processing
  • ASIC Application) Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • other programmable logic device discrete gate or transistor logic device, discrete hardware components.
  • the processor of the computer device can obtain the executable code stored in the computer readable storage medium and run the executable code to obtain the tree network by constructing a tree topology.
  • the task attributes of each node in the current network layer that have been trained in the topology based on the task attributes, perform cluster analysis on each node in the current network layer, and use the common part of the task attributes of multiple nodes in the same category as the parent node.
  • the task attribute so that the network parameters of each parent node can be trained according to the task attributes of each parent node.
  • the deep neural network corresponding to the tree network topology structure can be determined, and the training is completed.
  • the deep neural network can achieve multiple tasks.
  • the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer.
  • the operation of the tree-like network topology layer by layer that is, a complete deep neural network can be used to implement multiple specified tasks, and the parent node of the node belonging to the same category is constructed by clustering the nodes based on the task belongings.
  • the parent node can implement the common task of the child nodes, so the redundancy between the neural networks can be effectively reduced, thereby improving the computational efficiency of deep learning.
  • the application program and the computer device embodiment since the method content involved is basically similar to the foregoing method embodiment, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment. can.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)
  • Character Discrimination (AREA)

Abstract

一种深度神经网络训练方法、装置及计算机设备，其中，深度神经网络训练方法包括：针对树状网络拓扑结构中的当前网络层，获取当前网络层中各节点的任务属性(S101)；基于当前网络层中各节点的任务属性，对当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为多个节点的父节点的任务属性(S102)；基于各父节点的任务属性，训练各父节点的网络参数(S103)；在对各网络层中各节点完成训练后，确定树状网络拓扑结构对应的深度神经网络训练结束(S104)。通过本方法可以提高深度学习的运算效率。

Description

一种深度神经网络训练方法、装置及计算机设备
本申请要求于2017年12月12日提交中国专利局、申请号为201711319390.9发明名称为“一种深度神经网络训练方法、装置及计算机设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及机器学习技术领域，特别是涉及一种深度神经网络训练方法、装置及计算机设备。
背景技术
深度学习是人工智能神经网络基础上发展而来的一种机器学习方法，深度神经网络作为深度学习的主要模型，通过模仿人脑的机制来解释数据，是一种通过建立和模拟人脑进行分析学习的智能模型，其在语音识别、图像分类、人脸识别、自然语言处理、广告投放等应用领域已被广泛应用。
目前，大多数深度学习只针对单个任务，例如，对目标的属性进行检测、对目标的状态进行估计等。针对于复杂的场景，往往需要实现多个任务，通常使用的方法是，利用多个神经网络分别针对各任务进行运算，然后再将运算结果进行合并，这个过程非常消耗时间，并且由于每一个神经网络中存在高度的冗余性，导致深度学习的运算效率较低。
发明内容
本申请实施例的目的在于提供一种深度神经网络训练方法、装置及计算机设备，以提高深度学习的运算效率。具体技术方案如下：
第一方面，本申请实施例提供了一种深度神经网络训练方法，所述方法包括：
针对树状网络拓扑结构中的当前网络层，获取所述当前网络层中各节点的任务属性，其中，所述树状网络拓扑结构中的各节点为对应于不同任务的神经网络，所述树状网络拓扑结构中的叶节点为针对指定任务已完成训练的神经网络；
基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点 进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性；
基于各父节点的任务属性，训练各父节点的网络参数；
在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
可选的，所述基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性，包括：
根据所述当前网络层中各节点的任务属性，通过预设相似性度量算法，生成对应于各节点的任务属性的相似性度量矩阵；
根据所述相似性度量矩阵，将相似性大于预设阈值的多个节点确定为同一类别；
提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性。
可选的，所述基于各父节点的任务属性，训练各父节点的网络参数，包括：
针对任一父节点，基于该父节点的任务属性，将该父节点的各子节点的输出特征作为该父节点的输入，训练该父节点的网络参数。
可选的，所述基于各父节点的任务属性，训练各父节点的网络参数，包括：
针对任一父节点，基于该父节点的任务属性，利用具有特征信号控制机制的预设结构生成该父节点；获取并根据该父节点的各子节点的输出特征中与所述任务属性相关的各输出特征的加权组合，训练该父节点的网络参数。
可选的，所述在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束，包括：
从所述叶节点所处的网络层开始，按照从底层至顶层的训练顺序，依次 训练各网络层中节点的网络参数，完成各节点的训练；
在顶层的各节点训练完毕后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
第二方面，本申请实施例提供了一种深度神经网络训练装置，所述装置包括：
获取模块，用于针对树状网络拓扑结构中的当前网络层，获取所述当前网络层中各节点的任务属性，其中，所述树状网络拓扑结构中的各节点为对应于不同任务的神经网络，所述树状网络拓扑结构中的叶节点为针对指定任务已完成训练的神经网络；
聚类模块，用于基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性；
训练模块，用于基于各父节点的任务属性，训练各父节点的网络参数；
确定模块，用于在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
可选的，所述聚类模块，具体用于：
根据所述当前网络层中各节点的任务属性，通过预设相似性度量算法，生成对应于各节点的任务属性的相似性度量矩阵；
根据所述相似性度量矩阵，将相似性大于预设阈值的多个节点确定为同一类别；
提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性。
可选的，所述训练模块，具体用于：
针对任一父节点，基于该父节点的任务属性，将该父节点的各子节点的输出特征作为该父节点的输入，训练该父节点的网络参数。
可选的，所述训练模块，具体用于：
针对任一父节点，基于该父节点的任务属性，利用具有特征信号控制机制的预设结构生成该父节点；获取并根据该父节点的各子节点的输出特征中与所述任务属性相关的各输出特征的加权组合，训练该父节点的网络参数。
可选的，所述确定模块，具体用于：
从所述叶节点所处的网络层开始，按照从底层至顶层的训练顺序，依次训练各网络层中节点的网络参数，完成各节点的训练；
在顶层的各节点训练完毕后，确定所述树状网络拓扑结构对应的深度神经网络。
第三方面，本申请实施例提供了一种计算机可读存储介质，用于存储可执行代码，所述可执行代码用于在运行时执行：本申请实施例第一方面所提供的深度神经网络训练方法。
第四方面，本申请实施例提供了一种应用程序，用于在运行时执行：本申请实施例第一方面所提供的深度神经网络训练方法。
第五方面，本申请实施例提供了一种计算机设备，包括处理器和计算机可读存储介质，其中，
所述计算机可读存储介质，用于存放可执行代码；
所述处理器，用于执行所述计算机可读存储介质上所存放的可执行代码时执行：本申请实施例第一方面所提供的深度神经网络训练方法。
综上可见，本申请实施例提供的方案中，通过构建树状网络拓扑结构，获取该树状网络拓扑结构中已完成训练的当前网络层中各节点的任务属性，基于任务属性，对当前网络层中各节点进行聚类分析，并将同一类别中多个节点的任务属性的共性部分作为父节点的任务属性，这样就可以根据各父节点的任务属性训练各父节点的网络参数，在对各网络层中各节点完成训练后，可以确定树状网络拓扑结构对应的深度神经网络训练完毕，训练后的深度神经网络可以实现多个任务。由于树状网络拓扑结构中的叶节点为针对指定任务已训练好的神经网络，通过对网络层中各节点的任务属性的共性部分进行提取，可以对指定任务对应的神经网络进行复用，通过一层一层的树状网络 拓扑结构的运算，即可以利用一个完整的深度神经网络实现多个指定任务，并且基于任务属性对节点进行聚类，构建属于同一类别的节点的父节点，该父节点可以实现子节点的共性任务，因此可以有效减小神经网络间的冗余，进而提高深度学习的运算效率。
附图说明
为了更清楚地说明本申请实施例和现有技术的技术方案，下面对实施例和现有技术中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例的深度神经网络训练方法的流程示意图；
图2为本申请实施例的深度神经网络训练装置的结构示意图；
图3为本申请实施例的计算机设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案、及优点更加清楚明白，以下参照附图并举实施例，对本申请进一步详细说明。显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
下面通过具体实施例，对本申请进行详细的说明。
为了提高深度学习的运算效率，本申请实施例提供了一种深度神经网络训练方法、装置及计算机设备。
下面，首先对本申请实施例所提供的深度神经网络训练方法进行介绍。
本申请实施例所提供的一种深度神经网络训练方法的执行主体可以为实现多个指定任务的计算机设备，执行主体中至少包括具有数据处理能力的核心处理芯片。实现本申请实施例所提供的一种深度神经网络训练方法的方式可以为设置于执行主体中的软件、硬件电路和逻辑电路中的至少一种方式。
如图1所示，为本申请实施例所提供的一种深度神经网络训练方法，该深 度神经网络训练方法可以包括如下步骤：
S101，针对树状网络拓扑结构中的当前网络层，获取当前网络层中各节点的任务属性。
针对多个指定任务，利用预设训练算法，例如反向传播算法、正向传播算法、梯度下降训练算法等，可以得到针对各指定任务的神经网络，每个神经网络可以实现一个具体的任务，例如目标属性检测、目标状态估计等，这些任务之间往往存在一定的相似性，因此实现这些任务的各神经网络中也存在冗余，也就是，已训练好的神经网络之间存在一些相同的网络模型用于实现相同的任务。基于上述考虑，可以基于用于执行不同指定任务的神经网络，设计一个完整的深度神经网络，实现各指定任务。该深度神经网络可以为树状网络拓扑结构，其中每个节点可以为对应于不同任务的神经网络。该树状网络拓扑结构中一共存在三种类型的节点：叶节点、根节点及中间节点。由于最终目的是实现各指定任务，因此，叶节点为针对指定任务已完成训练的神经网络，可以将叶节点视为编码器。中间节点和根节点为基于下一层的节点，向上一层一层训练出来的神经网络。
S102，基于当前网络层中各节点的任务属性，对该当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为多个节点的父节点的任务属性。
由于树状网络拓扑结构中的一个网络层中，各节点所对应的任务之间具有一定的相似性，例如，识别目标的性别和识别目标的年龄时，由于这两个指定任务都需要首先检测出人体或者人脸区域，以此为基础进行性别分类与年龄估计，则可以将这两个指定任务对应的两个节点划分为同一类别，并且提取这两个指定任务的共性部分，即对人体或者人脸区域的检测，则可以将对人体或者人脸区域的检测作为上述两个节点的父节点的任务属性。这样，在通过深度神经网络进行运算时，可以先执行对人体或者人脸区域进行检测的任务，在检测出人体或者人脸区域后，再进行目标的性别识别和目标的年龄估计的任务。
聚类分析是将数据分类到不同的类别的过程，因此，同一类别中的对象有很大的相似性。聚类分析的方式有多种，例如系统聚类法、分解法、加入 法、动态聚类法等，这里不做限定。在本实施例中，聚类分析就是将任务属性相近的节点划分为同一类别，因此可以先根据任务属性的相似性进行任务的聚类，将同一类别的任务属性对应的节点设置为拥有公共的父节点。为了提高聚类分析的效率，可以利用相似性度量进行聚类，因此，可选的，对网络层中各节点进行聚类得到父节点的方式可以包括如下步骤：
第一步，根据当前网络层中各节点的任务属性，通过预设相似性度量算法，生成对应于各节点的任务属性的相似性度量矩阵。
第二步，根据相似性度量矩阵，将相似性大于预设阈值的多个节点确定为同一类别。
第三步，提取同一类别中多个节点的任务属性的共性部分，作为多个节点的父节点的任务属性。
预设相似性度量算法可以为欧氏距离、曼哈顿距离、切比雪夫距离、闵可夫斯基距离、标准化欧氏距离、马氏距离、夹角余弦、汉明距离等，这里不做具体的限定，通过这些算法可以生成任务属性与任务属性之间的相似性度量矩阵，根据该相似性度量矩阵，如果多个任务属性的相似性大于预设阈值，则说明多个任务属性较为相近，则可以将实现这些任务属性的节点确定为同一类别，并且这些任务属性的共性部分可以只运算一次，在基于对任务属性的共性部分的执行后，再执行各任务属性的特性部分，这样就可以有效减少各神经网络之间的冗余，提高运行效率。
S103，基于各父节点的任务属性，训练各父节点的网络参数。
在确定各父节点的任务属性后，即可以确定各父节点需要实现的任务，可以使用实现该任务的传统的神经网络模型，但是各父节点的网络参数需要进行训练才可以确定，也就是使得各父节点的输出可以满足作为其子节点的神经网络的任务的要求，因此，可以通过如下步骤训练各父节点的网络参数：
针对任一父节点，基于该父节点的任务属性，将该父节点的各子节点的输出特征作为该父节点的输入，训练该父节点的网络参数。
网络参数的训练过程可以是通过将各父节点的子节点的输出特征作为该父节点的输入，然后通过对网络参数进行不断的调整使得各子节点的输出特 征满足指定任务。针对树状网络拓扑结构的叶节点，由于神经网络的倒数第一层往往为瞬时函数，而倒数第二层为特征层，即倒数第二层的输出为编码的特征，因此将倒数第二层的输出作为上一层级的节点的输入。
由于不同特征之间会有相互影响，如果输入父节点的特征权值相同，在经过父节点和子节点的运算后，得到的结果可能与原始的任务结果有较大的差别，因此，父节点可以由任意具有信号控制机制的结构构成，例如LSTM(Long Short-Term Memory，长短期记忆)网络机制、注意力Attention机制、GRU(Gated Recurrent Unit，门控循环单元)网络机制等，通过对父节点的输入进行加权组合，训练父节点的网络参数，并在训练的过程中调整权值。即，还可以通过如下步骤训练各父节点的网络参数：
针对任一父节点，基于该父节点的任务属性，利用具有特征信号控制机制的预设结构生成该父节点；获取并根据该父节点的各子节点的输出特征中与所述任务属性相关的各输出特征的加权组合，训练该父节点的网络参数。
父节点可以由任意具有信号控制机制的结构构成的本质就是针对特定的任务属性，自动选择相关的特征信号、消除与任务属性不相关的特征信号。并且通过加权组合，可以给子节点中的有效部分分配较大的权值，给无效部分分配较小的权值，这样，就可以在任务结果中突显期望达到的任务效果，同时屏蔽无效部分，从而可以提高指定任务上的性能。
S104，在对各网络层中各节点完成训练后，确定树状网络拓扑结构对应的深度神经网络训练结束。
基于上述过程可以得到树状网络拓扑结构中一个网络层的各节点，通过对各网络层中各节点进行训练，训练结束后即可确定树状网络拓扑结构对应的深度神经网络。
可选的，确定深度神经网络训练结束的步骤，具体可以包括：
从叶节点所处的网络层开始，按照从底层至顶层的训练顺序，依次训练各网络层中节点的网络参数，完成各节点的训练；
在顶层的各节点训练完毕后，确定树状网络拓扑结构对应的深度神经网络训练结束。
执行主体可以周期性的检测是否有新的神经网络，如果新添加了一个指定任务对应的神经网络，则可以按照上述过程，将该神经网络作为树状网络拓扑结构的叶节点，一步步自底至顶训练，直至根节点。为了保证更新的一致性，同一网络层中每一个节点的训练过程是相互独立的，不会受到其他节点的干扰。
应用本实施例，通过构建树状网络拓扑结构，获取该树状网络拓扑结构中已完成训练的当前网络层中各节点的任务属性，基于任务属性，对当前网络层中各节点进行聚类分析，并将同一类别中多个节点的任务属性的共性部分作为父节点的任务属性，这样就可以根据各父节点的任务属性训练各父节点的网络参数，在对各网络层中各节点完成训练后，可以确定树状网络拓扑结构对应的深度神经网络训练完毕，训练后的深度神经网络可以实现多个任务。由于树状网络拓扑结构中的叶节点为针对指定任务已训练好的神经网络，通过对网络层中各节点的任务属性的共性部分进行提取，可以对指定任务对应的神经网络进行复用，通过一层一层的树状网络拓扑结构的运算，即可以利用一个完整的深度神经网络实现多个指定任务，并且通过基于任务属对节点进行聚类，构建属于同一类别的节点的父节点，该父节点可以实现子节点的共性任务，因此可以有效减小神经网络间的冗余，进而提高深度学习的运算效率。
相应于上述方法实施例，本申请实施例提供了一种深度神经网络训练装置，如图2所示，该深度神经网络训练装置可以包括：
获取模块210，用于针对树状网络拓扑结构中的当前网络层，获取所述当前网络层中各节点的任务属性，其中，所述树状网络拓扑结构中的各节点为对应于不同任务的神经网络，所述树状网络拓扑结构中的叶节点为针对指定任务已完成训练的神经网络；
聚类模块220，用于基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性；
训练模块230，用于基于各父节点的任务属性，训练各父节点的网络参数；
确定模块240，用于在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
可选的，所述聚类模块220，具体可以用于：
根据所述当前网络层中各节点的任务属性，通过预设相似性度量算法，生成对应于各节点的任务属性的相似性度量矩阵；
根据所述相似性度量矩阵，将相似性大于预设阈值的多个节点确定为同一类别；
提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性。
可选的，所述训练模块230，具体可以用于：
针对任一父节点，基于该父节点的任务属性，将该父节点的各子节点的输出特征作为该父节点的输入，训练该父节点的网络参数。
可选的，所述训练模块230，具体可以用于：
针对任一父节点，基于该父节点的任务属性，利用具有特征信号控制机制的预设结构生成该父节点；获取并根据该父节点的各子节点的输出特征中与所述任务属性相关的各输出特征的加权组合，训练该父节点的网络参数。
可选的，所述确定模块240，具体可以用于：
从所述叶节点所处的网络层开始，按照从底层至顶层的训练顺序，依次训练各网络层中节点的网络参数，完成各节点的训练；
在顶层的各节点训练完毕后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
应用本实施例，通过构建树状网络拓扑结构，获取该树状网络拓扑结构中已完成训练的当前网络层中各节点的任务属性，基于任务属性，对当前网络层中各节点进行聚类分析，并将同一类别中多个节点的任务属性的共性部分作为父节点的任务属性，这样就可以根据各父节点的任务属性训练各父节 点的网络参数，在对各网络层中各节点完成训练后，可以确定树状网络拓扑结构对应的深度神经网络训练完毕，训练后的深度神经网络可以实现多个任务。由于树状网络拓扑结构中的叶节点为针对指定任务已训练好的神经网络，通过对网络层中各节点的任务属性的共性部分进行提取，可以对指定任务对应的神经网络进行复用，通过一层一层的树状网络拓扑结构的运算，即可以利用一个完整的深度神经网络实现多个指定任务，并且通过基于任务属对节点进行聚类，构建属于同一类别的节点的父节点，该父节点可以实现子节点的共性任务，因此可以有效减小神经网络间的冗余，进而提高深度学习的运算效率。
另外，相应于上述实施例所提供的深度神经网络训练方法，本申请实施例提供了一种计算机可读存储介质，用于存储可执行代码，所述可执行代码用于在运行时执行：上述实施例所提供的深度神经网络训练方法。
本实施例中，计算机可读存储介质存储有在运行时执行本申请实施例所提供的深度神经网络训练方法的可执行代码，因此能够实现：通过构建树状网络拓扑结构，获取该树状网络拓扑结构中已完成训练的当前网络层中各节点的任务属性，基于任务属性，对当前网络层中各节点进行聚类分析，并将同一类别中多个节点的任务属性的共性部分作为父节点的任务属性，这样就可以根据各父节点的任务属性训练各父节点的网络参数，在对各网络层中各节点完成训练后，可以确定树状网络拓扑结构对应的深度神经网络训练完毕，训练后的深度神经网络可以实现多个任务。由于树状网络拓扑结构中的叶节点为针对指定任务已训练好的神经网络，通过对网络层中各节点的任务属性的共性部分进行提取，可以对指定任务对应的神经网络进行复用，通过一层一层的树状网络拓扑结构的运算，即可以利用一个完整的深度神经网络实现多个指定任务，并且通过基于任务属对节点进行聚类，构建属于同一类别的节点的父节点，该父节点可以实现子节点的共性任务，因此可以有效减小神经网络间的冗余，进而提高深度学习的运算效率。
另外，相应于上述实施例所提供的深度神经网络训练方法，本申请实施 例提供了一种应用程序，用于在运行时执行：上述实施例所提供的深度神经网络训练方法。
本实施例中，应用程序在运行时执行本申请实施例所提供的深度神经网络训练方法，因此能够实现：通过构建树状网络拓扑结构，获取该树状网络拓扑结构中已完成训练的当前网络层中各节点的任务属性，基于任务属性，对当前网络层中各节点进行聚类分析，并将同一类别中多个节点的任务属性的共性部分作为父节点的任务属性，这样就可以根据各父节点的任务属性训练各父节点的网络参数，在对各网络层中各节点完成训练后，可以确定树状网络拓扑结构对应的深度神经网络训练完毕，训练后的深度神经网络可以实现多个任务。由于树状网络拓扑结构中的叶节点为针对指定任务已训练好的神经网络，通过对网络层中各节点的任务属性的共性部分进行提取，可以对指定任务对应的神经网络进行复用，通过一层一层的树状网络拓扑结构的运算，即可以利用一个完整的深度神经网络实现多个指定任务，并且通过基于任务属对节点进行聚类，构建属于同一类别的节点的父节点，该父节点可以实现子节点的共性任务，因此可以有效减小神经网络间的冗余，进而提高深度学习的运算效率。
本申请实施例还提供了一种计算机设备，如图3所示，包括处理器301和计算机可读存储介质302，其中，
计算机可读存储介质302，用于存放可执行代码；
处理器301，用于执行计算机可读存储介质302上所存放的可执行代码时执行：上述实施例所提供的深度神经网络训练方法。
计算机可读存储介质302与处理器301之间可以通过有线连接或者无线连接的方式进行数据传输，并且计算机设备可以通过有线通信接口或者无线通信接口与其他的设备进行通信。
上述计算机可读存储介质可以包括RAM(Random Access Memory，随机存取存储器)，也可以包括NVM(Non-Volatile Memory，非易失性存储器)，例如至少一个磁盘存储器。可选的，计算机可读存储介质还可以是至少一个 位于远离上述处理器的存储装置。
上述处理器可以是通用处理器，包括CPU(Central Processing Unit，中央处理器)、NP(Network Processor，网络处理器)等；还可以是DSP(Digital Signal Processing，数字信号处理器)、ASIC(Application Specific Integrated Circuit，专用集成电路)、FPGA(Field-Programmable Gate Array，现场可编程门阵列)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。
本实施例中，该计算机设备的处理器通过读取计算机可读存储介质中存储的可执行代码，并通过运行该可执行代码，能够实现：通过构建树状网络拓扑结构，获取该树状网络拓扑结构中已完成训练的当前网络层中各节点的任务属性，基于任务属性，对当前网络层中各节点进行聚类分析，并将同一类别中多个节点的任务属性的共性部分作为父节点的任务属性，这样就可以根据各父节点的任务属性训练各父节点的网络参数，在对各网络层中各节点完成训练后，可以确定树状网络拓扑结构对应的深度神经网络训练完毕，训练后的深度神经网络可以实现多个任务。由于树状网络拓扑结构中的叶节点为针对指定任务已训练好的神经网络，通过对网络层中各节点的任务属性的共性部分进行提取，可以对指定任务对应的神经网络进行复用，通过一层一层的树状网络拓扑结构的运算，即可以利用一个完整的深度神经网络实现多个指定任务，并且通过基于任务属对节点进行聚类，构建属于同一类别的节点的父节点，该父节点可以实现子节点的共性任务，因此可以有效减小神经网络间的冗余，进而提高深度学习的运算效率。
对于计算机可读存储介质、应用程序以及计算机设备实施例而言，由于其所涉及的方法内容基本相似于前述的方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要 素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
本说明书中的各个实施例均采用相关的方式描述，各个实施例之间相同相似的部分互相参见即可，每个实施例重点说明的都是与其他实施例的不同之处。尤其，对于装置、计算机可读存储介质、应用程序以及计算机设备实施例而言，由于其基本相似于方法实施例，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (13)

  1. 一种深度神经网络训练方法，其特征在于，所述方法包括：
    针对树状网络拓扑结构中的当前网络层，获取所述当前网络层中各节点的任务属性，其中，所述树状网络拓扑结构中的各节点为对应于不同任务的神经网络，所述树状网络拓扑结构中的叶节点为针对指定任务已完成训练的神经网络；
    基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性；
    基于各父节点的任务属性，训练各父节点的网络参数；
    在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性，包括：
    根据所述当前网络层中各节点的任务属性，通过预设相似性度量算法，生成对应于各节点的任务属性的相似性度量矩阵；
    根据所述相似性度量矩阵，将相似性大于预设阈值的多个节点确定为同一类别；
    提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性。
  3. 根据权利要求1所述的方法，其特征在于，所述基于各父节点的任务属性，训练各父节点的网络参数，包括：
    针对任一父节点，基于该父节点的任务属性，将该父节点的各子节点的输出特征作为该父节点的输入，训练该父节点的网络参数。
  4. 根据权利要求1所述的方法，其特征在于，所述基于各父节点的任务 属性，训练各父节点的网络参数，包括：
    针对任一父节点，基于该父节点的任务属性，利用具有特征信号控制机制的预设结构生成该父节点；获取并根据该父节点的各子节点的输出特征中与所述任务属性相关的各输出特征的加权组合，训练该父节点的网络参数。
  5. 根据权利要求1所述的方法，其特征在于，所述在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束，包括：
    从所述叶节点所处的网络层开始，按照从底层至顶层的训练顺序，依次训练各网络层中节点的网络参数，完成各节点的训练；
    在顶层的各节点训练完毕后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
  6. 一种深度神经网络训练装置，其特征在于，所述装置包括：
    获取模块，用于针对树状网络拓扑结构中的当前网络层，获取所述当前网络层中各节点的任务属性，其中，所述树状网络拓扑结构中的各节点为对应于不同任务的神经网络，所述树状网络拓扑结构中的叶节点为针对指定任务已完成训练的神经网络；
    聚类模块，用于基于所述当前网络层中各节点的任务属性，对所述当前网络层中各节点进行聚类分析，提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性；
    训练模块，用于基于各父节点的任务属性，训练各父节点的网络参数；
    确定模块，用于在对各网络层中各节点完成训练后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
  7. 根据权利要求6所述的装置，其特征在于，所述聚类模块，具体用于：
    根据所述当前网络层中各节点的任务属性，通过预设相似性度量算法，生成对应于各节点的任务属性的相似性度量矩阵；
    根据所述相似性度量矩阵，将相似性大于预设阈值的多个节点确定为同 一类别；
    提取同一类别中多个节点的任务属性的共性部分，作为所述多个节点的父节点的任务属性。
  8. 根据权利要求6所述的装置，其特征在于，所述训练模块，具体用于：
    针对任一父节点，基于该父节点的任务属性，将该父节点的各子节点的输出特征作为该父节点的输入，训练该父节点的网络参数。
  9. 根据权利要求6所述的装置，其特征在于，所述训练模块，具体用于：
    针对任一父节点，基于该父节点的任务属性，利用具有特征信号控制机制的预设结构生成该父节点；获取并根据该父节点的各子节点的输出特征中与所述任务属性相关的各输出特征的加权组合，训练该父节点的网络参数。
  10. 根据权利要求6所述的装置，其特征在于，所述确定模块，具体用于：
    从所述叶节点所处的网络层开始，按照从底层至顶层的训练顺序，依次训练各网络层中节点的网络参数，完成各节点的训练；
    在顶层的各节点训练完毕后，确定所述树状网络拓扑结构对应的深度神经网络训练结束。
  11. 一种计算机可读存储介质，其特征在于，用于存储可执行代码，所述可执行代码用于在运行时执行：权利要求1-5任一项所述的深度神经网络训练方法。
  12. 一种应用程序，其特征在于，用于在运行时执行：权利要求1-5任一项所述的深度神经网络训练方法。
  13. 一种计算机设备，其特征在于，包括处理器和计算机可读存储介质，其中，
    所述计算机可读存储介质，用于存放可执行代码；
    所述处理器，用于执行所述计算机可读存储介质上所存放的可执行代码时执行：权利要求1-5任一项所述的深度神经网络训练方法。
PCT/CN2018/119725 2017-12-12 2018-12-07 一种深度神经网络训练方法、装置及计算机设备 WO2019114618A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18889433.1A EP3726435A4 (en) 2017-12-12 2018-12-07 METHOD AND DEVICE FOR TRAINING DEEP NEURAL NETWORKS AND COMPUTER DEVICE
US16/771,944 US11514315B2 (en) 2017-12-12 2018-12-07 Deep neural network training method and apparatus, and computer device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201711319390.9A CN109919296A (zh) 2017-12-12 2017-12-12 一种深度神经网络训练方法、装置及计算机设备
CN201711319390.9 2017-12-12

Publications (1)

Publication Number Publication Date
WO2019114618A1 true WO2019114618A1 (zh) 2019-06-20

Family

ID=66819940

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/119725 WO2019114618A1 (zh) 2017-12-12 2018-12-07 一种深度神经网络训练方法、装置及计算机设备

Country Status (4)

Country Link
US (1) US11514315B2 (zh)
EP (1) EP3726435A4 (zh)
CN (1) CN109919296A (zh)
WO (1) WO2019114618A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110704694B (zh) * 2019-09-29 2021-12-31 哈尔滨工业大学（威海） 一种基于网络表示学习的组织层级划分方法及其应用
CN112990424B (zh) * 2019-12-17 2024-05-10 杭州海康威视数字技术股份有限公司 神经网络模型训练的方法和装置
JP7343820B2 (ja) * 2020-04-10 2023-09-13 日本電信電話株式会社 オントロジを用いた文章データ解析情報生成装置、文章データ解析情報生成方法、および文章データ解析情報生成プログラム
CN114124654B (zh) * 2020-08-10 2023-10-27 中国移动通信集团浙江有限公司 告警合并方法、装置、计算设备及计算机存储介质
CN112328398B (zh) * 2020-11-12 2024-09-27 清华大学 任务处理方法及装置、电子设备和存储介质
US20230153601A1 (en) * 2021-11-15 2023-05-18 International Business Machines Corporation Global neural transducer models leveraging sub-task networks

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106295803A (zh) * 2016-08-10 2017-01-04 中国科学技术大学苏州研究院 深度神经网络的构建方法
US9564123B1 (en) * 2014-05-12 2017-02-07 Soundhound, Inc. Method and system for building an integrated user profile
CN107368294A (zh) * 2017-05-25 2017-11-21 阿里巴巴集团控股有限公司 应用原型生成方法、模板训练方法、装置及系统
CN107424612A (zh) * 2017-07-28 2017-12-01 北京搜狗科技发展有限公司 处理方法、装置和机器可读介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9564123B1 (en) * 2014-05-12 2017-02-07 Soundhound, Inc. Method and system for building an integrated user profile
CN106295803A (zh) * 2016-08-10 2017-01-04 中国科学技术大学苏州研究院 深度神经网络的构建方法
CN107368294A (zh) * 2017-05-25 2017-11-21 阿里巴巴集团控股有限公司 应用原型生成方法、模板训练方法、装置及系统
CN107424612A (zh) * 2017-07-28 2017-12-01 北京搜狗科技发展有限公司 处理方法、装置和机器可读介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3726435A4

Also Published As

Publication number Publication date
EP3726435A4 (en) 2021-03-17
EP3726435A1 (en) 2020-10-21
US11514315B2 (en) 2022-11-29
US20210073628A1 (en) 2021-03-11
CN109919296A (zh) 2019-06-21

Similar Documents

Publication Publication Date Title
WO2019114618A1 (zh) 一种深度神经网络训练方法、装置及计算机设备
US11086912B2 (en) Automatic questioning and answering processing method and automatic questioning and answering system
JP7202465B2 (ja) エンティティ意味関係分類
US12014257B2 (en) Domain specific language for generation of recurrent neural network architectures
KR102570278B1 (ko) 교사 모델로부터 학생 모델을 트레이닝하는데 사용되는 학습 데이터를 생성하는 장치 및 방법
CN111414987B (zh) 神经网络的训练方法、训练装置和电子设备
Sun et al. Modeling mention, context and entity with neural networks for entity disambiguation
WO2017092380A1 (zh) 用于人机对话的方法、神经网络系统和用户设备
CN109783666B (zh) 一种基于迭代精细化的图像场景图谱生成方法
CN107729322B (zh) 分词方法及装置、建立句子向量生成模型方法及装置
CN112037912A (zh) 基于医疗知识图谱的分诊模型训练方法、装置及设备
WO2020224433A1 (zh) 基于机器学习的目标对象属性预测方法及相关设备
Tu et al. Unsupervised structure learning of stochastic and-or grammars
CN108804591A (zh) 一种病历文本的文本分类方法及装置
JP2018022496A (ja) 自然言語処理装置に用いられるトレーニングデータを作成する方法と機器
CN113673244A (zh) 医疗文本处理方法、装置、计算机设备和存储介质
CN115204171A (zh) 基于超图神经网络的文档级事件抽取方法及系统
Chrol-Cannon et al. On the correlation between reservoir metrics and performance for time series classification under the influence of synaptic plasticity
CN111160049A (zh) 文本翻译方法、装置、机器翻译系统和存储介质
CN113326383B (zh) 一种短文本实体链接方法、装置、计算设备与存储介质
CN107085655B (zh) 基于属性的约束概念格的中医数据处理方法及系统
Banerjee et al. Relation extraction using multi-encoder lstm network on a distant supervised dataset
WO2024119831A1 (zh) 问题生成方法、生成装置及计算机设备和存储介质
CN114881124B (zh) 因果关系图的构建方法、装置、电子设备和介质
CN112699907A (zh) 数据融合的方法、装置和设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18889433

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018889433

Country of ref document: EP

Effective date: 20200713