WO2018209651A1 - 随机权网络泛化能力改进方法及装置 - Google Patents

随机权网络泛化能力改进方法及装置 Download PDF

Info

Publication number
WO2018209651A1
WO2018209651A1 PCT/CN2017/084906 CN2017084906W WO2018209651A1 WO 2018209651 A1 WO2018209651 A1 WO 2018209651A1 CN 2017084906 W CN2017084906 W CN 2017084906W WO 2018209651 A1 WO2018209651 A1 WO 2018209651A1
Authority
WO
WIPO (PCT)
Prior art keywords
sample
training
output
random
rwn
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/084906
Other languages
English (en)
French (fr)
Inventor
何玉林
敖威
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen University
Original Assignee
Shenzhen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen University filed Critical Shenzhen University
Priority to PCT/CN2017/084906 priority Critical patent/WO2018209651A1/zh
Publication of WO2018209651A1 publication Critical patent/WO2018209651A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/091—Active learning

Definitions

  • the present invention relates to the field of data mining technologies, and in particular, to a method and apparatus for improving generalization capability of a random weight network.
  • Random Weight Network is a full-link feedforward neural network that does not rely on iterative weight update strategy. Unlike the traditional weight update strategy based on error back propagation method, RWN randomly selects the input layer weight. By solving the pseudo inverse of the hidden layer output matrix, the analytical solution of the output layer weight is calculated. Since the iterative weight adjustment is avoided, the random weight network obtains extremely fast training speed. At the same time, the universal approximation theorem theoretically guarantees the convergence of the random weight network.
  • the existing random weight network improvement work is the passive use of training data, and it is only used as a test field to test the effect of improving the work, instead of actively mining the intrinsic information of the training data to guide how to generalize the random weight network.
  • the main object of the present invention is to provide a method and a device for improving the generalization capability of a random weight network, which aims to solve the problem that the intrinsic information of the active mining training data is not implemented in the prior art to guide how to improve the generalization ability of the random weight network.
  • a first aspect of the present invention provides a method for improving randomization network generalization capability, including:
  • Step 1 Use training samples Training the random weight network RWN (r) to obtain the trained random weight network RWN (r+1) , and the training sample The uncertainty value of each sample, where the initial value of r is 0, and For the initial training sample, RWN (0) is the initial random weight network;
  • Step 2 From the training sample Selecting a target sample having the largest uncertainty value, and generating a simulation sample by using the target sample and a preset neighborhood control factor;
  • Step 3 Calculate the simulation sample and the training sample Union, using the union as a new training sample
  • the second aspect of the present invention further provides an apparatus for improving randomization network generalization capability, including:
  • Training module for using training samples Training the random weight network RWN (r) to obtain the trained random weight network RWN (r+1) , and the training sample The uncertainty value of each sample, where the initial value of r is 0, and For the initial training sample, RWN (0) is the initial random weight network;
  • a calculation module configured to calculate the simulation sample and the training sample Union, using the union as a new training sample
  • R is the number of iteration trainings set in advance.
  • the invention provides a method for improving the generalization ability of a random weight network, in which the randomization is not changed
  • the simulation samples are generated by mining the target samples with the largest uncertainty value in the training samples, and the random weight network is trained in an iterative manner based on the simulation samples, so that the active mining training samples can be realized.
  • the purpose of improving the generalization ability of random weight networks is not changed.
  • FIG. 1 is a schematic flowchart of a method for improving a generalization capability of a random weight network according to a first embodiment of the present invention
  • step 101 is a training sample used in step 101 in the first embodiment of the present invention. Training the random weight network RWN (r) to obtain the training sample Schematic diagram of the refinement step of the uncertainty value of each sample;
  • FIG. 3 is a schematic flowchart of a refinement step of generating a simulation sample by using a target sample and a preset neighborhood control factor in step 102 in the first embodiment of the present invention
  • FIG. 5 is a schematic diagram of functional modules of a random weight network generalization capability improving apparatus according to a second embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a refinement function module of a training module 501 according to a second embodiment of the present invention.
  • FIG. 7 is a schematic diagram of an additional function module according to a second embodiment of the present invention.
  • FIG. 8 is a schematic diagram of a refinement function module of a selection generation module in a second embodiment of the present invention.
  • the present invention proposes a method and apparatus for improving randomization network generalization capability.
  • the simulation samples are generated by mining the target samples with the largest uncertainty value in the training samples, and the random weight network is trained in an iterative manner based on the simulation samples, so that the initiative can be realized. Mining training samples to improve the generalization ability of random weight networks.
  • the method for improving the generalization capability of the random weight network in the embodiment of the present invention not only significantly improves the generalization ability of the random weight network, but also has a strong ability to control the over-fitting of the random weight network.
  • FIG. 1 is a schematic flowchart of a method for improving a generalization capability of a random weight network according to a first embodiment of the present invention, where the method includes:
  • Step 101 Using training samples Training the random weight network RWN (r) to obtain the trained random weight network RWN (r+1) , and the training sample The uncertainty value of each sample, where the initial value of r is 0, and For the initial training sample, RWN (0) is the initial random weight network;
  • an initial training sample Is a data set containing N D-dimensional training samples, as follows:
  • the output layer output matrix of the initial random weight network RWN (0) is:
  • the implicit input matrix of the random weight network RWN (0) is:
  • the hidden layer output matrix of the random weight network RWN (0) is:
  • the interval [0, 1] is subject to a uniformly distributed random number.
  • the initial training sample is utilized Training the initial random weight network RWN (0) , will get the trained random weight network RWN (1) , and training samples The uncertainty of each sample.
  • the number of trainings can be represented by r, and the initial value of r is 0, that is, after one training, the obtained random weight network is RWN (1) .
  • RWN (r) represents the random weight network obtained after the rth training.
  • the uncertainty is obtained based on the actual output and the real output of the output layer output matrix of the random weight network.
  • the training sample is used in step 101 in the first embodiment of the present invention.
  • Training the random weight network RWN (r) to obtain the training sample A schematic flow diagram of the refinement step of the uncertainty value of each sample, including:
  • Step 201 using training samples Training the random weight network RWN (r) to obtain an output layer output matrix
  • Step 202 The output layer output matrix is used as a real output of the training sample, and an error between the real output and an actual output of the training sample is calculated, and the error is used as each sample in the training sample. Uncertainty value.
  • Training samples obtained for the rth training And the random weight network obtained from the rth training The training sample The training is performed by inputting the random weight network RWN (r) to obtain an output layer output matrix.
  • the actual output is obtained by using the test sample, and the initial random weight network RWN (0) is input by using the preset test sample, and the obtained output layer output matrix is taken as the actual output, and in the iterative training process, the used Is the actual output.
  • each sample has a corresponding output layer matrix value, and the output layer value of each sample in the output layer output matrix is used as a real output, and the error between the actual output and the real output of each sample is calculated, so as to obtain the Deterministic value.
  • the output layer output of the random weight network obtained after the rth training for the nth training sample is the real output
  • U represents the uncertainty value. Represents the input value of sample n in the input layer input matrix.
  • Step 102 From the training sample Selecting a target sample with the largest uncertainty value, and generating a simulation sample by using the target sample and a preset neighborhood control factor;
  • a training sample After completing a training, a training sample will be obtained.
  • the uncertainty value of each sample in the training sample In each sample the sample with the largest uncertainty value is selected as the target sample, and the simulation sample is generated by using the target sample and the preset neighborhood control factor.
  • training samples The samples are as follows:
  • samples with large uncertainty values play a more important role in improving the generalization ability of random weight networks.
  • the error is 0, that is, it has no uncertainty, so it is not necessary to adjust the current learning algorithm to fit the sample.
  • FIG. 3 is a schematic flowchart of a refinement step of generating a simulation sample by using a target sample and a preset neighborhood control factor in step 102 according to the first embodiment of the present invention, including:
  • Step 301 Determine, by using the target sample and the neighborhood control factor, a value range of an input layer input matrix of the simulation sample to be generated and a value range of the output layer output matrix.
  • Step 302 Randomly extract a random number from a value range of the input layer input matrix, generate an input layer input matrix of the simulation sample by using the extracted random number, and randomly extract a random number from a value range of the output layer output matrix.
  • the output layer output matrix of the simulation sample is generated by using the extracted random number.
  • a simulation sample that is approximately the same as the target sample with high uncertainty is obtained based on the target sample with high uncertainty.
  • the target sample and the preset neighborhood control factor are used to determine the value range of the input layer input matrix of the simulation sample to be generated and the value range of the output layer output matrix.
  • the target sample The corresponding domain control factor is a D+1 dimensional "ultra-cuboid", and the value range of the input layer input matrix and the output layer output matrix of the obtained simulation sample to be generated are:
  • ⁇ represents the neighborhood control factor
  • the random number is randomly extracted from the value range of the input layer input matrix, and the input layer input matrix of the simulation sample is generated by using the extracted random number, and randomly extracted from the value range of the output layer output matrix.
  • the random number is used to generate an output layer output matrix of the simulation sample by using the extracted random number to obtain a simulation sample.
  • the neighborhood control factor is set in the embodiment of the present invention, so that the simulation sample that is approximately the same as the sample with the largest uncertainty value can be obtained, that is, generated in the ⁇ neighborhood of the high uncertainty sample to make the current random Simulation samples with reduced network error.
  • Step 103 Calculate the simulation sample and the training sample Union, using the union as a new training sample
  • the weight network RWN (R) where R is the number of iteration trainings set in advance.
  • the output layer weight matrix of the random weight network RWN (R) is:
  • W is the input layer weight matrix and B is the hidden layer offset.
  • FIG. 4 is a schematic flowchart of an additional step of the first embodiment of the present invention, including:
  • Step 401 Take M random numbers as input layer weights and P random numbers as hidden layer offsets from preset preset intervals;
  • Step 402 Calculate a first average value of the M random numbers, set an input layer weight of the initial random weight network RWN (0) according to the first average value, and calculate a second average of the P random numbers. a value, the implicit layer offset of the initial random weight network RWN (0) is set according to the second average value.
  • the input layer weight and the implicit layer offset of the initial random weight network need to be set, and the specific setting may be completed as follows:
  • 100 random numbers can be taken from any interval, and the average of the 100 random numbers is used as an input layer weight, 100 random numbers are taken from the arbitrary interval, and the average of the 100 random numbers is taken as Implicit layer offset.
  • the input layer weight and the hidden layer offset are set by taking the average of a plurality of random numbers. It can effectively eliminate the influence of input layer weight and hidden layer offset randomization on random weight network training.
  • the simulation sample is generated by mining the target sample with the largest uncertainty value in the training sample without changing the structure of the random weight network framework, and the random weight network is iteratively based on the simulation sample. Training is performed to enable active mining of training samples to improve the generalization ability of the random weight network. Further, the method for improving the generalization capability of the random weight network in the embodiment of the present invention not only significantly improves the generalization ability of the random weight network, but also has a strong ability to control the over-fitting of the random weight network.
  • the core problem to be solved by the embodiment of the present invention is how to generate a data set x containing R ⁇ 1 simulation samples. (1) , x (2) ,...,x (R) , based on The trained random weight network RWN (R) has a test error E (R) ⁇ E (0) on the data set of the test sample. Therefore, the data set of the test sample can be input into the random weight network RWN (R) to obtain the output test error E (R) , and the test error E (R) is compared with the test error E (0) to verify the pass.
  • the technical solution in the embodiment of the present invention improves the generalization ability brought by the training of the random weight network.
  • the improvement in the embodiment of the present invention is an improvement on the output layer weight of the random weight network, and the generalization ability of the random weight network is improved by iteratively optimizing the output layer weight of the random weight network by using the simulation sample.
  • the technical solution in the embodiment of the present invention starts from the training sample without introducing the random weight network frame structure, and introduces more simulation samples.
  • the improvement of the output layer weight of the random weight network is realized.
  • the technical solution in the embodiment of the present invention has the following advantages over the prior art: no modification to the framework structure of the random weight network is required, so that the improvement of the generalization capability of the random weight network is more convenient and Easy to implement; very scalable for any type of needs Based on the improved random weight network of the framework structure, the knowledge space represented by the training samples is expanded, and a batch of simulation samples distributed with high uncertainty training samples are generated; and the ability to control over-fitting is greatly reduced. The probability of overfitting.
  • the apparatus includes:
  • Training module 501 for utilizing training samples Training the random weight network RWN (r) to obtain the trained random weight network RWN (r+1) , and the training sample The uncertainty value of each sample, where the initial value of r is 0, and For the initial training sample, RWN (0) is the initial random weight network;
  • a selection generation module 502 for using the training sample Selecting a target sample having the largest uncertainty value, and generating a simulation sample by using the target sample and a preset neighborhood control factor;
  • a calculation module 503 configured to calculate the simulation sample and the training sample Union, using the union as a new training sample
  • R is the number of iteration trainings set in advance.
  • FIG. 6 is a schematic diagram of a refinement function module of the training module 501 according to the second embodiment of the present invention, including:
  • Network training module 601 for utilizing training samples Train the random weight network RWN (r) to obtain the output layer output matrix and the trained random weight network RWN (r+1) ;
  • An uncertainty calculation module 602 configured to use the output layer output matrix as a real output of the training sample, calculate an error between the real output and an actual output of the training sample, and use the error as the The uncertainty value of each sample in the training sample.
  • FIG. 7 is a schematic diagram of an additional function module according to a second embodiment of the present invention.
  • the additional function module includes:
  • the extracting module 701 is configured to take M random numbers as input layer weights from any preset preset interval And taking P random numbers that are biased as hidden layers;
  • the mean value calculation module 702 is configured to calculate a first average value of the M random numbers, set an input layer weight of the initial random weight network RWN (0) according to the first average value, and calculate the P random numbers. a second average value, the implicit layer offset of the initial random weight network RWN (0) is set according to the second average value.
  • FIG. 8 is a schematic diagram of a refinement function module of the selection generation module 502 according to the second embodiment of the present invention.
  • the selection generation module 502 includes:
  • Selection module 801 for using the training sample Select the target sample with the largest uncertainty value
  • a range determining module 802 configured to determine, by using the target sample and the neighborhood control factor, a value range of an input layer input matrix of a simulation sample to be generated and a value range of an output layer output matrix;
  • a sample generation module 803 configured to randomly extract a random number from a value range of the input layer input matrix, and generate an input layer input matrix of the simulation sample by using the extracted random number; from a value range of the output layer output matrix Randomly extracting random numbers, and generating an output layer output matrix of the simulated samples by using the extracted random numbers;
  • the value range of the input layer input matrix and the output layer output matrix are respectively:
  • the simulation sample is generated by mining the target sample with the largest uncertainty value in the training sample without changing the structure of the random weight network framework, and the random weight network is iteratively based on the simulation sample. Training is performed to enable active mining of training samples to improve the generalization ability of the random weight network. Further, the random weight network in the embodiment of the present invention The generalization ability improvement method not only significantly improves the generalization ability of the random weight network, but also has a strong ability to control the over-fitting of the random weight network.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the modules is only a logical function division.
  • there may be another division manner for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or otherwise.
  • the modules described as separate components may or may not be physically separated.
  • the components displayed as modules may or may not be physical modules, that is, may be located in one place, or may be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated modules if implemented in the form of software functional modules and sold or used as separate products, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium.
  • a number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本发明公开了一种随机权网络泛化能力改进方法及装置，该方法及装置在不改变随机权网络框架结构的前提下，通过挖掘训练样本中的不确定性值最大的目标样本，生成与不确定性值最大目标样本近似同分布的仿真样本，并基于仿真样本迭代式地对随机权网络输出层权重进行更新，使得能够实现主动挖掘训练样本潜藏信息进而对随机权网络泛化能力进行改进的目的。

Description

随机权网络泛化能力改进方法及装置 技术领域
本发明涉及数据挖掘技术领域，尤其涉及一种随机权网络泛化能力改进方法及装置。
背景技术
随机权网络(Random Weight Network，RWN)是一种不依赖迭代式权重更新策略的全链接前馈神经网络，与传统的基于误差反传方法的权重更新策略不同，RWN随机地选择输入层权重，通过求解隐含层输出矩阵的伪逆，进而计算出输出层权重的解析解。由于避免了迭代式的权重调整，因此，随机权网络获得了极快的训练速度，同时，万能逼近定理理论上保证了随机权网络的收敛性。
当前，大量关于随机权网络泛化能力改进的研究工作被提出，主要集中在随机权网络框架的改进上，包括输入层权重的改进，最佳隐含层节点个数的选取，多随机权网络的集成等，这些改进在一定程度上提升了随机权网络的预测表现，但是忽略了对训练数据本身所包含信息的深层次利用。换言之，现有的随机权网络改进工作是对训练数据的被动使用，仅将其作为检验改进工作效果的试验田，而不是主动挖掘训练数据内在信息用以指导如何对随机权网络的泛化能力进行改进。
发明内容
本发明的主要目的在于提供一种随机权网络泛化能力改进方法及装置，旨在解决现有技术中，未实现主动挖掘训练数据内在信息用以指导如何对随机权网络的泛化能力进行改进的技术问题。
为实现上述目的，本发明第一方面提供一种随机权网络泛化能力改进方法，包括：
步骤1：利用训练样本
Figure PCTCN2017084906-appb-000001
训练随机权网络RWN(r)，得到训练后的随机权网络RWN(r+1)，及所述训练样本
Figure PCTCN2017084906-appb-000002
中各样本的不确定性值，其中，r的初始值为0，且
Figure PCTCN2017084906-appb-000003
为初始训练样本，RWN(0)为初始随机权网络；
步骤2：从所述训练样本
Figure PCTCN2017084906-appb-000004
中选择不确定性值最大的目标样本，利用所述目标样本与预置的邻域控制因子生成仿真样本；
步骤3：计算所述仿真样本与所述训练样本
Figure PCTCN2017084906-appb-000005
的并集，将所述并集作为新的训练样本
Figure PCTCN2017084906-appb-000006
步骤4：令r＝r+1，返回所述步骤1，直至r＝R时，在执行所述步骤1之后，结束训练过程，以得到改进后的随机权网络RWN(R)，R为预先设置的迭代训练次数。
为实现上述目的，本发明第二方面还提供一种随机权网络泛化能力改进装置，包括：
训练模块，用于利用训练样本
Figure PCTCN2017084906-appb-000007
训练随机权网络RWN(r)，得到训练后的随机权网络RWN(r+1)，及所述训练样本
Figure PCTCN2017084906-appb-000008
中各样本的不确定性值，其中，r的初始值为0，且
Figure PCTCN2017084906-appb-000009
为初始训练样本，RWN(0)为初始随机权网络；
选择生成模块，用于从所述训练样本
Figure PCTCN2017084906-appb-000010
中选择不确定性值最大的目标样本，利用所述目标样本与预置的邻域控制因子生成仿真样本；
计算模块，用于计算所述仿真样本与所述训练样本的并集，将所述并集作为新的训练样本
Figure PCTCN2017084906-appb-000012
返回结束模块，用于令r＝r+1，返回所述训练模块，直至r＝R时，在执行所述训练模块之后，结束训练过程，以得到改进后的随机权网络RWN(R)，R为预先设置的迭代训练次数。
本发明提供一种随机权网络泛化能力改进方法，该方法中，在不改变随机 权网络框架结构的前提下，通过挖掘训练样本中的不确定性值最大的目标样本，生成仿真样本，并基于仿真样本通过迭代的方式对随机权网络进行训练，使得能够实现主动挖掘训练样本以对随机权网络的泛化能力进行改进的目的。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明第一实施例中随机权网络泛化能力改进方法的流程示意图；
图2为本发明第一实施例中步骤101中利用训练样本
Figure PCTCN2017084906-appb-000013
对随机权网络RWN(r)进行训练，得到所述训练样本
Figure PCTCN2017084906-appb-000014
中各样本的不确定性值的细化步骤的流程示意图；
图3为本发明第一实施例中步骤102中利用目标样本与预置的邻域控制因子生成仿真样本的细化步骤的流程示意图；
图4为本发明第一实施例的追加步骤的流程示意图；
图5为本发明第二实施例中随机权网络泛化能力改进装置的功能模块的示意图；
图6为本发明第二实施例中训练模块501的细化功能模块的示意图；
图7为本发明第二实施例的追加功能模块的示意图；
图8为本发明第二实施例中选择生成模块的细化功能模块的示意图。
具体实施方式
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而非全部实施例。基 于本发明中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
由于现有技术中，存在未实现主动挖掘训练数据内在信息用以指导如何对随机权网络的泛化能力进行改进的技术问题。
为了解决上述技术问题，本发明提出随机权网络泛化能力改进方法及装置。在不改变随机权网络框架结构的前提下，通过挖掘训练样本中的不确定性值最大的目标样本，生成仿真样本，并基于仿真样本通过迭代的方式对随机权网络进行训练，使得能够实现主动挖掘训练样本以对随机权网络的泛化能力进行改进的目的。且进一步的，本发明实施例中的随机权网络泛化能力改进方法不仅显著改进了随机权网络的泛化能力，同时具有极强的控制随机权网络过拟合的能力。
请参阅图1，为本发明第一实施例中随机权网络泛化能力改进方法的流程示意图，该方法包括：
步骤101：利用训练样本
Figure PCTCN2017084906-appb-000015
训练随机权网络RWN(r)，得到训练后的随机权网络RWN(r+1)，及所述训练样本
Figure PCTCN2017084906-appb-000016
中各样本的不确定性值，其中，r的初始值为0，且
Figure PCTCN2017084906-appb-000017
为初始训练样本，RWN(0)为初始随机权网络；
在本发明实施例中，初始训练样本
Figure PCTCN2017084906-appb-000018
是一个含有N个D维训练样本的数据集，如下：
Figure PCTCN2017084906-appb-000019
上述的初始训练样本
Figure PCTCN2017084906-appb-000020
用于训练初始随机权网络RWN(0)，其中，该初始随机权网络RWN(0)的输入层节点的个数为D，隐含层节点格式为K，输出层节点个数为1，其中，初始随机权网络RWN(0)的输入层输入矩阵为：
Figure PCTCN2017084906-appb-000021
其中，初始随机权网络RWN(0)的输出层输出矩阵为：
Figure PCTCN2017084906-appb-000022
其中，随机权网络RWN(0)的隐含输入矩阵为：
Figure PCTCN2017084906-appb-000023
其中，随机权网络RWN(0)的隐含层输出矩阵为：
Figure PCTCN2017084906-appb-000024
其中，
Figure PCTCN2017084906-appb-000025
为输入层权重矩阵，
Figure PCTCN2017084906-appb-000026
为隐含层偏置。
其中，wkd，bk，k＝1，2，...，K为任意区间内的随机数。例如，可以是区间[0，1]内服从均匀分布的随机数。
其中，
Figure PCTCN2017084906-appb-000027
为Sigmoid激活函数。
在本发明实施例中，利用初始训练样本
Figure PCTCN2017084906-appb-000028
对初始随机权网络RWN(0)进行训练，将得到训练后的随机权网络RWN(1)，及训练样本
Figure PCTCN2017084906-appb-000029
中各样本的不确定性。
其中，训练次数可以用r表示，且r的初始值为0，即一次训练之后，得到的随机权网络为RWN(1)。且RWN(r)表示第r次训练之后得到的随机权网络。
在本发明实施例中，不确定性是基于随机权网络的输出层输出矩阵的实际输出及真实输出得到的，请参阅图2，为本发明第一实施例中步骤101中利用训练样本
Figure PCTCN2017084906-appb-000030
对随机权网络RWN(r)进行训练，得到所述训练样本
Figure PCTCN2017084906-appb-000031
中各样本的不 确定性值的细化步骤的流程示意图，包括：
步骤201、利用训练样本
Figure PCTCN2017084906-appb-000032
对随机权网络RWN(r)进行训练，得到输出层输出矩阵；
步骤202、将所述输出层输出矩阵作为所述训练样本的真实输出，计算所述真实输出与所述训练样本的实际输出之间的误差，将所述误差作为所述训练样本中各样本的不确定性值。
对于第r次训练得到的训练样本
Figure PCTCN2017084906-appb-000033
及第r次训练得到的随机权网络
Figure PCTCN2017084906-appb-000034
将该训练样本
Figure PCTCN2017084906-appb-000035
输入随机权网络RWN(r)中进行训练，得到输出层输出矩阵。
将该输出层输出矩阵作为训练样本
Figure PCTCN2017084906-appb-000036
的真实输出，计算该真实输出与实际输出之间的误差，将该误差作为该训练样本
Figure PCTCN2017084906-appb-000037
中各样本的不确定性值。
其中，实际输出是利用测试样本得到的，利用预先设置的测试样本输入初始随机权网络RWN(0)，将得到的输出层输出矩阵作为实际输出，且在迭代式的训练过程中，使用的均是该实际输出。
其中，由于输出层输出矩阵为：
Figure PCTCN2017084906-appb-000038
即每一个样本都有一个对应的输出层矩阵值，将输出层输出矩阵中各样本的输出层值作为真实输出，计算各样本的实际输出与真实输出之间的误差，以便得到各样本的不确定性值。
其中，不确定性值为
Figure PCTCN2017084906-appb-000039
其中，
Figure PCTCN2017084906-appb-000040
表示第n个样本的实际输出，
Figure PCTCN2017084906-appb-000041
表示第r次训练后得到的随机权网络关于第n个训练样本的输出层输出，即为真实输出，U表示不确定性值，
Figure PCTCN2017084906-appb-000042
表示输入层输入矩阵中样本n的输入值。
步骤102：从所述训练样本
Figure PCTCN2017084906-appb-000043
中选择不确定性值最大的目标样本，利用所 述目标样本与预置的邻域控制因子生成仿真样本；
在本发明实施例中，在完成一次训练之后，将得到训练样本
Figure PCTCN2017084906-appb-000044
中各样本的不确定性值，并从该训练样本
Figure PCTCN2017084906-appb-000045
的各样本中，选择不确定性值最大的样本作为目标样本，利用该目标样本与预置的邻域控制因子生成仿真样本。
其中，训练样本
Figure PCTCN2017084906-appb-000046
各样本如下：
Figure PCTCN2017084906-appb-000047
在第r次训练时，使用的是上一次得到的训练样本
Figure PCTCN2017084906-appb-000048
及上一次得到随机权网络RWN(r-1)。在得到上述训练
Figure PCTCN2017084906-appb-000049
中的各样本的不确定性值之后，不确定性值最大的样本为
Figure PCTCN2017084906-appb-000050
则利用该样本
Figure PCTCN2017084906-appb-000051
生成仿真样本。
相比于不确定性值小的样本，不确定性值大的样本对随机权网络泛化能力的改进起着更加重要的作用。举一个离子，如果样本
Figure PCTCN2017084906-appb-000052
的误差为0，即其不具有不确定性，也就没必要对当前的学习算法进行调整以适应该样本。
请参阅图3，为本发明第一实施例中步骤102中利用目标样本与预置的邻域控制因子生成仿真样本的细化步骤的流程示意图，包括：
步骤301、利用所述目标样本及所述邻域控制因子，确定待生成的仿真样本的输入层输入矩阵的取值范围及输出层输出矩阵的取值范围；
步骤302、从所述输入层输入矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输入层输入矩阵；从所述输出层输出矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输出层输出矩阵。
在本发明实施例中，将基于高不确定性的目标样本，得到与该高不确定性的目标样本近似同分布的仿真样本。
具体的，利用目标样本及预先设置的邻域控制因子，确定待生成的仿真样本的输入层输入矩阵的取值范围及输出层输出矩阵的取值范围。
其中，目标样本
Figure PCTCN2017084906-appb-000053
对应的领域控制因子为一个D+1维的“超长方体”， 且得到的待生成的仿真样本的输入层输入矩阵的取值范围及输出层输出矩阵的取值范围分别为：
Figure PCTCN2017084906-appb-000054
其中，
Figure PCTCN2017084906-appb-000055
表示第r次训练得到的目标样本的输入层输入，
Figure PCTCN2017084906-appb-000056
表示第r次训练得到的目标样本的输出层输出，δ表示所述邻域控制因子，
Figure PCTCN2017084906-appb-000057
表示所述目标样本的输入层输入中最大值与最小值之间的差值，
Figure PCTCN2017084906-appb-000058
表示所述目标样本的输出层输出。其中，邻域控制因子δ＞0。
在得到取值范围之后，将从输入层输入矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输入层输入矩阵，并从输出层输出矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输出层输出矩阵，以得到仿真样本。
可以理解的是，在本发明实施例中设置邻域控制因子，使得能够得到与不确定性值最大的样本近似同分布的仿真样本，即在高不确定性样本的δ邻域内生成使得当前随机权网络误差降低的仿真样本。
步骤103：计算所述仿真样本与所述训练样本
Figure PCTCN2017084906-appb-000059
的并集，将所述并集作为新的训练样本
Figure PCTCN2017084906-appb-000060
步骤104：令r＝r+1，返回所述步骤101，直至r＝R时，在执行所述步骤101之后，结束训练过程，以得到改进后的随机权网络RWN(R)，R为预先设置的迭代训练次数。
在本发明实施例中，需要计算仿真样本与训练样本
Figure PCTCN2017084906-appb-000061
的并集，将该并集作为新的训练样本
Figure PCTCN2017084906-appb-000062
因此，
Figure PCTCN2017084906-appb-000063
即，每次训练都将基于此次训练使用的训练样本及此次训练得到的仿真样本得到下一次训练使用的训练样本。
在本发明实施例中，在得到新的训练样本
Figure PCTCN2017084906-appb-000064
之后，令r＝r+1，返回步骤101，使得能够通过迭代的方式对随机权网络进行训练，且在r＝R时，在执行完步骤 101之后，结束训练过程，以得到改进后的随机权网络RWN(R)，其中，R为预先设置的迭代训练次数。
其中，随机权网路RWN(R)的输出层权重矩阵为：
Figure PCTCN2017084906-appb-000065
其中，
Figure PCTCN2017084906-appb-000066
其中，W表示输入层权重矩阵，B为隐含层偏置。
进一步的，请参阅图4，为本发明第一实施例的追加步骤的流程示意图，包括：
步骤401、从预置的任意区间内取M个作为输入层权重的随机数及取P个作为隐含层偏置的随机数；
步骤402、计算所述M个随机数的第一平均值，按照所述第一平均值设置所述初始随机权网络RWN(0)的输入层权重，计算所述P个随机数的第二平均值，按照所述第二平均值设置所述初始随机权网络RWN(0)的隐含层偏置。
在本发明实施例中，在对初始随机权网络进行训练之前，需要设置该初始随机权网络的输入层权重及隐含层偏置，具体可以按下如下方式完成设置：
从预置的任意区间内取M个作为输入层权重的随机数及取P个作为隐含层偏置的随机数，计算该M个随机数的第一平均值，按照该第一平均值设置初始随机权网络RWN(0)的输入层权重，计算该P个随机数的第二平均值，按照该第二平均值设置该初始随机权网络RWN(0)的隐含层偏置。
例如，可以从任意区间中取100个随机数，并将该100个随机数的平均值作为输入层权重，从该任意区间中取100个随机数，并将该100个随机数的平均值作为隐含层偏置。
可以理解的是，通过取多个随机数的平均值设置输入层权重及隐含层偏置， 能够有效消除输入层权重和隐含层偏置随机化对随机权网络训练的影响。
在本发明实施例中，在不改变随机权网络框架结构的前提下，通过挖掘训练样本中的不确定性值最大的目标样本，生成仿真样本，并基于仿真样本通过迭代的方式对随机权网络进行训练，使得能够实现主动挖掘训练样本以对随机权网络的泛化能力进行改进的目的。且进一步的，本发明实施例中的随机权网络泛化能力改进方法不仅显著改进了随机权网络的泛化能力，同时具有极强的控制随机权网络过拟合的能力。
可以理解的是，在通过R次训练得到随机权网络RWN(R)之后，还可以对该随机权网络RWN(R)的泛化能力进行验证，具体可以如下：
假设初始随机权网络RWN(0)在独立的测试样本的数据集上的测试误差为E(0)，本发明实施例拟解决的核心问题是如何生成含有R≥1个仿真样本的数据集x(1)，x(2)，...，x(R)，使得基于
Figure PCTCN2017084906-appb-000067
训练得到的随机权网络RWN(R)在测试样本的数据集上的测试误差E(R)＜E(0)。因此，可将测试样本的数据集输入随机权网络RWN(R)，得到输出的测试误差E(R)，并利用该测试误差E(R)与测试误差E(0)进行比较，以验证通过本发明实施例中的技术方案对随机权网络进行训练带来的泛化能力的提升。
可以理解的是，本发明实施例中的改进是对随机权网络的输出层权重的改进，通过利用仿真样本迭代式地优化随机权网络的输出层权重从而改进随机权网络的泛化能力。相对于现有技术中通过对随机权网络框架结构的改进，本发明实施例中的技术方案，是在不改变随机权网络框架结构的基础上，从训练样本入手，通过引入更多的仿真样本来达到提升随机权网络预测表现的目的，即实现了对随机权网络的输出层权重的改进。
可以理解的是，本发明实施例中的技术方案相对于现有技术主要有以下优势：不需要对随机权网络的框架结构进行任何修改，使得对随机权网络的泛化能力的改进更加简便且易于实现；具有很强的扩展性，适用于任意类型的需要 基于框架结构改进的随机权网络、对训练样本所表示的知识空间进行了扩充，生成了一批与高不确定性训练样本同分布的仿真样本；以及高效的控制过拟合的能力，大大降低过拟合现象出现的概率。
请参阅图5，为本发明第二实施例中随机权网络泛化能力改进装置的功能模块的示意图，该装置包括：
训练模块501，用于利用训练样本
Figure PCTCN2017084906-appb-000068
训练随机权网络RWN(r)，得到训练后的随机权网络RWN(r+1)，及所述训练样本
Figure PCTCN2017084906-appb-000069
中各样本的不确定性值，其中，r的初始值为0，且
Figure PCTCN2017084906-appb-000070
为初始训练样本，RWN(0)为初始随机权网络；
选择生成模块502，用于从所述训练样本
Figure PCTCN2017084906-appb-000071
中选择不确定性值最大的目标样本，利用所述目标样本与预置的邻域控制因子生成仿真样本；
计算模块503，用于计算所述仿真样本与所述训练样本
Figure PCTCN2017084906-appb-000072
的并集，将所述并集作为新的训练样本
Figure PCTCN2017084906-appb-000073
返回结束模块504，用于令r＝r+1，返回所述训练模块501，直至r＝R时，在执行所述训练模块之后，结束训练过程，以得到改进后的随机权网络RWN(R)，R为预先设置的迭代训练次数。
进一步的，请参阅图6，为本发明第二实施例中训练模块501的细化功能模块的示意图，包括：
网络训练模块601，用于利用训练样本
Figure PCTCN2017084906-appb-000074
训练随机权网络RWN(r)，得到输出层输出矩阵及训练后的随机权网络RWN(r+1)；
不确定性计算模块602，用于将所述输出层输出矩阵作为所述训练样本的真实输出，计算所述真实输出与所述训练样本的实际输出之间的误差，将所述误差作为所述训练样本中各样本的不确定性值。
进一步的，请参阅图7，为本发明第二实施例的追加功能模块的示意图，该追加功能模块包括：
提取模块701，用于从预置的任意区间内取M个作为输入层权重的随机数 及取P个作为隐含层偏置的随机数；
均值计算模块702，用于计算所述M个随机数的第一平均值，按照所述第一平均值设置所述初始随机权网络RWN(0)的输入层权重，计算所述P个随机数的第二平均值，按照所述第二平均值设置所述初始随机权网络RWN(0)的隐含层偏置。
进一步的，请参阅图8，为本发明第二实施例中选择生成模块502的细化功能模块的示意图，该选择生成模块502包括：
选择模块801用于从所述训练样本
Figure PCTCN2017084906-appb-000075
中选择不确定性值最大的目标样本；
范围确定模块802，用于利用所述目标样本及所述邻域控制因子，确定待生成的仿真样本的输入层输入矩阵的取值范围及输出层输出矩阵的取值范围；
样本生成模块803，用于从所述输入层输入矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输入层输入矩阵；从所述输出层输出矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输出层输出矩阵；
其中，所述输入层输入矩阵的取值范围及输出层输出矩阵的取值范围分别为：
Figure PCTCN2017084906-appb-000076
其中，
Figure PCTCN2017084906-appb-000077
表示第r次训练得到的目标样本的输入层输入，
Figure PCTCN2017084906-appb-000078
表示第r次训练得到的目标样本的输出层输出，δ表示所述邻域控制因子，
Figure PCTCN2017084906-appb-000079
表示所述目标样本的输入层输入中最大值与最小值之间的差值，
Figure PCTCN2017084906-appb-000080
表示所述目标样本的输出层输出。
在本发明实施例中，在不改变随机权网络框架结构的前提下，通过挖掘训练样本中的不确定性值最大的目标样本，生成仿真样本，并基于仿真样本通过迭代的方式对随机权网络进行训练，使得能够实现主动挖掘训练样本以对随机权网络的泛化能力进行改进的目的。且进一步的，本发明实施例中的随机权网 络泛化能力改进方法不仅显著改进了随机权网络的泛化能力，同时具有极强的控制随机权网络过拟合的能力。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以 存储程序代码的介质。
需要说明的是，对于前述的各方法实施例，为了简便描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本发明并不受所描述的动作顺序的限制，因为依据本发明，某些步骤可以采用其它顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定都是本发明所必须的。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其它实施例的相关描述。
以上为对本发明所提供的一种随机权网络泛化能力改进方法及装置的描述，对于本领域的技术人员，依据本发明实施例的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本发明的限制。

Claims (8)

  1. 一种随机权网络泛化能力改进方法，其特征在于，所述方法包括：
    步骤1：利用训练样本
    Figure PCTCN2017084906-appb-100001
    训练随机权网络RWN(r)，得到训练后的随机权网络RWN(r+1)，及所述训练样本
    Figure PCTCN2017084906-appb-100002
    中各样本的不确定性值，其中，r的初始值为0，且
    Figure PCTCN2017084906-appb-100003
    为初始训练样本，RWN(0)为初始随机权网络；
    步骤2：从所述训练样本
    Figure PCTCN2017084906-appb-100004
    中选择不确定性值最大的目标样本，利用所述目标样本与预置的邻域控制因子生成仿真样本；
    步骤3：计算所述仿真样本与所述训练样本
    Figure PCTCN2017084906-appb-100005
    的并集，将所述并集作为新的训练样本
    Figure PCTCN2017084906-appb-100006
    步骤4：令r＝r+1，返回所述步骤1，直至r＝R时，在执行所述步骤1之后，结束训练过程，以得到改进后的随机权网络RWN(R)，R为预先设置的迭代训练次数。
  2. 根据权利要求1所述的方法，其特征在于，所述利用训练样本
    Figure PCTCN2017084906-appb-100007
    对随机权网络RWN(r)进行训练，得到所述训练样本
    Figure PCTCN2017084906-appb-100008
    中各样本的不确定性值，包括：
    利用训练样本
    Figure PCTCN2017084906-appb-100009
    对随机权网络RWN(r)进行训练，得到输出层输出矩阵；
    将所述输出层输出矩阵作为所述训练样本的真实输出，计算所述真实输出与所述训练样本的实际输出之间的误差，将所述误差作为所述训练样本中各样本的不确定性值。
  3. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    从预置的任意区间内取M个作为输入层权重的随机数及取P个作为隐含层偏置的随机数；
    计算所述M个随机数的第一平均值，按照所述第一平均值设置所述初始随机权网络RWN(0)的输入层权重，计算所述P个随机数的第二平均值，按照所述第二平均值设置所述初始随机权网络RWN(0)的隐含层偏置。
  4. 根据权利要求1所述的方法，其特征在于，所述利用所述目标样本与预 置的邻域控制因子生成仿真样本，包括：
    利用所述目标样本及所述邻域控制因子，确定待生成的仿真样本的输入层输入矩阵的取值范围及输出层输出矩阵的取值范围；
    从所述输入层输入矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输入层输入矩阵；从所述输出层输出矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输出层输出矩阵；
    其中，所述输入层输入矩阵的取值范围及输出层输出矩阵的取值范围分别为：
    Figure PCTCN2017084906-appb-100010
    其中，
    Figure PCTCN2017084906-appb-100011
    表示第r次训练得到的目标样本的输入层输入，
    Figure PCTCN2017084906-appb-100012
    表示第r次训练得到的目标样本的输出层输出，δ表示所述邻域控制因子，
    Figure PCTCN2017084906-appb-100013
    表示所述目标样本的输入层输入中最大值与最小值之间的差值，
    Figure PCTCN2017084906-appb-100014
    表示所述目标样本的输出层输出。
  5. 一种随机权网络泛化能力改进装置，其特征在于，所述装置包括：
    训练模块，用于利用训练样本
    Figure PCTCN2017084906-appb-100015
    训练随机权网络RWN(r)，得到训练后的随机权网络RWN(r+1)，及所述训练样本
    Figure PCTCN2017084906-appb-100016
    中各样本的不确定性值，其中，r的初始值为0，且
    Figure PCTCN2017084906-appb-100017
    为初始训练样本，RWN(0)为初始随机权网络；
    选择生成模块，用于从所述训练样本
    Figure PCTCN2017084906-appb-100018
    中选择不确定性值最大的目标样本，利用所述目标样本与预置的邻域控制因子生成仿真样本；
    计算模块，用于计算所述仿真样本与所述训练样本
    Figure PCTCN2017084906-appb-100019
    的并集，将所述并集作为新的训练样本
    Figure PCTCN2017084906-appb-100020
    返回结束模块，用于令r＝r+1，返回所述训练模块，直至r＝R时，在执行所述训练模块之后，结束训练过程，以得到改进后的随机权网络RWN(R)，R为预先设置的迭代训练次数。
  6. 根据权利要求5所述的装置，其特征在于，所述训练模块包括：
    网络训练模块，用于利用训练样本
    Figure PCTCN2017084906-appb-100021
    训练随机权网络RWN(r)，得到输出层输出矩阵及训练后的随机权网络RWN(r+1)；
    不确定性计算模块，用于将所述输出层输出矩阵作为所述训练样本的真实输出，计算所述真实输出与所述训练样本的实际输出之间的误差，将所述误差作为所述训练样本中各样本的不确定性值。
  7. 根据权利要求5所述的装置，其特征在于，所述装置还包括：
    提取模块，用于从预置的任意区间内取M个作为输入层权重的随机数及从取P个作为隐含层偏置的随机数；
    均值计算模块，用于计算所述M个随机数的第一平均值，按照所述第一平均值设置所述初始随机权网络RWN(0)的输入层权重，计算所述P个随机数的第二平均值，按照所述第二平均值设置所述初始随机权网络RWN(0)的隐含层偏置。
  8. 根据权利要求5所述的装置，其特征在于，所述选择生成模块包括：
    选择模块，用于从所述训练样本
    Figure PCTCN2017084906-appb-100022
    中选择不确定性值最大的目标样本；
    范围确定模块，用于利用所述目标样本及所述邻域控制因子，确定待生成的仿真样本的输入层输入矩阵的取值范围及输出层输出矩阵的取值范围；
    样本生成模块，用于从所述输入层输入矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输入层输入矩阵；从所述输出层输出矩阵的取值范围中随机提取随机数，利用提取的随机数生成仿真样本的输出层输出矩阵；
    其中，所述输入层输入矩阵的取值范围及输出层输出的取值范围分别为：
    Figure PCTCN2017084906-appb-100023
    其中，
    Figure PCTCN2017084906-appb-100024
    表示第r次训练得到的目标样本的输入层输入，
    Figure PCTCN2017084906-appb-100025
    表示第r次训练得到的目标样本的输出层输出，δ表示所述邻域控制因子，
    Figure PCTCN2017084906-appb-100026
    表示所述目标样本的输入层输入中最大值与最小值之间的差值，
    Figure PCTCN2017084906-appb-100027
    表示所述目标样本的输出层输出。
PCT/CN2017/084906 2017-05-18 2017-05-18 随机权网络泛化能力改进方法及装置 Ceased WO2018209651A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/084906 WO2018209651A1 (zh) 2017-05-18 2017-05-18 随机权网络泛化能力改进方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/084906 WO2018209651A1 (zh) 2017-05-18 2017-05-18 随机权网络泛化能力改进方法及装置

Publications (1)

Publication Number Publication Date
WO2018209651A1 true WO2018209651A1 (zh) 2018-11-22

Family

ID=64273208

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/084906 Ceased WO2018209651A1 (zh) 2017-05-18 2017-05-18 随机权网络泛化能力改进方法及装置

Country Status (1)

Country Link
WO (1) WO2018209651A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982373A (zh) * 2012-12-31 2013-03-20 山东大学 一种混合svm回归算法的oin神经网络训练方法
CN104598972A (zh) * 2015-01-22 2015-05-06 清华大学 一种大规模数据回归神经网络快速训练方法
CN105550744A (zh) * 2015-12-06 2016-05-04 北京工业大学 一种基于迭代的神经网络聚类方法
WO2016145516A1 (en) * 2015-03-13 2016-09-22 Deep Genomics Incorporated System and method for training neural networks

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982373A (zh) * 2012-12-31 2013-03-20 山东大学 一种混合svm回归算法的oin神经网络训练方法
CN104598972A (zh) * 2015-01-22 2015-05-06 清华大学 一种大规模数据回归神经网络快速训练方法
WO2016145516A1 (en) * 2015-03-13 2016-09-22 Deep Genomics Incorporated System and method for training neural networks
CN105550744A (zh) * 2015-12-06 2016-05-04 北京工业大学 一种基于迭代的神经网络聚类方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HUANG , GUANG-BIN: "Extreme learning machine: Theory and applications", NEUROCOMPUTING, vol. 70, no. 1-3, 16 May 2006 (2006-05-16), pages 489 - 501, XP025111002 *

Similar Documents

Publication Publication Date Title
US10984319B2 (en) Neural architecture search
US20230252327A1 (en) Neural architecture search for convolutional neural networks
US12400121B2 (en) Regularized neural network architecture search
CN110334742B (zh) 一种用于文档分类的基于强化学习的通过添加虚假节点的图对抗样本生成方法
US20210295163A1 (en) Neural architecture search
US10546066B2 (en) End-to-end learning of dialogue agents for information access
WO2020182122A1 (zh) 用于生成文本匹配模型的方法和装置
CN111460384B (zh) 策略的评估方法、装置和设备
CN112257815A (zh) 模型生成方法、目标检测方法、装置、电子设备及介质
WO2019114413A1 (zh) 模型训练
CN114281976B (zh) 一种模型训练方法、装置、电子设备及存储介质
EP3542319A1 (en) Training neural networks using a clustering loss
KR20220134627A (ko) 하드웨어-최적화된 신경 아키텍처 검색
CN111862987A (zh) 语音识别方法和装置
US20170228375A1 (en) Using combined coefficients for viral action optimization in an on-line social network
US20190295688A1 (en) Processing biological sequences using neural networks
CN104361462B (zh) 基于文化基因算法的社交网络影响最大化方法
CN113657468A (zh) 预训练模型的生成方法、装置、电子设备和存储介质
CN113657466A (zh) 预训练模型的生成方法、装置、电子设备和存储介质
WO2021253938A1 (zh) 一种神经网络的训练方法、视频识别方法及装置
Du et al. Structure tuning method on deep convolutional generative adversarial network with nondominated sorting genetic algorithm II
CN121586902A (zh) 用于使用主动学习训练机器学习模型的批次选择策略
CN107256425B (zh) 随机权网络泛化能力改进方法及装置
WO2018209651A1 (zh) 随机权网络泛化能力改进方法及装置
WO2019205811A1 (zh) 一种随机权网络泛化能力改进方法、装置和计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17909786

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 25.02.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17909786

Country of ref document: EP

Kind code of ref document: A1