WO2020147450A1 - 基于计算图进化的ai模型自动生成的方法 - Google Patents

基于计算图进化的ai模型自动生成的方法 Download PDF

Info

Publication number
WO2020147450A1
WO2020147450A1 PCT/CN2019/123267 CN2019123267W WO2020147450A1 WO 2020147450 A1 WO2020147450 A1 WO 2020147450A1 CN 2019123267 W CN2019123267 W CN 2019123267W WO 2020147450 A1 WO2020147450 A1 WO 2020147450A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
computational graph
models
generation
computational
Prior art date
Application number
PCT/CN2019/123267
Other languages
English (en)
French (fr)
Inventor
钱广锐
宋煜
傅志文
吴开源
Original Assignee
探智立方(北京)科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 探智立方(北京)科技有限公司 filed Critical 探智立方(北京)科技有限公司
Publication of WO2020147450A1 publication Critical patent/WO2020147450A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/12—Computing arrangements based on biological models using genetic models

Definitions

  • the invention relates to the related technical field of AI models (AI models, namely artificial intelligence models), and in particular to a method for automatically generating AI models based on computational graph evolution.
  • AI models namely artificial intelligence models
  • the automatic generation of AI models is a frontier research field. Automatic model generation can generate simpler and more efficient neural networks based on the distribution of data.
  • the search space automatically generated by the AI model is f n ⁇ 2 n(n-1)/2 , where f is the number of different neuron operators, and n is the maximum depth of the neural network. It can be seen that during the generation process, with the increase of the supported neural network operators and the deepening of the network, the complexity of the problem may become a problem approaching an infinite search space, resulting in failure to solve.
  • the main search methods include reinforcement learning (ie reinforcement learning), Monte Carlo tree search (random sampling or statistical experimental methods) and other methods.
  • reinforcement learning ie reinforcement learning
  • Monte Carlo tree search random sampling or statistical experimental methods
  • these methods all need to accumulate certain statistical information first, and generate a prior probability that is effective for model design, and then it is possible to search for a better neural network model structure within a finite period.
  • the traditional algorithm fits and selects the network, it can only be run in its entirety to find further search directions.
  • the actual time of a training session will be tens of minutes or even tens of hours.
  • the network difference is also getting smaller, and the training results of similar networks will be very similar, which causes the entire model search process to be very long.
  • a complete training of deep learning can range from several hours to several weeks, and automatic neural network design requires a lot of training to find the optimal solution.
  • the network deepens under the existing computing power, it will almost Becomes an unsolvable problem.
  • the purpose of the present invention is:
  • a method for automatic generation of AI models based on computational graph evolution which can do machine learning and deep learning at the same time; avoid repeated calculations of the same model, improve model design efficiency; ensure diversity and The uniform distribution of the sampling space realizes the search within the local optimal range, and at the same time realizes the jump out of the local optimal; guarantees the efficiency of the search and prevents the performance of the search network from declining; it can be directly evaluated without training with actual data.
  • a method for automatically generating AI models based on computational graph evolution including the following steps:
  • Step (1) According to the data preset by the user, prepare the data, set the production parameters of the model design platform, and start the automatic model design;
  • Step (2) Use genetic algorithm operators to generate the first-generation computational graph model
  • Step (4) According to the fitness of the model, remove invalid models and repeated models, and use the remaining models as candidate models and keep them as next-generation seeds;
  • Step (5) According to the next-generation seeds retained in step (4), select a number of optimal models
  • Step (6) According to the candidate model selected as the next-generation seed in step (4), a genetic algorithm operator is used to generate a new computational graph model;
  • Step (7) Determine whether the new computational graph model generated in step (6) is a computational graph model that has already been generated, if not, go to step (8); if yes, return to step (6);
  • Step (8) Save the calculation graph model of steps (5) and (7) as a new generation calculation graph model
  • Step (9) Determine whether the number of new-generation computational graph models in step (8) meets the preset data in step (1), if yes, go to the next step; if not, return to step (6);
  • a hyperparameter search is performed to find the optimal solution or a sub-optimal solution close to the optimal solution.
  • the definition of the life cycle of more than three generations is the same as the definition of "algebraic" number in genetic algorithm. Since the model structure first appeared in this process, it is counted as the first generation, and the model retained after the hyperparameter search enters step (11);
  • Step (11) Judge whether the new computational graph model meets the evolution end condition preset in step (1), if so, go to step (12); if not, go back to step (3b);
  • the data preset by the user in the step (1) includes the statistical distribution of the data, the correlation coefficient between data dimensions, and/or the statistical correlation between the data dimensions and tags.
  • the production parameters of the model design platform set in the step (1) include computing resources, job running time, job goals such as the number of new generation computational graph models, evolution end conditions, and/or genetic algorithm parameters.
  • job objective includes the fitness threshold of the computational graph model: it includes the fitness threshold that is considered to meet the evolution end condition and the fitness threshold that is considered to be an invalid model.
  • the genetic algorithm operators in the steps (2) and (6) include random operators, crossover operators and/or mutation operators.
  • the random operator is to randomly select the number of neurons, randomly select the types of neurons, and/or randomly determine the connection relationships of neurons.
  • the complexity mentioned in the steps (3) and (10) refers to the complexity calculated according to the number of nodes and the number of edges of the calculation graph.
  • the hyperparameters in the hyperparameter search in the step (10) refer to the control parameters of the neural network in the AI, including the learning rate, that is, the parameter and/or the weight attenuation parameter.
  • the method of the present invention has a wide range of usage scenarios: the method of the present invention is based on the coding method of computational graphs, and can realize the unified coding of machine learning and deep learning networks.
  • the same set of frameworks are used to realize the automatic design of the network. Both learning (such as stacking mode) and deep learning neural networks can be used.
  • the method of the present invention compares the same model, which can avoid the number of repeated calculations of the same model, thereby improving the model design efficiency.
  • the method of the present invention can use different operators-random operators and crossover operators can To ensure the diversity and uniform distribution of the sampling space, the mutation operator can realize the search within the local optimal range while achieving the characteristics of jumping out of the local optimal.
  • each generation retains the optimal model to ensure the efficiency of the search and prevent the performance of the search network from declining.
  • the method of the present invention performs scoring based on model data, and can be directly evaluated without training on actual data.
  • Figure 1 is a general flow chart of the implementation of the present invention.
  • Step (1) According to the data preset by the user, prepare the data, set the production parameters of the model design platform, and start the automatic model design.
  • the data preset by the user includes the statistical distribution of the data, the correlation coefficient between data dimensions, and/or the statistical correlation between data dimensions and labels.
  • the production parameters of the model design platform set in step (1) include computing resources, job running time, job goals such as the number of new-generation computational graph models, evolution end conditions, and/or genetic algorithm parameters-such as population number, algebra , Mutation, crossover and random percentage.
  • the job objective includes the fitness threshold of the computational graph model: it includes the fitness threshold that is considered to meet the evolution end condition and the fitness threshold that is considered to be an invalid model.
  • the specific data preset in this step is determined according to the needs of users in actual production.
  • Step (2) Use genetic algorithm operators to generate the first-generation computational graph model; the genetic algorithm operators include random operators, crossover operators and/or mutation operators.
  • the random operator is to randomly select the number of neurons, randomly select the types of neurons, and/or randomly determine the connection relationships of neurons.
  • the complexity mentioned in this step refers to the complexity calculated based on the number of nodes and the number of edges in the calculation graph.
  • Step (4) According to the fitness of the model, remove invalid models and repeated models, and use the remaining models as candidate models and keep them as next-generation seeds;
  • Step (5) According to the next-generation seeds retained in step (4), select a number of optimal models
  • Step (6) According to the candidate model selected in step (4) as the next-generation seed, a genetic algorithm operator is used to generate a new computational graph model.
  • the genetic algorithm operators include random operators, crossover operators and/or mutation operators.
  • the random operator is to randomly select the number of neurons, randomly select the types of neurons, and/or randomly determine the connection relationships of neurons.
  • Step (7) Determine whether the new computational graph model generated in step (6) is a computational graph model that has already been generated, if not, go to step (8); if yes, return to step (6);
  • Step (8) Save the calculation graph model of steps (5) and (7) as a new generation calculation graph model
  • Step (9) Determine whether the number of new-generation computational graph models in step (8) meets the preset data in step (1), if yes, go to the next step; if not, return to step (6);
  • a hyperparameter search is performed to find the optimal solution or a sub-optimal solution close to the optimal solution.
  • the definition of the life cycle of more than three generations is the same as the definition of "algebraic" number in genetic algorithm. Since the model structure first appeared in this process, it is counted as the first generation, and the model retained after the hyperparameter search enters step (11).
  • Model hyperparameters are configurations external to the model. Their values cannot be estimated from data, but are usually specified directly by practitioners. When applied to the process of estimating model parameters, grid search, random search, heuristic search, and shell Yes search and other methods to set up, adjust according to the given predictive modeling problem.
  • the hyperparameters in the "superparameter search” i.e., hyperparameter search
  • the hyperparameters in the "superparameter search” refer to the control parameters of the neural network in the AI, including the learning rate, that is, the parameters and/or the weight attenuation parameters.
  • step (11) For models with a life cycle of no more than three generations, calculate the model performance according to the calculation graph structure, calculate the fitness of each calculation graph model according to the calculation graph performance and complexity, and then proceed to step (11).
  • the complexity mentioned in this step refers to the complexity calculated based on the number of nodes and the number of edges in the calculation graph.
  • Step (11) Determine whether the new model of the calculation graph meets the preset evolution end condition in step (1).
  • the preset evolution end condition is based on the content defined by the user, such as: the accuracy rate exceeds the user's expectation, and the time used reaches the user's setting Maximum duration, etc., if satisfied, go to step (12); if not satisfied, go back to step (3b);
  • the user prepares numerical calculation data (using csv format or picture format), and the data includes a label column; set the maximum evolutionary generation of the model to 3 generations, and the number of model populations in each generation to 5; preset fitness The smaller the model, the better the performance; the fitness threshold of the computational graph model—if the optimal model fitness is less than 50, the evolution end condition is considered to be satisfied, and the calculation stops; the model with the preset fitness over 1000 is considered an invalid model.
  • step (2) use genetic random operators to randomly generate 5 models of the first generation, which are: randomly generate 5 models of the first generation, namely: computational graph model 1, computational graph model 2, computational graph model 3. Computational graph model 4 and computational graph model 5.
  • step (3) the model expression vectors of computational graph model 1, computational graph model 2, computational graph model 3, computational graph model 4, and computational graph model 5 are coded:
  • the performance of the calculation graph is measured by the accuracy of the calculation graph model.
  • P is used to represent the accuracy.
  • N Computational complexity (N is used to represent complexity in the following).
  • N 1 6 for graph model 1
  • N 2 for graph model 2 9
  • N 3 for graph model 3 10
  • the invalid model and the repeated model are removed. Because the above calculation graph model 1-5 does not have a repeated model, the repeated model is removed. According to the preset of this embodiment, when the fitness exceeds 1000, it is considered invalid The model is used, so the corresponding calculation graph model 4 is removed; the remaining models are the candidate models for the calculation graph model 1, the calculation graph model 2, the calculation graph model 3, and the calculation graph model 5, and are retained as the next generation seeds.
  • F is the smallest in computational graph model 1, as the optimal model, and retained as a new generation of computational graph model.
  • step (6) use the genetic random operator to generate the computational graph model a, and the model expression vector is:
  • step (7) it is judged whether the model is a computational graph model that has been generated, model a and existing models (computation graph model 1, computational graph model 2, computational graph model 3, computational graph model 4, and computational graph Model 5) is not the same or the performance is not similar, so it is a new model.
  • the computational graph model a is saved as a new generation computational graph model 6.
  • step (9) because the preset number of model populations for each generation is 5, and currently there is only one model of computational graph model 6, so the preset conditions are not met, should return to step (6) to continue generating computational graph models .
  • step (11) since the fitness F 10 of the computational graph model 10 is 48, the evolution condition that the fitness is less than 50 is satisfied.
  • step (1) of this embodiment it is preset that the smaller the fitness, the better the model performance.
  • the fitness of the computational graph model 10 is the smallest, and the computational graph model 10 is the optimal model.
  • the automatic model generation is over.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Genetics & Genomics (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本发明提供一种基于计算图进化的AI模型自动生成的方法,主要包括如下步骤：预设数据；利用遗传算法算子生成第一代计算图模型并根据其计算图结构计算模型性能；去除无效用模型和重复模型，剩余模型作为备选模型并保留作为下一代种子；挑出若干数量最优模型；备选模型使用遗传算法算子产生一个计算图新模型；判断上一步产生的计算图新模型是否生成过；保存新模型为新一代计算图模型，判断其是否满足预设数据和进化结束条件；进化计算结果汇总，选择出最优模型。本发明可以同时做机器学习和深度学习；避免相同模型的被重复计算的次数，提升模型设计效率；跳出局部最优；防止搜索网络的性能的衰退；不需要通过实际数据进行训练就可直接评估。

Description

基于计算图进化的AI模型自动生成的方法 技术领域
本发明涉及AI模型(AI模型即人工智能模型)相关技术领域，具体涉及一种基于计算图进化的AI模型自动生成的方法。
背景技术
AI模型自动生成是前沿的研究领域。自动模型生成可以根据数据的分布生成更加简单高效的神经网络。AI模型自动生成的搜索空间为f n×2 n(n-1)/2，其中f是不同神经元算子个数，n为神经网络最大深度。可以看到在生成过程中，随着支持的神经网络算子的增加和网络的加深，问题的复杂度可能会变成趋近于无穷搜索空间的问题，从而导致无法求解。
目前主要的搜索方法有强化学习(即增强学习)、蒙特卡洛树搜索(随机抽样或统计实验方法)等方法。但是这些方法都需要先积累一定的统计信息，生成一个对模型设计有效的先验概率以后才可能在有限周期内，搜索到较好的神经网络模型结构。在传统算法拟合选出网络后，只能完整运行来寻找进一步的搜索方向。但是在深度学习领域，一次训练的实际时间会是数十分钟，甚至数十小时。而很多时候，当搜索朝向最优解逼近的时候，网络差异也在变小，而类似网络的训练结果会很相近，这样就导致整个模型搜索过程非常的长。目前深度学习的一次完整训练在数小时到数周不等，而自动神经网络设计需要大量的训练为基础才能找到最优解，随着网络的加深，在现有计算力的情况下，几乎会变成一个无解的问题。
发明内容
本发明的目的是：
针对上述现有技术的不足，提供了基于计算图进化的AI模型自动生成的方法，可以同时做机器学习和深度学习；避免相同模型的被重复计算的次数，提升模型设计效率；保证多样性和采样空间的均匀分布，实现在局域最优范围内的搜索，同时实现跳出局部最优； 保证搜索的效率，防止搜索网络的性能的衰退；不需要通过实际数据进行训练就可以直接评估。
本发明的目的可通过以下的技术方案来实现：
一种基于计算图进化的AI模型自动生成的方法，包括以下步骤：
步骤(1)：根据用户预设的数据，进行数据准备，设置模型设计平台生产参数，开始模型自动设计；
步骤(2)：利用遗传算法算子生成第一代计算图模型；
步骤(3)：
a、根据第一代计算图结构计算模型性能；
b、根据计算图性能(比如其准确率)和复杂度，计算每一个计算图模型的适应度；
步骤(4)：根据模型适应度，去除无效用模型和重复模型，剩余模型作为备选模型，并保留作为下一代种子；
步骤(5)：根据步骤(4)保留的下一代种子，挑出若干数量最优模型；
步骤(6)：根据步骤(4)选择出的作为下一代种子的备选模型使用遗传算法算子产生一个计算图新模型；
步骤(7)：判断步骤(6)产生的计算图新模型是否为已经生成过的计算图模型，如否，进入步骤(8)；如是，返回步骤(6)；
步骤(8)：保存步骤(5)、(7)的计算图模型为新一代计算图模型；
步骤(9)：判断是否步骤(8)的新一代计算图模型的数量满足步骤(1)中预设的数据，如是，进入下一步；如否，返回步骤(6)；
步骤(10)：
a、对生命周期超过三代的模型，则再进行寻找最优解或者靠近最优解的次优解的超参搜索，所述生命周期超过三代的定义同遗传算法中“代”数的定义，自该模型结构第一次出现在本流程中开始算为第一代，超参搜索后保留下来的模型进入步骤(11)；
b、对生命周期没有超过三代的模型，根据计算图结构计算模型性能，根据计算图性能和复杂度计算每一个计算图模型的适应度，然后进入步骤(11)；
步骤(11)：判断计算图新模型是否满足步骤(1)中预设的进化结束条件，如满足，进入步骤(12)；如不满足，返回步骤(3b)；
步骤(12)：进化计算结果汇总，根据模型复杂度和准确率进行综合评分，选择出最优模型。
所述步骤(1)中用户预设的数据，包括了数据的统计分布、数据维度间相关系数和/ 或数据各维度和标签之间的统计相关性。
所述步骤(1)中设置的模型设计平台生产参数包括了计算资源、作业运行时间、作业目标如新一代计算图模型数量、进化结束条件，和/或遗传学算法参数。所述作业目标包含了计算图模型的适应度阀值：包括了被认为满足进化结束条件的适应度阀值，和被认为是无效用模型的适应度阀值。
所述步骤(2)、(6)所述遗传算法算子包括了随机算子、交叉算子和/或变异算子。
所述随机算子为随机选取神经元个数、随机选取神经元种类和/或随机决定神经元连接关系。
所述步骤(3)、(10)中所述的复杂度指根据计算图的节点数量和边的数量来计算的复杂度。
所述步骤(10)所述超参搜索中超参指的是AI里面神经网络的控制参数，包括了学习率、即参数和/或权值衰减参数。
本发明的有益效果在于：
1、本发明的方法有广泛的使用场景：本发明的方法基于计算图的编码方式，可以实现对机器学习、深度学习的网络的统一编码，采用同一套的框架实现网络的自动化设计，在机器学习(如堆叠(Stacking)模式)和深度学习的神经网络都可以使用。
2、提升模型设计效率：本发明的方法比对相同模型，可以避免相同模型的被重复计算的次数，从而提升了模型设计效率。
3、保证多样性和采样空间的均匀分布，实现在局域最优范围内的搜索，同时实现跳出局部最优：本发明的方法可使用不同的算子——随机算子和交叉算子可以保证多样性和采样空间的均匀分布，变异算子可以实现在局域最优范围内的搜索的同时实现跳出局部最优的特点。
4、保证搜索的效率，防止搜索网络的性能的衰退：每代都保留最优模型，可以保证搜索的效率，防止搜索网络的性能的衰退。
5、本发明的方法根据模型数据进行评分，不需要通过实际数据进行训练就可以直接评估。
附图说明
图1为本发明实现的总流程图。
具体实施方式
下面结合附图和具体实施例对本发明作进一步说明。
如图1所示，本发明实现的步骤如下：
步骤(1)：根据用户预设的数据，进行数据准备，设置模型设计平台生产参数，开始模型自动设计。用户预设的数据，包括了数据的统计分布、数据维度间相关系数和/或数据各维度和标签之间的统计相关性。步骤(1)中设置的模型设计平台生产参数包括了计算资源、作业运行时间、作业目标如新一代计算图模型数量、进化结束条件，和/或遗传学算法参数——如种群个数、代数、变异、交叉及随机的所占百分比。所述作业目标包含了计算图模型的适应度阀值：包括了被认为满足进化结束条件的适应度阀值，和被认为是无效用模型的适应度阀值。本步骤中预设的具体数据根据实际生产中用户的需求而定。
步骤(2)：利用遗传算法算子生成第一代计算图模型；所述遗传算法算子包括了随机算子、交叉算子和/或变异算子。所述随机算子为随机选取神经元个数、随机选取神经元种类和/或随机决定神经元连接关系。
步骤(3)：
a、根据第一代计算图结构计算模型性能(比如其准确率)；
b、根据计算图性能和复杂度，计算每一个计算图模型的适应度。本步骤中所述的复杂度指根据计算图的节点数量和边的数量来计算的复杂度。
步骤(4)：根据模型适应度，去除无效用模型和重复模型，剩余模型作为备选模型，并保留作为下一代种子；
步骤(5)：根据步骤(4)保留的下一代种子，挑出若干数量最优模型；
步骤(6)：根据步骤(4)选择出的作为下一代种子的备选模型使用遗传算法算子产生一个计算图新模型。所述遗传算法算子包括了随机算子、交叉算子和/或变异算子。所述随机算子为随机选取神经元个数、随机选取神经元种类和/或随机决定神经元连接关系。
步骤(7)：判断步骤(6)产生的计算图新模型是否为已经生成过的计算图模型，如否，进入步骤(8)；如是，返回步骤(6)；
步骤(8)：保存步骤(5)、(7)的计算图模型为新一代计算图模型；
步骤(9)：判断是否步骤(8)的新一代计算图模型的数量满足步骤(1)中预设的数据，如是，进入下一步；如否，返回步骤(6)；
步骤(10)：
a、对生命周期超过三代的模型，则再进行寻找最优解或者靠近最优解的次优解的超参 搜索，所述生命周期超过三代的定义同遗传算法中“代”数的定义，自该模型结构第一次出现在本流程中开始算为第一代，超参搜索后保留下来的模型进入步骤(11)。模型超参数是模型外部的配置，其值不能从数据估计得到，而是通常由实践者直接指定，应用于估计模型参数的过程中，通常可以使用网格搜索、随机搜索、启发式搜索、贝叶斯搜索等方法来设置，根据给定的预测建模问题而调整。本步骤所述“超参搜索”(即超参数搜索)中的超参数指的是AI里面神经网络的控制参数，包括了学习率、即参数和/或权值衰减参数。
b、对生命周期没有超过三代的模型，根据计算图结构计算模型性能，根据计算图性能和复杂度计算每一个计算图模型的适应度，然后进入步骤(11)。本步骤中所述的复杂度指根据计算图的节点数量和边的数量来计算的复杂度。
步骤(11)：判断计算图新模型是否满足步骤(1)中预设的进化结束条件，预设的进化结束条件根据用户具体定义的内容，比如：准确率超过用户期望、用时达到用户设置的最大时长等，如满足，进入步骤(12)；如不满足，返回步骤(3b)；
步骤(12)：进化计算结果汇总，根据模型复杂度和准确率进行综合评分，选择出最优模型。
实施例一：
如步骤(1)所述，用户准备数值计算数据(采用csv格式或图片格式)，且数据中包标签列；设置模型最大进化代数为3代，每一代模型种群数量为5；预设置适应度越小模型性能越优；计算图模型的适应度阀值——如果最优模型适应度小于50则认为满足进化结束条件，计算停止；预设适应度超过1000的模型被认为是无效用模型。
如步骤(2)所述，利用遗传随机算子随机生成第一代5个模型，分别为：随机生成第一代5个模型，分别为：计算图模型1、计算图模型2、计算图模型3、计算图模型4和计算图模型5。
如步骤(3)所述，对计算图模型1、计算图模型2、计算图模型3、计算图模型4和计算图模型5的模型表达向量进行编码：
·计算图模型1:[op1-op2，op2-op3，op2-op4，…，op5-op6]
·计算图模型2:[op1-op2，op1-op3，op2-op4，…，op8-op9]
·计算图模型3:[op1-op2，op1-op3，op1-op4，…，op9-op10]
·计算图模型4:[op1-op2，op2-op3，op2-op4，…，op14-op15]
·计算图模型5:[op1-op2，op1-op3，op2-op3，…，op7-op8]
本实施例中，计算图的性能以计算图模型的准确率为衡量标准。分别计算计算图模型1、计算图模型2、计算图模型3、计算图模型4和计算图模型5的准确率(以下采用P来 表示准确率)，本实施例中：计算图模型1的P 1＝10，计算图模型2的P 2＝200，计算图模型3的P 3＝500，计算图模型4的P 4＝800，计算图模型5的P 5＝300。
计算复杂度(以下采用N来表示复杂度)，本实施例中：计算图模型1的N 1＝6，计算图模型2的N 2＝9，计算图模型3的N 3＝10，计算图模型4的N 4＝15，计算图模型5的N 5＝8。
计算每个计算图模型的适应度(以下采用F来表示适应度)，采用公式F＝P+10*N(适应度公式也可以采用其他公式)，本实施例中：计算图模型1的F 1＝70，计算图模型2的F 2＝290，计算图模型3的F 3＝600，计算图模型4的F 4＝1050，计算图模型5的F 5＝380。
如步骤(4)所述，去除无效用模型和重复模型，因上述计算图模型1-5没有重复模型，故去除重复模型，根据本实施例的预设，当适应度超过1000则认为是无效用模型，故去除相应计算图模型4；剩余模型对于计算图模型1、计算图模型2、计算图模型3和计算图模型5作为备选模型，并保留作为下一代种子。
如步骤(5)所述，计算图模型1中F最小，作为最优模型，保留为新一代计算图模型。
如步骤(6)所述，使用遗传随机算子生成计算图模型a，模型表达向量为：
[op1-op2，op1-op3，op2-op4，…，op11-op12]
所述步骤(7)所述，判断模型是否是已经生成过的计算图模型，模型a和已有模型(计算图模型1、计算图模型2、计算图模型3、计算图模型4和计算图模型5)是不相同或者性能不相似的，所以是新模型。计算图模型a的准确率P a＝250，复杂度N a＝12，适应度F a＝370。
如步骤(8)所述，把计算图模型a保存为新一代计算图模型6。
如步骤(9)所述，因为预设的每一代模型种群数量为5个，而目前只有计算图模型6这一个模型，所以不满足预设条件，应返回步骤(6)继续生成计算图模型。
当重复上述步骤(6)—(8)后，得到满足条件的新一代计算图模型7、计算图模型8、计算图模型9和计算图模型10后，种群数量达到5，则满足了预设数据，进入下一步。
如步骤(10)所述，目前保留下来的计算图模型不超过三代。已知计算图模型1的P 1＝10，N 1＝6，F 1＝70。计算步骤(9)产生的计算图模型6、计算图模型7、计算图模型8、计算图模型9和计算图模型10的适应度——计算图模型6的准确率P 6＝250，复杂度N 6＝12，适应度为F 6＝370；计算图模型7的准确率P 7＝810，复杂度N 7＝15，适应度为F 6＝998；计算图模型8的准确率P 8＝22，复杂度N 8＝8，适应度为F 8＝92；计算图模型9的准确率P 9＝42，复杂度N 9＝9，适应度为F 9＝130；计算图模型10的准确率P 10＝4，复杂度N 10＝5，适应度为F 10＝48。
如步骤(11)所述，因计算图模型10的适应度F 10＝48，满足适应度小于50进化条件。
如步骤(12)所述，“根据模型复杂度和性能进行评分”：本实施例的步骤(1)中预设了适应度越小模型性能越优，本实施例的适应度的计算采用公式F＝P+10*N。经比较， 计算图模型10的适应度最小，计算图模型10为最优模型。模型自动生成结束。
虽然在这里通过某个或某些特殊配置描述和阐明本发明，然而其目的并不在于限制所述细节，因为可能在专利要求范围内有各种修改和结构变更，并不偏离发明精神。
本发明涉及部分均与现有技术相同或可采用现有技术加以实现。

Claims (8)

  1. 一种基于计算图进化的AI模型自动生成的方法，其特征在于包括以下步骤：
    步骤(1)：根据用户预设的数据，进行数据准备，设置模型设计平台生产参数，开始模型自动设计；
    步骤(2)：利用遗传算法算子生成第一代计算图模型；
    步骤(3)：
    a、根据第一代计算图结构计算模型性能；
    b、根据计算图性能和复杂度，计算每一个计算图模型的适应度；
    步骤(4)：根据模型适应度，去除无效用模型和重复模型，剩余模型作为备选模型，并保留作为下一代种子；
    步骤(5)：根据步骤(4)保留的下一代种子，挑出若干数量最优模型；
    步骤(6)：根据步骤(4)选择出的作为下一代种子的备选模型使用遗传算法算子产生一个计算图新模型；
    步骤(7)：判断步骤(6)产生的计算图新模型是否为已经生成过的计算图模型，如否，进入步骤(8)；如是，返回步骤(6)；
    步骤(8)：保存步骤(5)、(7)的计算图模型为新一代计算图模型；
    步骤(9)：判断是否步骤(8)的新一代计算图模型的数量满足步骤(1)中预设的数据，如是，进入下一步；如否，返回步骤(6)；
    步骤(10)：
    a、对生命周期超过三代的模型，则再进行寻找最优解或者靠近最优解的次优解的超参搜索，所述生命周期超过三代的定义同遗传算法中“代”数的定义，自该模型结构第一次出现在本流程中开始算为第一代，超参搜索后保留下来的模型进入步骤(11)；
    b、对生命周期没有超过三代的模型，根据计算图结构计算模型性能，根据计算图性能和复杂度计算每一个计算图模型的适应度，然后进入步骤(11)；
    步骤(11)：判断计算图新模型是否满足步骤(1)中预设的进化结束条件，如满足，进入步骤(12)；如不满足，返回步骤(3b)；
    步骤(12)：进化计算结果汇总，根据模型复杂度和准确率进行综合评分，选择出最优模型。
  2. 如权利要求1所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述步骤(1)中用户预设的数据，包括了数据的统计分布、数据维度间相关系数和/或数据 各维度和标签之间的统计相关性。
  3. 如权利要求1所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述步骤(1)中设置的模型设计平台生产参数包括了计算资源、作业运行时间、作业目标和/或遗传学算法参数。
  4. 如权利要求3所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述作业目标包含了计算图模型的适应度阀值：包括了被认为满足进化结束条件的适应度阀值，和被认为是无效用模型的适应度阀值。
  5. 如权利要求1所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述步骤(2)、(6)所述遗传算法算子包括了随机算子、交叉算子和/或变异算子。
  6. 如权利要求5所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述随机算子为随机选取神经元个数、随机选取神经元种类和/或随机决定神经元连接关系。
  7. 如权利要求1所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述步骤(3)、(10)中所述的复杂度指根据计算图的节点数量和边的数量来计算的复杂度。
  8. 如权利要求1所述的基于计算图进化的AI模型自动生成的方法，其特征在于所述步骤(10)所述超参搜索中超参指的是AI里面神经网络的控制参数，包括了学习率、即参数和/或权值衰减参数。
PCT/CN2019/123267 2019-01-15 2019-12-05 基于计算图进化的ai模型自动生成的方法 WO2020147450A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910036186.9 2019-01-15
CN201910036186.9A CN109784497B (zh) 2019-01-15 2019-01-15 基于计算图进化的ai模型自动生成的方法

Publications (1)

Publication Number Publication Date
WO2020147450A1 true WO2020147450A1 (zh) 2020-07-23

Family

ID=66500583

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/123267 WO2020147450A1 (zh) 2019-01-15 2019-12-05 基于计算图进化的ai模型自动生成的方法

Country Status (2)

Country Link
CN (1) CN109784497B (zh)
WO (1) WO2020147450A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109784497B (zh) * 2019-01-15 2020-12-25 探智立方(北京)科技有限公司 基于计算图进化的ai模型自动生成的方法
CN110276442B (zh) * 2019-05-24 2022-05-17 西安电子科技大学 一种神经网络架构的搜索方法及装置
CN110766072A (zh) * 2019-10-22 2020-02-07 探智立方(北京)科技有限公司 一种基于结构相似性的计算图进化ai模型自动生成方法
CN114626284A (zh) * 2020-12-14 2022-06-14 华为技术有限公司 一种模型处理方法及相关装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160275413A1 (en) * 2015-03-20 2016-09-22 Xingtian Shi Model vector generation for machine learning algorithms
CN106067028A (zh) * 2015-04-19 2016-11-02 北京典赞科技有限公司 基于gpu的自动化机器学习的建模方法
CN108334949A (zh) * 2018-02-11 2018-07-27 浙江工业大学 一种优化深度卷积神经网络结构的快速进化方法
CN109784497A (zh) * 2019-01-15 2019-05-21 探智立方(北京)科技有限公司 基于计算图进化的ai模型自动生成的方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7526460B2 (en) * 2004-09-16 2009-04-28 Neal Solomon Mobile hybrid software router
CN102800107B (zh) * 2012-07-06 2015-04-22 浙江工业大学 一种基于改进最小交叉熵的运动目标检测方法
CN102930291B (zh) * 2012-10-15 2015-04-08 西安电子科技大学 用于图形图像的k近邻局部搜索遗传自动聚类方法
CN106339756B (zh) * 2016-08-25 2019-04-02 北京百度网讯科技有限公司 训练数据的生成方法、搜索方法以及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160275413A1 (en) * 2015-03-20 2016-09-22 Xingtian Shi Model vector generation for machine learning algorithms
CN106067028A (zh) * 2015-04-19 2016-11-02 北京典赞科技有限公司 基于gpu的自动化机器学习的建模方法
CN108334949A (zh) * 2018-02-11 2018-07-27 浙江工业大学 一种优化深度卷积神经网络结构的快速进化方法
CN109784497A (zh) * 2019-01-15 2019-05-21 探智立方(北京)科技有限公司 基于计算图进化的ai模型自动生成的方法

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ATYUN: "Automatic Machine Learning: Optimizing Recurrent Neural Networks Using Genetic Algorithms)", HTTP://WWW.YIDIANZIXUN.COM/ARTICLE/0IFCZKFR, 29 January 2018 (2018-01-29) *
QIU, LULU: "AutoML Startup Cube Discovery: Model's Natural Choice and Survival of the Fittest", HTTPS://MP.WEIXIN.QQ.COM/S/-QLLEDRJGJ6CWQGP1OEULA, 18 July 2018 (2018-07-18), pages 1 - 11, XP055719603 *

Also Published As

Publication number Publication date
CN109784497A (zh) 2019-05-21
CN109784497B (zh) 2020-12-25

Similar Documents

Publication Publication Date Title
WO2020147450A1 (zh) 基于计算图进化的ai模型自动生成的方法
Esmin et al. Hybrid evolutionary algorithm based on PSO and GA mutation
Wagner et al. Optimizing the layout of 1000 wind turbines
CN107317699B (zh) 一种云制造服务组合的动态蚁群快速优选方法
CN105138717A (zh) 一种动态变异粒子群优化神经网络的变压器状态评估方法
CN110858805A (zh) 小区网络流量预测方法及装置
CN112966445B (zh) 一种基于强化学习模型fqi的水库防洪优化调度方法
CN115940294B (zh) 多级电网实时调度策略调整方法、系统、设备及存储介质
CN111917134B (zh) 一种基于数据驱动的配电网动态自主重构方法及系统
Xu et al. A fuzzy operator based bat algorithm for cloud service composition
CN108734349B (zh) 基于改进遗传算法的分布式电源选址定容优化方法及系统
CN114648178B (zh) 一种基于ddpg算法的电能计量装置运维策略优化方法
CN110705756B (zh) 一种基于输入凸神经网络的电力能耗优化控制方法
CN112380006A (zh) 一种数据中心资源分配方法及装置
CN105956680B (zh) 一个基于强化学习的自适应规则的生成和管理框架
CN110766072A (zh) 一种基于结构相似性的计算图进化ai模型自动生成方法
CN115528750B (zh) 一种面向电网安全稳定的数据模型混合驱动机组组合方法
CN110929851A (zh) 基于计算图子图的ai模型自动生成的方法
CN112182835A (zh) 一种考虑风电不确定性和储能调节的电力系统可靠性评估方法及系统
Huang et al. Short-term load forecasting based on the improved bas optimized elman neural network
Sharma et al. Halton based initial distribution in artificial bee colony algorithm and its application in software effort estimation
Aydın et al. A configurable generalized artificial bee colony algorithm with local search strategies
CN116523327A (zh) 一种基于强化学习的配电网运行策略智能生成方法及设备
CN110555742A (zh) 一种发电商代理报价的生成方法与系统
CN114298429A (zh) 一种配电网动案辅助决策方法、系统、装置和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19910202

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 161121)

122 Ep: pct application non-entry in european phase

Ref document number: 19910202

Country of ref document: EP

Kind code of ref document: A1