CN107766927A - Universal parallel method of the intelligent optimization algorithm based on individual population on Spark - Google Patents

Universal parallel method of the intelligent optimization algorithm based on individual population on Spark Download PDF

Info

Publication number
CN107766927A
CN107766927A CN201711072393.7A CN201711072393A CN107766927A CN 107766927 A CN107766927 A CN 107766927A CN 201711072393 A CN201711072393 A CN 201711072393A CN 107766927 A CN107766927 A CN 107766927A
Authority
CN
China
Prior art keywords
individual
population
interface
class
fitness
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711072393.7A
Other languages
Chinese (zh)
Inventor
邢焕来
周芯宇
杨慧
李可
叶佳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest Jiaotong University
Original Assignee
Southwest Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest Jiaotong University filed Critical Southwest Jiaotong University
Priority to CN201711072393.7A priority Critical patent/CN107766927A/en
Publication of CN107766927A publication Critical patent/CN107766927A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses universal parallel method of the intelligent optimization algorithm based on individual population on Spark, includes the individual abstract base class of encapsulation, the abstract base class and intermediate data format class of population；Setting individual realize class and population realize class；Initialize population, individual and global optimum's individual；Initialization context context, generate broadcasting amount and the broadcast of constant information；Parallelization generates Species structure formula data set；Fitness optimum individual is returned after each population is by evolving alone, population's fitness is assessed；Find out contemporary optimum individual and renewal global optimum individual；Data distribution formula data set is exchanged among parallelization generation；Carry out the steps such as the individual exchange between population.The beneficial effects of the present invention are, for the intelligent optimization algorithm based on individual population, a kind of universal parallel method is proposed on Spark platforms, script is operated in the algorithm on unit and is distributed on different computers while runs, to reduce run time, raising efficiency.

Description

Universal parallel method of the intelligent optimization algorithm based on individual-population on Spark
Technical field
The present invention relates to distributed computing technology field, particularly a kind of intelligent optimization algorithm based on individual-population exists Universal parallel method on Spark.
Background technology
Having a kind of intelligent optimization algorithm is come by the Biology seed coating or analog physical phenomenon of natural imitation circle, such calculation Method can under receivable expense (including time and space) provide problem to be solved a good quality feasible solution.Such Intelligent optimization algorithm is also referred to as meta-heuristic algorithm, more commonly used to have genetic algorithm, particle cluster algorithm, ant group algorithm, artificial Ant colony algorithm etc..It is not very perfect in such theory of algorithm, it is impossible to ensure to find problem most compared to traditional optimized algorithm Excellent solution, but such algorithm wide adaptability, the engineering neck within the acceptable range, widely applied in reality can be controlled by seeking solution expense In domain.Such intelligent optimization algorithm have one it is notable and common the characteristics of, have individual and population concept.Population is individual Set, and individual represents a solution of problem, by cooperating between some behaviors of individual or individual, promotes whole Individual population develops toward more preferable direction, developed, so as to search out the more excellent solution of problem.
As the complexity increase of problem, scale increase, solved and asked using such intelligent optimization algorithm under stand-alone environment The operation of topic is time-consuming also to be increased.In order to improve the execution efficiency of such algorithm, run time is reduced, meets actual answer With needs, algorithm can be transformed in itself to improve performance, but its reforming mode of different algorithms is different, even same Algorithm for it is different the problem of also have a various mutation, difficulty is larger.Another method is exactly to carry out parallelization to algorithm Processing, script is operated in the algorithm on unit and be distributed on different computers while run, reduce run time, lifting effect Rate.
With cloud computing, big data flourish, various distributed computing frameworks are suggested, Spark platforms It is exactly one of them, underlying basis has been established for parallelization.
The content of the invention
It is an object of the invention to provide a kind of universal parallel of intelligent optimization algorithm based on individual-population on Spark Change method.
Realize that the technical scheme of the object of the invention is as follows：
Universal parallel method of the intelligent optimization algorithm based on individual-population on Spark, including step：
(1) the abstract base class BaseIndividual of encapsulation individual, the abstract base class BasePopulation of population is encapsulated, Encapsulate the intermediate data format class ExchangeData exchanged between population for individual；
(2) set the individual of the intelligent optimization algorithm and realize class SomeIndividual, set the intelligent optimization The population of algorithm realizes class SomePopulation；The SomeIndividual is inherited from BaseIndividual, SomePopulation is inherited from BasePopulation；
(3) M population, M >=2 are initialized；Initialize individual in each population, N >=100；Initialize global optimum Body gbest, maximum iteration MAX_T is initialized, if current iteration number t=1；
(4) initialization context context SparkContext, the generation of SparkContext.broadcast () interface is called The broadcasting amount of constant information, it is broadcast in all calculate nodes on Spark platforms；
(5) SparkContext.parallelize () interface is called by M population object parallel, kind corresponding to generation Group's distributed data collection PopulationRDD；
(6) PopulationRDD.map () interface is called to perform parallel computation；The PopulationRDD.map () connects In mouthful, it is individual that each population returns to the optimal preceding K of fitness in the population by evolving alone, after population's fitness assessment successively Body, wherein K >=2；
(7) PopulationRDD.collect () interface is called, the K individual that each population returns is collected, therefrom finds out Contemporary optimum individual cbest；If cbest fitness is better than gbest, gbest is updated with cbest；
(8) the K individual returned to each population is packaged, and generation M is used for the intermediate data that individual exchanges, then right M intermediate data carries out upsetting mixing, then calls SparkContext.parallelize () interface to be carried out to it parallel Change, exchange data distribution formula data set ExchangeRDD corresponding to generation among individual；
(9) PopulationRDD.zipPartitions () interface and incoming ExchangeRDD are called, carry out population it Between individual exchange；
(10) t=t+1, if t<MAX_T, step (6) is gone to, otherwise exit circulation, export gbest.
The beneficial effects of the present invention are for the intelligent optimization algorithm based on individual-population, carry on Spark platforms A kind of universal parallel method is gone out, script is operated in the algorithm on unit and be distributed to same luck on different computers OK, to reduce run time, raising efficiency.
Brief description of the drawings
Fig. 1 is the flow chart of the present invention；
Fig. 2 is the BaseIndividual class figures in the specific embodiment of the invention；
Fig. 3 is the BasePopulation class figures in the specific embodiment of the invention；
Fig. 4 is the ExchangeData class figures in the specific embodiment of the invention；
Fig. 5 is the GAIndividual class figures in the specific embodiment of the invention；
Fig. 6 is the GAPopulation class figures in the specific embodiment of the invention.
Embodiment
The invention will be further described below in conjunction with the accompanying drawings.
As shown in figure 1, universal parallel method of the intelligent optimization algorithm based on individual-population on Spark, including step Suddenly：
(1) the abstract base class BaseIndividual of encapsulation individual, the abstract base class BasePopulation of population is encapsulated, Encapsulate the intermediate data format class ExchangeData exchanged between population for individual；
(2) set the individual of the intelligent optimization algorithm and realize class SomeIndividual, set the intelligent optimization The population of algorithm realizes class SomePopulation；The SomeIndividual is inherited from BaseIndividual, SomePopulation is inherited from BasePopulation；
(3) M population, M >=2 are initialized；Initialize individual in each population, N >=100；Initialize global optimum Body gbest, maximum iteration MAX_T is initialized, if current iteration number t=1；
(4) initialization context context SparkContext, the generation of SparkContext.broadcast () interface is called The broadcasting amount of constant information, it is broadcast in all calculate nodes on Spark platforms；
(5) SparkContext.parallelize () interface is called by M population object parallel, kind corresponding to generation Group's distributed data collection PopulationRDD；
(6) PopulationRDD.map () interface is called to perform parallel computation；The PopulationRDD.map () connects In mouthful, it is individual that each population returns to the optimal preceding K of fitness in the population by evolving alone, after population's fitness assessment successively Body, wherein K >=2；
(7) PopulationRDD.collect () interface is called, the K individual that each population returns is collected, therefrom finds out Contemporary optimum individual cbest；If cbest fitness is better than gbest, gbest is updated with cbest；
(8) the K individual returned to each population is packaged, and generation M is used for the intermediate data that individual exchanges, then right M intermediate data carries out upsetting mixing, then calls SparkContext.parallelize () interface to be carried out to it parallel Change, exchange data distribution formula data set ExchangeRDD corresponding to generation among individual；
(9) PopulationRDD.zipPartitions () interface and incoming ExchangeRDD are called, carry out population it Between individual exchange；
(10) t=t+1, if t<MAX_T, step (6) is gone to, otherwise exit circulation, export gbest.
In above-mentioned steps,
In step (1), class BaseIndividual has 4 attributes, is respectively：Fitness maximum MAX_FITNESS= 999999.0, fitness minimum value MIN _ FITNESS=-999999.0, the current fitness value fitness of the individual, problem Type mark isMinProblem.If isMinProblem=true, represent to solve is minimization problem, and what is otherwise solved is Maximization problems.During individual initialization, if isMinProblem=true, fitness=MAX_FITNESS, otherwise Fitness=MIN_FITNESS.
In step (1), class BasePopulation has 3 attributes, is respectively：Individual amount individual_num, it is individual Body object container individuals, problem types mark isMinProblem.Class BasePopulation has 1 abstraction interface, ExchangeIndividuals (ExchangeData), the incoming data of the interface is intermediate data format, and its major function is just It is to realize the individual exchange between population, the interface realizes class to rewrite and realize by the population of derivative specific optimized algorithm.
In step (1), class ExchangeData has 2 attributes, is respectively：Using mark used_flag, individual subject is held Device data.Data storages are transmitted to some population and a series of individual subjects for exchanging.If used_flag=true, represent The intermediate data was used by some population, skips and does not use, otherwise can be used.
In step (2), the individual of certain specific optimized algorithm realizes that class SomeIndividual is inherited from BaseIndividual, SomeIndividual must according to particular problem to be solved coded system reasonable in design, and according to The relevant information and coded system of problem provide Fitness analysis interface.SomeIndividual is also needed according to problem information and volume Code mode provides suitable initialization interface.
In step (2), the population of certain specific optimized algorithm realizes that class SomePopulation is inherited from BasePopulation, SomePopulation must provide initialization interface according to particular problem information to be solved, and according to The concrete property of this kind of optimized algorithm and its core operator, design specific member property and operator interface. SomePopulation must provide the Fitness analysis interface of population according to specific problem information, in the interface, call SomeIndividual Fitness analysis interface, calculate the fitness value of each individual in the population.SomePopulation In must rewrite BasePopulation exchangeIndividuals (ExchangeData) interface, realize specific kind Individual exchanged form between group.
In step (3), the population of initialization, refer to that the population of specific optimized algorithm realizes class SomePopulation. The individual initialized in population, refer to that the individual of specific optimized algorithm realizes class SomeIndividual.Global optimum's individual Gbest is also class SomeIndividual object.
In step (4), SparkContext is the environmental context that Spark platforms provide, and is run on Spark platforms The essential component of program, specific to code aspect, it can be understood as the class that Spark platforms provide.
In step (5), M kind group object after initialization is stored in an array, is called When SparkContext.parallelize () interface is to its parallelization, it is M to set the parallelization number of partitions.
In step (5), the full names of RDD are flexible type distributed data collection, are a kind of distributed datas that Spark platforms provide Collection.RDD can carry out distributed encapsulation to the data of localization, and some parallelization operations are carried out on RDD.Specific to generation Code layer face, RDD are also the class that Spark is provided.SparkContext.parallelize () interface is called to create RDD, to the RDD of M population object reference SparkContext.parallelize () interface generation, it is named as PopulationRDD。
In step (6), each population is evolved alone, refers to calling the population of specific optimized algorithm to realize class SomePopulation relational operator interface, update each individual in whole population and population.Each population is adapted to Degree is assessed, and refers to calling SomePopulation Fitness analysis interface, calculates the fitness value of each individual in population. Preferably K individual of fitness is returned, can be adjusted according to specific algorithm and problem, such as, return and sorted by fitness value Preceding 20% individual afterwards, or preceding 30% individual.
In step (7), each population returns to K individual, and M population returns to altogether M × K individual.All returned Body is stored in a two-dimensional array, and two-dimensional array length is M, wherein each element is one-dimension array, represents a population All individuals returned, one-dimension array length is K, wherein each element represents an individual.
In step (8), intermediate data, class ExchangeData is referred to.Each intermediate data encapsulates a population and returned The K individual returned, M population generate M intermediate data.In an array, each element represents M intermediate data storage One intermediate data, after upsetting mixing to the order of element in the array, SparkContext.parallelize () is called to connect When mouth is to its parallelization, it is M to set the parallelization number of partitions.
In step (9), the individual exchange between population, that is, the population of specific optimized algorithm is called to realize class ExchangeIndividuals (ExchangeData) interface after SomePopulation rewritings, wherein, ExchangeData The K individual returned containing certain another population, guarantor is interacted or screens according to certain strategy with the individual that current population possesses Stay, realize the individual exchange between population.
Here is a specific embodiment.
So that genetic algorithm (abbreviation GA) solves the problems, such as multicast routing optimization as an example, to illustrate the specific reality of universal parallelization Applying method.
Multicast routing problem model describes：Communication subnet is modeled to a undirected cum rights connected graph G=(V, E), V= {v1, v2..., vNRepresent node set, E={ e1, e2..., eMRepresent line set between connecting node, N and M difference table Show the number on node and side.S ∈ V are multicast source nodes,It is multicast destination node set, K Represent destination node number.To each edge e ∈ E, the expense that cost (e) is side e, i.e. weights on the e of side are defined.With source node s For root node, the one tree T={ V comprising all purposes set of node DT, ET, T is multicast tree, wherein VTIt is all nodes in T Set, ETIt is the set on all sides in T,Multicast tree T is a G subgraph.Multicast routing problem is A multicast tree T=(V is found in GT, ET) so that multicast tree T expenseIt is minimum.Multicast path What is asked by problem is minimal-overhead multicast tree of the source node to destination node collection, i.e. expense is smaller, the Xie Yueyou of problem, is minimum Change problem.
Multicast routing problem suppositive scenario：Possess 200 nodes, the connected graph communication subnet on 446 sides, wherein, every Overhead value on side takes the random value in (0,50).Multicast source node be numbering 5 node, multicast destination node collection be [13, 25,36,46,55,70,90,108,127,167,192,101,3,66]。
First to the abstract base class BaseIndividual, the abstract base class BasePopulation of population, centre of individual Data format class ExchangeData explains.
BaseIndividual classes are defined as follows Fig. 2.
Wherein,
MAX_FITNESS：Fitness maximum, is initialized as 999999.0.
MIN_FITNESS：Fitness minimum value, is initialized as -999999.0.
isMinProblem：Problem types to be solved, true is taken to represent minimization problem, otherwise maximization problems.
fitness：The current fitness value of individual, if minimization problem, is initialized as MAX_FITNESS, otherwise initially Turn to MIN_FITNESS.
BasePopulation classes definition such as Fig. 3.
Wherein,
individual_num：Individual quantity in the population.
individuals：The container of individual subject, store all individuals in the population.
isMinProblem：Problem types to be solved, value are same as above.
exchangeIndividuals(ExchangeData)：Abstraction interface, it is ExchangeData classes pair to be passed to parameter As, there is provided individual function of exchange between population, it is necessary to it is simultaneously real to rewrite that class is realized by the population of derivative specific optimized algorithm The now interface.
ExchangeData classes definition such as Fig. 4.
Wherein,
used_flag：Using mark, represent that the intermediate data was used by some population for true.
data：Individual subject container, storage will be transmitted to some population and a series of individual subjects for exchanging.
Below, according to specific multicast routing problem scene and GA algorithm characteristic, the individual for encapsulating GA realizes class and kind Group realizes class.If GA individual realizes that class is defined as GAIndividual (SomeIndividual in corresponding above flow), GA population realizes that class is defined as GAPopulation (SomePopulation in corresponding above flow), below it is right GAIndividual and GAPopulation are described in detail.
GAIndividual classes definition such as Fig. 5.
GAIndividual is inherited from BaseIndividual, possesses BaseIndividual all publicly-owned member's category Property, wherein,
gene：The gene code of individual is represented, according to multicast routing problem suppositive scenario, takes the binary system encoded by side Coded system.Specifically, there are 446 sides in problem scenes, then gene is the array that a length is 446, wherein each member The corresponding a line of element, the value of each element is true or false, represents that this side is selected, available for true, is false Represent that this side is not selected, unavailable.
init()：Individual initialization interface, according to the input information of multicast routing problem, initialize each of the individual subject Individual member property.
calcFitness()：Individual adaptation degree calculates interface, according to the input information of multicast routing problem and individual itself Gene code gene, calculate the current fitness value of the individual subject.Specifically, if problem scenes are originally inputted topology For G=(V, E), wherein, V is the set of node of 200 nodes, E be 446 while while collection, the individual gene gene=[l1, l2,...,l446], each of which position takes true or false, if lk=true, k ∈ { 1,2 ..., 446 }, then by lkCorresponding side It is added to new side collection En, newly-generated side collectionNew topological G is generated on hereinn=(V, En), in new topological GnOn, Multicast source node be 5, multicast destination node collection for [13,25,36,46,55,70,90,108,127,167,192,101,3, 66], it is source node with 5, EnIn it is each while overhead value for while on weights, run dijkstra's algorithm, if source node 5 with it is any Destination node is connected, then dijkstra's algorithm is run successfully, and source node 5 to all purposes node totally 14 is included in acquired results Bar shortest path, all collection E when forming new after duplicate removal included in 14 shortest pathsm, the individual is calculated by formula Fitness value：Wherein, cost (e) represents the overhead value on certain side e.If source node 5 and some Destination node does not connect, then dijkstra's algorithm operation failure, and the individual is illegal individual, makes its fitness fitness= MAX_FITNESS。
GAPopulation classes definition such as Fig. 6.
GAPopulation is inherited from BasePopulation, possesses BasePopulation all publicly-owned member properties And interface, wherein,
exchangeIndividuals(ExchangeData)：This for having rewritten abstract parent BasePopulation connects Mouthful, the function that individual exchanges between concrete implementation population.Specifically, it is assumed that contain certain another in ExchangeData 20% individual that population is transmitted, because multicast routing problem is minimization problem, ideal adaptation angle value is smaller more excellent, so to current All individuals of population carry out sequence from small to large according to fitness value, discard maximum rear 20% individual of fitness value, All individuals in ExchangeData are incorporated into current population again, individual is completed and exchanges.
populationInit()：Initialization of population interface, each member property of the population is initialized, and initialize and be somebody's turn to do All individuals in population.
fitnessEvaluate()：Population's fitness assesses interface, calls individual adaptation degree to calculate interface successively, calculates The fitness value of each individual in population.
crossover()：One of crossover operator interface, GA core operator, specific practice be, travels through population, every time from Two Different Individuals are randomly selected in population, two individuals are exchanged according to certain crossover probability, using certain interleaved mode Gene code, so as to generate new individual.Available interleaved mode has single-point intersection, multiple-spot detection, uniform crossover.
variation()：One of mutation operator interface, GA core operator, specific practice are population to be traveled through, to each Each gene position in individual, mutation operation is done according to certain mutation probability.For example, a certain position takes in certain individual gene True, the position takes false after mutation operation.
select()：One of selection opertor interface, GA core operator, specific practice is, by the way of championship, match Select fitness preferably individual in population and entail the next generation.
It is as follows that GA solves the step of universal parallel method of above-mentioned multicast routing problem scene：
(1) relevant information of multicast routing problem scene, including node, side, source node, destination node collection information are inputted.
(2) 6 GAPopulation kind group objects are initialized, in each population, initialize 100 GAIndividual Body object.Initialize global optimum individual gbest, maximum iteration MAX_T=200, current iteration number t=1.
(3) SparkContext is initialized, to the relevant information of multicast routing problem, such as node, side information, source node, Destination node collection information, call SparkContext.broadcast () to generate its broadcasting amount, be broadcast on Spark platforms In all calculate nodes.
(4) SparkContext.parallelize () is called by 6 GAPopulation population object parallels, generation Corresponding PopulationRDD.During parallelization, it is 6 to set its parallelization number of partitions, i.e. the corresponding population of a subregion.
(5) PopulationRDD.map () interface is called to perform parallel computation, in the interface, each population enters alone Change, Fitness analysis simultaneously return to fitness preferably some individuals, and specifically, the sub-step performed in the interface is as follows：
1st, GAPopulation.crossover (), crossover operation
2nd, GAPopulation.variation (), mutation operation
3rd, GAPopulation.fitnessEvaluate (), population's fitness are assessed
4th, optimal preceding 20 individuals of fitness are recorded
5th, GAPopulation.select (), selection operation
6th, the individual recorded in sub-step 4 is returned
(6) PopulationRDD.collect () is called, collecting 20 individuals that each population returns, (6 populations return altogether Return 120 individuals), contemporary optimum individual cbest is therefrom found out, if cbest fitness value is less than gbest, cbest is more It is excellent, update gbest with cbest.
(7) 120 individuals returned to 6 populations are packaged, and generate 6 ExchangeData class objects (among i.e. Data), 20 individuals are included in each ExchangeData, mixing then are upset to 6 ExchangeData order, then adjust With SparkContext.parallelize () to its parallelization, ExchangeRDD corresponding to generation.During parallelization, equally set It is 6 to put its parallelization number of partitions, control step (4), i.e. the corresponding intermediate data of the corresponding population of a subregion.
(8) PopulationRDD.zipPartitions () interface and incoming ExchangeRDD are called, in the interface, The individual exchange between population is realized, specifically, the sub-step performed in the interface is as follows：
1st, GAPopulation.exchangeIndividuals (ExchangeData), individual exchange is carried out
(9) t=t+1, if t<MAX_T, step (5) is gone to, otherwise exit circulation, export gbest, and obtain under gbest Multicast tree.

Claims (1)

1. universal parallel method of the intelligent optimization algorithm based on individual-population on Spark, it is characterised in that including step Suddenly：
(1) the abstract base class BaseIndividual of encapsulation individual, the abstract base class BasePopulation of population, encapsulation are encapsulated It is used for the intermediate data format class ExchangeData that individual exchanges between population；
(2) set the individual of the intelligent optimization algorithm and realize class SomeIndividual, set the intelligent optimization algorithm Population realize class SomePopulation；The SomeIndividual is inherited from BaseIndividual, SomePopulation is inherited from BasePopulation；
(3) M population, M >=2 are initialized；Initialize individual in each population, N >=100；Initialize global optimum's individual Gbest, maximum iteration MAX_T is initialized, if current iteration number t=1；
(4) initialization context context SparkContext, SparkContext.broadcast () interface generation constant is called The broadcasting amount of information, it is broadcast in all calculate nodes on Spark platforms；
(5) SparkContext.parallelize () interface is called by M population object parallel, population corresponding to generation point Cloth data set PopulationRDD；
(6) PopulationRDD.map () interface is called to perform parallel computation；The PopulationRDD.map () interface In, it is individual that each population returns to the optimal preceding K of fitness in the population by evolving alone, after population's fitness assessment successively Body, wherein K >=2；
(7) PopulationRDD.collect () interface is called, the K individual that each population returns is collected, therefrom finds out the present age Optimum individual cbest；If cbest fitness is better than gbest, gbest is updated with cbest；
(8) the K individual returned to each population is packaged, and generation M is used for the intermediate data that individual exchanges, then to M Intermediate data carries out upsetting mixing, then calls SparkContext.parallelize () interface to carry out parallelization to it, raw Data distribution formula data set ExchangeRDD is exchanged among into corresponding individual；
(9) call PopulationRDD.zipPartitions () interface and be passed to ExchangeRDD, between progress population Individual exchanges；
(10) t=t+1, if t<MAX_T, step (6) is gone to, otherwise exit circulation, export gbest.
CN201711072393.7A 2017-11-03 2017-11-03 Universal parallel method of the intelligent optimization algorithm based on individual population on Spark Pending CN107766927A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711072393.7A CN107766927A (en) 2017-11-03 2017-11-03 Universal parallel method of the intelligent optimization algorithm based on individual population on Spark

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711072393.7A CN107766927A (en) 2017-11-03 2017-11-03 Universal parallel method of the intelligent optimization algorithm based on individual population on Spark

Publications (1)

Publication Number Publication Date
CN107766927A true CN107766927A (en) 2018-03-06

Family

ID=61272173

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711072393.7A Pending CN107766927A (en) 2017-11-03 2017-11-03 Universal parallel method of the intelligent optimization algorithm based on individual population on Spark

Country Status (1)

Country Link
CN (1) CN107766927A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109871934A (en) * 2019-01-16 2019-06-11 湖北工业大学 Feature selection approach based on the distributed parallel binary of Spark a flying moth darts into the fire algorithm
CN110086855A (en) * 2019-03-29 2019-08-02 江苏开拓信息与系统有限公司 Spark task Intellisense dispatching method based on ant group algorithm

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109871934A (en) * 2019-01-16 2019-06-11 湖北工业大学 Feature selection approach based on the distributed parallel binary of Spark a flying moth darts into the fire algorithm
CN110086855A (en) * 2019-03-29 2019-08-02 江苏开拓信息与系统有限公司 Spark task Intellisense dispatching method based on ant group algorithm
CN110086855B (en) * 2019-03-29 2021-11-05 江苏开拓信息与系统有限公司 Intelligent Spark task perception scheduling method based on ant colony algorithm

Similar Documents

Publication Publication Date Title
Pierre et al. A genetic algorithm for designing distributed computer network topologies
CN110287031A (en) A method of it reducing distributed machines and learns communication overhead
Abdel-Kader Hybrid discrete PSO with GA operators for efficient QoS-multicast routing
CN105677648B (en) A kind of Combo discovering method and system based on label propagation algorithm
US20060003823A1 (en) Dynamic player groups for interest management in multi-character virtual environments
CN105719000A (en) Neuron hardware structure and method of simulating pulse neural network by adopting neuron hardware structure
CN109995580B (en) VN mapping method based on GA _ PSO hybrid algorithm in 5G network slice
Hajimirsadeghi et al. A hybrid IWO/PSO algorithm for fast and global optimization
CN106776796A (en) Based on cloud computing and big data unmanned plane task grouping and method
CN107911300B (en) Multicast routing optimization method based on whale algorithm and application of multicast routing optimization method on Spark platform
CN106022469A (en) Method for scalable neurosynaptic networks
CN108846602A (en) A kind of logistics transportation dispatching method and device based on quantum fireworks algorithm
CN103281211B (en) Large-scale network node system for managing in groups and management method
CN108413963A (en) Bar-type machine people&#39;s paths planning method based on self study ant group algorithm
CN113312177B (en) Wireless edge computing system and optimizing method based on federal learning
CN107766927A (en) Universal parallel method of the intelligent optimization algorithm based on individual population on Spark
CN110263236A (en) Social network user multi-tag classification method based on dynamic multi-view learning model
Abdel-Kader An improved discrete PSO with GA operators for efficient QoS-multicast routing
CN116915746B (en) IPv6 addressing method based on Internet of things
CN108880909A (en) A kind of network energy-saving method and device based on intensified learning
CN107169561A (en) Towards the hybrid particle swarm impulsive neural networks mapping method of power consumption
Vijayalakshmi et al. Artificial immune based hybrid GA for QoS based multicast routing in large scale networks (AISMR)
CN108898527A (en) A kind of traffic data fill method based on the generation model for having loss measurement
Roshani et al. Parallel genetic algorithm for shortest path routing problem with collaborative neighbors
CN114298244A (en) Decision control method, device and system for intelligent agent group interaction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180306

WD01 Invention patent application deemed withdrawn after publication