CN110321938A - A kind of state space construction method and device of Intelligent unattended cluster - Google Patents
A kind of state space construction method and device of Intelligent unattended cluster Download PDFInfo
- Publication number
- CN110321938A CN110321938A CN201910539923.7A CN201910539923A CN110321938A CN 110321938 A CN110321938 A CN 110321938A CN 201910539923 A CN201910539923 A CN 201910539923A CN 110321938 A CN110321938 A CN 110321938A
- Authority
- CN
- China
- Prior art keywords
- intelligent unattended
- cluster
- node
- state
- intelligent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/043—Architecture, e.g. interconnection topology based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
Abstract
The present invention provides a kind of state space construction method and device of Intelligent unattended cluster, this method comprises: according between Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node relative distance and relative positional relationship formed have ordinal number to collection, the state at any one moment when the operation of each Intelligent unattended node is described；The state at any one moment when being run according to each Intelligent unattended node, constructs the state space of Intelligent unattended cluster.Multiple shot array characteristic of the present invention for the continuity variation and ambient condition of Intelligent unattended cluster oneself state, using between Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node relative distance and relative positional relationship formed have ordinal number to collection, to describe the state of each Intelligent unattended node in Intelligent unattended cluster, and then construct the state space of Intelligent unattended cluster, the relationship for specifying node state and cluster state, facilitates expression and calculating.
Description
Technical field
The present invention relates to Intelligent unattended control technology fields, and in particular to a kind of state space construction of Intelligent unattended cluster
Method and device.
Background technique
Under specific natural environment scene, the behavior collaboration work between one group of Intelligent unattended node of appointed task is completed
Make, is called Intelligent unattended cluster.With the continuous extension of application field, communication, control and the cooperation of Intelligent unattended cluster
Difficulty will be significantly increased, and it is current urgently to be resolved for how efficiently controlling each Intelligent unattended node collaboration completion task therein
Problem, good synergistic mechanism can be improved the flexibility of Intelligent unattended clustered control, improve communication efficiency, guarantee the height of task
Effect is reliably completed.
The behavior collaboration of Intelligent unattended cluster is varied, such as the collaboration search and rescue of Intelligent unattended machine, collaborative navigation, machinery
The collaboration carrying etc. of the collaborative assembly and mobile robot or unmanned plane of arm.It is cooperateed with currently for Intelligent unattended Aggregation behaviour
Research mostly be based on control theory propose solution, there is no using intensified learning method solve Aggregation behaviour collaboration
Problem, and the solution based on intensified learning more focuses on feedback mechanism, the more conducively Collaborative Control to Intelligent unattended node.
Although work compound type is different, the behavior collaboration that its essence is all Intelligent unattended node is studied carefully, that is,
The change of the system mode of Intelligent unattended cluster.Thus, in intensified learning method it needs to be determined that the state space of cluster, and show
There is a kind of method in technology still without state space for constructing Intelligent unattended cluster.
Summary of the invention
The embodiment of the present invention provides a kind of state space construction method and device of Intelligent unattended cluster, to solve existing skill
When solving Aggregation behaviour Research on Interactive Problem using intensified learning method in art, need to construct asking for the state space of Intelligent unattended cluster
Topic.
In a first aspect, the embodiment of the present invention provides a kind of state space construction method of Intelligent unattended cluster, the method
Include:
According to the relative distance and phase between Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node
There is ordinal number to collection to what positional relationship was formed, describes the state at any one moment when the operation of each Intelligent unattended node；
The state at any one moment, constructs the Intelligent unattended cluster when being run according to each Intelligent unattended node
State space.
As the preferred embodiment of first aspect present invention, described any one moment when the operation of each Intelligent unattended node
State when, i-th of Intelligent unattended node is described in the state of t moment by following equation:
N is
The total number of Intelligent unattended node in Intelligent unattended cluster；
Wherein, { (d1,o1),…,(di-1,oi-1),(di+1,oi+1),…,(dn,on) it is to indicate the in Intelligent unattended cluster
Relative distance and relative positional relationship between i Intelligent unattended node and remaining n-1 Intelligent unattended node have ordinal number pair
Collection, dj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate i-th of Intelligent unattended node and j-th in Intelligent unattended cluster
Relative distance between Intelligent unattended node；oj=αi+αj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate Intelligent unattended
Relative positional relationship in cluster between i-th of Intelligent unattended node and j-th of Intelligent unattended node, αiIndicate i-th of intelligence
The line direction of the directional velocity of unmanned node and i-th of Intelligent unattended node and j-th of Intelligent unattended node is along side clockwise
To angle, αjIndicate the directional velocity and i-th of Intelligent unattended node and j-th of Intelligent unattended of j-th of Intelligent unattended node
The angle of the line direction of node along clockwise direction.
As the preferred embodiment of first aspect present invention, when constructing the state space of the Intelligent unattended cluster, under
The state space of Intelligent unattended cluster is described in column formula:
As the preferred embodiment of first aspect present invention, the method also includes:
By Adaptive Fuzzy Neural-network clustering method, the state space of the Intelligent unattended cluster is clustered,
State space after generating the Intelligent unattended cluster cluster.
It is described by Adaptive Fuzzy Neural-network clustering method as the preferred embodiment of first aspect present invention, to institute
The state space for stating Intelligent unattended cluster is clustered, and the state space after generating the Intelligent unattended cluster cluster includes:
The state of any one Intelligent unattended node outside the determining sample state set currently newly obtained and the sample
Relative distance in state set between the state of each Intelligent unattended node, and the sample is determined according to each relative distance
The weight of the state of each Intelligent unattended node in state set；
According to ECM clustering algorithm, pass through the weight pair of the state of each Intelligent unattended node in the sample state set
The state of each Intelligent unattended node is classified in presently described Intelligent unattended cluster, generates at least one cluster, described poly-
The parameter of class includes cluster centre and cluster radius；
Using the cluster centre of the cluster and cluster radius as the center of fuzzy membership function and width, and benefit
The parameter in the fuzzy membership function can be solved with gradient descent algorithm；
State space to the fuzzy membership function de-fuzzy, after generating the Intelligent unattended cluster cluster.
Second aspect, the embodiment of the present invention provide a kind of state space construction device of Intelligent unattended cluster, described device
Include:
State description unit, for according to Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node it
Between relative distance and relative positional relationship formed have ordinal number to collection, describe any one when the operation of each Intelligent unattended node
The state at a moment；
Space construction unit, the state at any one moment when for being run according to each Intelligent unattended node, building
The state space of the Intelligent unattended cluster.
As the preferred embodiment of second aspect of the present invention, the state description unit describes each Intelligent unattended node fortune
When row when the state at any one moment, i-th of Intelligent unattended node is retouched in the state of t moment by following equation
It states:
N is
The total number of Intelligent unattended node in Intelligent unattended cluster；
Wherein, { (d1,o1),…,(di-1,oi-1),(di+1,oi+1),…,(dn,on) it is to indicate the in Intelligent unattended cluster
Relative distance and relative positional relationship between i Intelligent unattended node and remaining n-1 Intelligent unattended node have ordinal number pair
Collection, dj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate i-th of Intelligent unattended node and j-th in Intelligent unattended cluster
Relative distance between Intelligent unattended node；oj=αi+αj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate Intelligent unattended
Relative positional relationship in cluster between i-th of Intelligent unattended node and j-th of Intelligent unattended node, αiIndicate i-th of intelligence
The line direction of the directional velocity of unmanned node and i-th of Intelligent unattended node and j-th of Intelligent unattended node is along side clockwise
To angle, αjIndicate the directional velocity and i-th of Intelligent unattended node and j-th of Intelligent unattended of j-th of Intelligent unattended node
The angle of the line direction of node along clockwise direction.
As the preferred embodiment of second aspect of the present invention, the space construction unit constructs the shape of the Intelligent unattended cluster
When state space, it is described by state space of the following equation to Intelligent unattended cluster:
As the preferred embodiment of second aspect of the present invention, described device further include:
Space clustering unit, for passing through Adaptive Fuzzy Neural-network clustering method, to the Intelligent unattended cluster
State space is clustered, the state space after generating the Intelligent unattended cluster cluster.
As the preferred embodiment of second aspect of the present invention, the space clustering unit is specifically used for:
The state of any one Intelligent unattended node outside the determining sample state set currently newly obtained and the sample
Relative distance in state set between the state of each Intelligent unattended node, and the sample is determined according to each relative distance
The weight of the state of each Intelligent unattended node in state set；
According to ECM clustering algorithm, pass through the weight pair of the state of each Intelligent unattended node in the sample state set
The state of each Intelligent unattended node is classified in presently described Intelligent unattended cluster, generates at least one cluster, described poly-
The parameter of class includes cluster centre and cluster radius；
Using the cluster centre of the cluster and cluster radius as the center of fuzzy membership function and width, and benefit
The parameter in the fuzzy membership function can be solved with gradient descent algorithm；
State space to the fuzzy membership function de-fuzzy, after generating the Intelligent unattended cluster cluster.This
The state space construction method and device for the Intelligent unattended cluster that inventive embodiments provide, for Intelligent unattended cluster oneself state
Continuity variation and ambient condition multiple shot array characteristic, using Intelligent unattended node each in Intelligent unattended cluster and remaining intelligence
What relative distance and relative positional relationship between the unmanned node of energy were formed has ordinal number to collection, every in Intelligent unattended cluster to describe
The state of a Intelligent unattended node, and then the state space of Intelligent unattended cluster is constructed, specify node state and cluster state
Relationship, facilitate expression and calculating.
Thus, it is possible to establish the local behavior cooperation model of Intelligent unattended cluster based on this, and then propose using fuzzy strong
Change the research approach that learning algorithm solves the collaboration of Intelligent unattended Aggregation behaviour.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is a kind of process signal of state space construction method of Intelligent unattended cluster provided in an embodiment of the present invention
Figure；
Fig. 2 is a kind of state description schematic diagram of Intelligent unattended node provided in an embodiment of the present invention；
Fig. 3 is a kind of relative positional relationship schematic diagram of Intelligent unattended node provided in an embodiment of the present invention；
Fig. 4 is a kind of state space initial clustering situation schematic diagram of Intelligent unattended cluster provided in an embodiment of the present invention；
Fig. 5 is a kind of structural representation of the state space construction device of Intelligent unattended cluster provided in an embodiment of the present invention
Figure.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.
The embodiment of the invention discloses a kind of state space construction methods of Intelligent unattended cluster, shown referring to Fig.1, should
Method specifically includes that
101, according to the relative distance between Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node
There is ordinal number to collection with what relative positional relationship was formed, describes the state at any one moment when the operation of each Intelligent unattended node；
102, the state at any one moment when being run according to each Intelligent unattended node, constructs the state of Intelligent unattended cluster
Space.
In a step 101, collaboration be in cluster some or certain several Intelligent unattended nodes relative to remaining Intelligent unattended
The process of the relativeness adjustment of node, this is the process of a dynamic consecutive variations, portrays cluster and environment for clarity
Current state can use one group of continuous state StIndicate collaborative variation process, wherein t is the time, state is to move at any time
It moves.
Preferably, when describing the state at any one moment when the operation of each Intelligent unattended node, by following equation to i-th
A Intelligent unattended node is described in the state of t moment:
N is
The total number of Intelligent unattended node in Intelligent unattended cluster；
Wherein, { (d1,o1),…,(di-1,oi-1),(di+1,oi+1),…,(dn,on) it is to indicate the in Intelligent unattended cluster
Relative distance and relative positional relationship between i Intelligent unattended node and remaining n-1 Intelligent unattended node have ordinal number pair
Collection, dj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate i-th of Intelligent unattended node and j-th in Intelligent unattended cluster
Relative distance between Intelligent unattended node；oj=αi+αj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate Intelligent unattended
Relative positional relationship in cluster between i-th of Intelligent unattended node and j-th of Intelligent unattended node, αiIndicate i-th of intelligence
The line direction of the directional velocity of unmanned node and i-th of Intelligent unattended node and j-th of Intelligent unattended node is along side clockwise
To angle, αjIndicate the directional velocity and i-th of Intelligent unattended node and j-th of Intelligent unattended of j-th of Intelligent unattended node
The angle of the line direction of node along clockwise direction.
Referring to shown in Fig. 2, Fig. 2 shows i-th of Intelligent unattended node and jth Intelligent unattendeds in Intelligent unattended cluster
Relative distance and relative positional relationship between node.According to the safe condition of Intelligent unattended node itself and sensing data
It can sensing capability, it is known that the relative distance d between two Intelligent unattended nodes is a continuous variable, therefore relative distance d is carved
What is drawn is a continuous state.
Referring to shown in Fig. 3, the phase between i-th of Intelligent unattended node and j-th of Intelligent unattended node is had been shown in particular in Fig. 3
To positional relationship, these four situations of a, b, c, d shown in Fig. 3 can be divided into.As seen from the figure, for i-th of Intelligent unattended section
The description of relative positional relationship o, can intuitively see a in the state of pointiAnd ajRange be respectively { ai∈ [0 °, 360 °),
aj∈ [0 °, 360 °) }, so o ∈ [0 °, 720 °).Relative position between Intelligent unattended node is also a continuous variable, is carved
What is drawn is a continuous state.
Due in Intelligent unattended cluster i-th of Intelligent unattended node in the state of t momentIt is by two continuous states
(dk, ok) indicate, wherein k=1,2 ..., i-1, i+1 ..., n, and k！=i, thereforeIt is similarly continuous state.
In a step 102, when being run according to each Intelligent unattended node in the Intelligent unattended cluster obtained in above-mentioned steps
The state at any one moment further constructs the state space of Intelligent unattended cluster.
Preferably, when constructing the state space of Intelligent unattended cluster, by following equation to Intelligent unattended cluster in t moment
State space be described:
In above formula, StIt is a n-dimensional vector, indicates the state space of Intelligent unattended cluster, whereinI=1,2 ..., n
The state at any one moment when being the operation of each Intelligent unattended node.
Since the state space of the above-mentioned Intelligent unattended cluster constructed is also continuously, size is with Intelligent unattended
The variation of the total number n of node and continually changing, the problem of being easy to cause dimension calamity.Moreover, when changing Intelligent unattended node
Number when need to redefine the state space of Intelligent unattended cluster, this is infeasible in general Fuzzy Reinforcement Learning
's.Since time and computing resource are limited, the state space of Intelligent unattended cluster is uncertain, and the behavior of Intelligent unattended node
Space can not reduce, and be equal to reduce the ability of Intelligent unattended node because reducing action space, it cannot be guaranteed that task quilt
It executes completely.Therefore, in order to improve the convergence and convergence rate of Fuzzy Reinforcement Learning, it is necessary to press its state space
Contracting.
Preferably, this method further comprises:
103, by Adaptive Fuzzy Neural-network clustering method, the state space of Intelligent unattended cluster is clustered,
State space after generating Intelligent unattended cluster cluster.
In step 103, (Adaptive Fuzzy Neural Network is clustered using Adaptive Fuzzy Neural-network
Clustering, AFNNC) method carries out the compression of the state space of Intelligent unattended cluster.It is outer locating for Intelligent unattended cluster
Boundary's environment is usually complicated and changeable, and influences each other between each factor, cross-coupling, perception of the Intelligent unattended node for ambient condition
Also point-device measuring and calculating is hardly resulted in, describes this problem well using the fuzzy logic table Danone in AFNNC method.
In addition, AFNNC method combines the study optimization ability of neural network again, which is reconstructed according to training data
Network structure, adjustment parameter simultaneously generate corresponding fuzzy rule.
Preferably, in one possible implementation, step 103 can be embodied as follows:
1031, the state and sample of any one Intelligent unattended node outside the sample state set currently newly obtained are determined
Relative distance in state set between the state of each Intelligent unattended node, and sample state is determined according to each relative distance
The weight of the state of each Intelligent unattended node in set；
1032, according to ECM clustering algorithm, pass through the weight pair of the state of Intelligent unattended node each in sample state set
The state of each Intelligent unattended node is classified in current Intelligent unattended cluster, generates at least one cluster, the parameter of cluster
Including cluster centre and cluster radius；
1033, using the cluster centre of cluster and cluster radius as the center of fuzzy membership function and width, and
The parameter in fuzzy membership function can be solved using gradient descent algorithm；
1034, the state space to fuzzy membership function de-fuzzy, after generating Intelligent unattended cluster cluster.
The specific implementation process of step 103 for ease of understanding below will open up in detail above-mentioned steps 1031~1034
Open explanation:
(1) Euclidean distance between two vectors x and y is defined first are as follows:
X in formula, y ∈ RP, wherein P indicates the length of sequence, and has ‖ x-y ‖ ∈ [0,1].
(2) Intelligent unattended cluster has the generation polymerization of part Intelligent unattended node and forms partial status collection in initial operating stage
It closes, sample state set can also be called.Wherein, NqIndicate the quantity of Intelligent unattended node in sample state set, q < n, sample
The state of all Intelligent unattended nodes in this state set is in the Intelligent unattended node outside sample state set
It is all to close on state.
Any one Intelligent unattended node outside the sample state set newly obtained is calculated using Euclidean distance formula (1-1)
xiCurrent state xqWith the N in sample state setqThe relative distance d of a Intelligent unattended node closed between state=
[d1,d2,…,dn], N hereqNumber basis for selecting experience determine, then each Intelligent unattended node in sample state set
The weight of state may be expressed as:
wi=1- (di-mini(d)), i=1,2 ..., Nq (1-2)
In formula, diIndicate an Intelligent unattended node x outside sample state setiCurrent state to sample state set
Middle NqThe relative distance of a Intelligent unattended node closed between state, miniIt (d) is relative distance d=[d1,d2,…,dn] in
Minimum value.
(3) passed through using ECM (Evolving Clustering Method) clustering algorithm each in sample state set
The weight of the state of Intelligent unattended node clusters the state of each Intelligent unattended node outside sample state set, tool
Body is as follows:
A, an Intelligent unattended node is chosen simply from Intelligent unattended cluster first to cluster as firstIt is poly-
Class centerAnd classification radius at this timeIt is set as 0, continuously performs n times, determines the shape of n Intelligent unattended node
The cluster of state
B, the Intelligent unattended node x outside the sample state set newly obtained is calculatediCurrent state with the n that has determined
The cluster centre C of clusterCjRelative distance d (i, j), which can be calculated by formula (1-1):
D (i, j)=| | xi-CCj| |, j=1,2 ..., n. (1-3)
If c, the relative distance d (i, j) that formula (1-2) is calculated is no more than at least one cluster in existing cluster
When cluster radius, by the Intelligent unattended node x outside the sample state set newly obtainediIt is merged into poly- with the shortest distance with it
In class, i.e.,
D (i, m)=‖ xi-CCm‖=min (| | xi-CCj| |), j=1,2 ..., n, (1-4)
In formula,Indicate the Intelligent unattended for having in cluster and outside the sample state set newly obtained
Node xiApart from the smallest cluster radius.Step b is gone to after the completion of sorting out to continue to calculate the intelligence outside next sample state set
It can unmanned node xi+1。
If the Intelligent unattended node x outside sample state set d, newly obtainediSo that existing cluster occur to update or
Person is unsatisfactory for above situation, then needs to calculate Intelligent unattended node xiAt a distance between the cluster for needing to update, and and threshold value
It is compared judgement, new cluster is then established if it is greater than twice of threshold value, otherwise just by Intelligent unattended node xiIt is included into it
In preceding ready-portioned cluster:
The minimum range that selection formula (1-5) is calculated is defined as s (i, a), by clustering CaWith its cluster radiusTable
Show as follows:
Need to consider there are two types of situation at this time, first is that: when s (i, a) > 2D when, need to establish new cluster Cnew, cluster
Radius isSecond is that: when s (i, a)≤2D when, need to update cluster CaWith its cluster radiusMore
New cluster isCluster radius is
E, it either establishes new cluster and still updates existing cluster, cluster centre is in the sample state set newly obtained
Intelligent unattended node x outside conjunctioniOnto original cluster centre line, and new cluster centre is to the sample state set newly obtained
Outer Intelligent unattended node xiDistance be equal to cluster radius, then by Intelligent unattended node xiIt is ready-portioned poly- before being included into
In class, step b is gone to after the completion of sorting out and continues to calculate Intelligent unattended node x outside next sample state seti+1, Zhi Daosuo
Intelligent unattended node clustering outside some sample state sets terminates.
(4) according to ECM clustering algorithm in previous step obtain as a result, using the cluster centre of cluster as fuzzy membership letter
Several centers, cluster radius is as its width, it may be assumed that
In formula (1-7), GijFor the output of fuzzy membership function, wherein xijFor i-th in Intelligent unattended cluster intelligent nothing
J-th of state value of people's node, mijAnd σijJ-th of state value of i-th of Intelligent unattended node respectively in Intelligent unattended cluster
The mean value and variance of corresponding fuzzy membership function, n are the number of Intelligent unattended node in Intelligent unattended cluster, and l is rule
Number.
(5) fuzzy rule is constructed, form is as follows:
Rl:IF x1is Fl1andx2is l2and…xn is FlpTHEN y=nl, (1-8)
F in formulaljIt is fuzzy set, is defined by the fuzzy membership function in formula (1-7).Its output can indicate are as follows:
nl=bl0+bl1x1+bl2x2+…+blpxp, (1-9)
Using the center method of average of optimization for the Intelligent unattended node x outside the sample state set that newly obtainsiState xi
=[x1,x2..., xp] de-fuzzy, output are as follows:
L is regular number in formula, and p is the state number of the Intelligent unattended node outside the sample state set newly obtained, is utilized
Gradient descent algorithm can find out the parameter alpha in fuzzy membership functionlj、mijAnd σij。
bl0(k+1)=bl0(k)-ηbwiΦ(xi)[f(k)(xi)-ti] (1-11)
blj(k+1)=blj(k)-ηbwiΦ(xi)[f(k)(xi)-ti] (1-12)
In formula, Φ (xi) are as follows:
η in formulab, ηα, ηmAnd ησRespectively parameter bj, αlj, mljAnd σljLearning rate.All kinds of following tables respectively indicate:
Wherein, i is the number of Intelligent unattended node in Intelligent unattended cluster, i=1,2 ..., N；
The dimension of the Intelligent unattended node outside sample state set newly obtained is j, j=1,2 ..., P；
M indicates fuzzy rule number, l=1,2 ..., M；
Iteration step length is k, k=1,2 ....
Finally, the state space after the Intelligent unattended cluster cluster of generation is S after end of clusteringt=(d, o, k).
In addition, an index that can describe Exist Network Structure performance is cluster global error, E is usediIt indicates, intelligent nothing
The weighted error function of people's cluster can be calculated with following formula.
W in formulaiBy being calculated in formula (1-2), wiIndicate the power of the state of each Intelligent unattended node in sample state set
Weight.
In fact, being exactly the classification to state space to the compression of state space, i.e., certain states are closed using classifier
And be a kind of state, realize that the dynamic of state space divides, this can promote the convergence and convergence rate of Fuzzy Reinforcement Learning, adds
Fast pace of learning.
After being explained further and carrying out cluster compression to the state space of Intelligent unattended cluster described in the embodiment of the present invention
As a result, will be described in detail below with specific example.
For i-th of Intelligent unattended node in Intelligent unattended cluster, using the node as reference, by its own
State space is divided into St=(d, o, k), k here indicates the quantity in Intelligent unattended node in this state, by institute
There is the target of Intelligent unattended node identical, therefore all Intelligent unattended nodes under same state can take identical behavior, i.e.,
In the case where state space determines, the behavior of Intelligent unattended node also determines therewith.
After carrying out cluster compression to the state space of Intelligent unattended cluster, can will originally continuous state space it is discrete
Change, to facilitate the description of problem.According to each Intelligent unattended node state in which can it is wrong slightly by any two Intelligent unattended
Relative distance d between node is divided into three kinds of states, and use -1 indicates precarious position respectively, and 0 indicates safe condition, and 1 expression can
Adjustment state.It is as follows:
In formula, R indicates the inherently safe distance between two Intelligent unattended nodes.
Then, defining relative positional relationship is eight sections, and angular range divides as follows:
After completing cluster, the continuous state of Intelligent unattended node is discretized, and for the shape of each Intelligent unattended node
State size is 3 × 8 × (n-1), then the state space size of Intelligent unattended cluster is (3 × 8 × (n-1))n。
Referring to shown in Fig. 4 (a), the state space of Intelligent unattended cluster can be divided into 24, each shape after cluster
The number of Intelligent unattended node may be different under state space.Referring to shown in Fig. 4 (b), Intelligent unattended cluster is indicated at a time
State, have 1 and 2 the two Intelligent unattended nodes in A class, there is 3 and 4 the two Intelligent unattended nodes in B class, have 5 in C class,
6 and 7 these three Intelligent unattended nodes do not have node in other classes.
State space after compressed Intelligent unattended cluster cluster is as shown in following table 1-1, at this point for an intelligent nothing
The state space of people's cluster can gather for following 3 × 8=24 class.
State space after the cluster of table 1-1 Intelligent unattended cluster
In table, ki=0,1 ..., n (i=1,2 ..., 24), and have k1+k2+…+k24=n.
By cluster result it is found that having k Intelligent unattended in hypothesis relative distance and relative positional relationship (d, o) at this time
Node, at this time it can be seen that the value of k is only there are two types of situation: first is that, when k=0, indicates do not have Intelligent unattended section under such state
Point, does not need co-operating；Second is that k > 0 indicates there be k Intelligent unattended node under such state, there is k Intelligent unattended node
When need make collaboration, due to each Intelligent unattended node need to complete common target and be in identical state, can be by this k
A Intelligent unattended node is classified as one kind, can be indicated with 1, can obtain this k Intelligent unattended node according to the cluster consistency principle and adopt
With identical movement, then the state space of Intelligent unattended cluster is represented by St=(d, o, k), wherein { -1,0,1 } d ∈, o ∈
{ 1,2,3,4,5,6,7,8 }, k ∈ { 0,1 }.Here (d, o) shares 3 × 8=24 kind state, k to cluster state is different
It describes under cluster state herein with the presence or absence of Intelligent unattended node, i.e. k=0 or k=1.
At this point, the state space of Intelligent unattended cluster can be converted into 24 integer representations, wherein each value
It is 0 or 1, that is to say, that one 24 binary number representations, the i.e. size of state space can be used are as follows: 224=16,777,
216.For n Intelligent unattended node in Intelligent unattended cluster, using relative distance and opposite position when state description
Relationship is set, can be indicated using identical Q matrix:
It should be noted that for simple description, therefore, it is stated as a series of for the embodiment of the above method
Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described.Secondly,
Those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, related movement
It is not necessarily essential to the invention.
Based on the same inventive concept, the embodiment of the invention also discloses a kind of state space construction of Intelligent unattended cluster dresses
It sets, referring to Figure 5, the device mainly includes:
State description unit 51, for according to Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node
Between relative distance and relative positional relationship formed have ordinal number to collection, any one when operation of each Intelligent unattended node is described
The state at moment；
Space construction unit 52, the state at any one moment when for being run according to each Intelligent unattended node, constructs intelligence
The state space of the unmanned cluster of energy.
Preferably, when state description unit 51 describes the state at any one moment when the operation of each Intelligent unattended node, lead to
It crosses following equation and i-th of Intelligent unattended node is described in the state of t moment:
N is
The total number of Intelligent unattended node in Intelligent unattended cluster；
Wherein, { (d1,o1),…,(di-1,oi-1),(di+1,oi+1),…,(dn,on) it is to indicate the in Intelligent unattended cluster
Relative distance and relative positional relationship between i Intelligent unattended node and remaining n-1 Intelligent unattended node have ordinal number pair
Collection, dj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate i-th of Intelligent unattended node and j-th in Intelligent unattended cluster
Relative distance between Intelligent unattended node；oj=αi+αj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate Intelligent unattended
Relative positional relationship in cluster between i-th of Intelligent unattended node and j-th of Intelligent unattended node, αiIndicate i-th of intelligence
The line direction of the directional velocity of unmanned node and i-th of Intelligent unattended node and j-th of Intelligent unattended node is along side clockwise
To angle, αjIndicate the directional velocity and i-th of Intelligent unattended node and j-th of Intelligent unattended of j-th of Intelligent unattended node
The angle of the line direction of node along clockwise direction.
Preferably, when space construction unit 52 constructs the state space of Intelligent unattended cluster, by following equation to intelligence
The state space of unmanned cluster is described:
Preferably, the device further include:
Space clustering unit 53, for passing through Adaptive Fuzzy Neural-network clustering method, to the shape of Intelligent unattended cluster
State space is clustered, the state space after generating Intelligent unattended cluster cluster.
Preferably, space clustering unit 53 is specifically used for:
Determine the state and sample state of any one Intelligent unattended node outside the sample state set currently newly obtained
Relative distance in set between the state of each Intelligent unattended node, and sample state set is determined according to each relative distance
In each Intelligent unattended node state weight；
According to ECM clustering algorithm, by the weight of the state of Intelligent unattended node each in sample state set to current
The state of each Intelligent unattended node is classified in Intelligent unattended cluster, generates at least one cluster, and the parameter of cluster includes
Cluster centre and cluster radius；
Using the cluster centre of cluster and cluster radius as the center of fuzzy membership function and width, and utilize ladder
Degree descent algorithm can solve the parameter in fuzzy membership function；
State space to fuzzy membership function de-fuzzy, after generating Intelligent unattended cluster cluster.
In conclusion the state space construction method and device of Intelligent unattended cluster provided in an embodiment of the present invention, for
The multiple shot array characteristic of the continuity variation and ambient condition of Intelligent unattended cluster oneself state, using each in Intelligent unattended cluster
What relative distance and relative positional relationship between Intelligent unattended node and remaining Intelligent unattended node were formed has ordinal number to collection, comes
The state of each Intelligent unattended node in Intelligent unattended cluster is described, and then constructs the state space of Intelligent unattended cluster, it is clear
The relationship of node state and cluster state, facilitates expression and calculating.Thus, it is possible to establish the office of Intelligent unattended cluster based on this
Portion's behavior cooperation model, and then propose the research side that the collaboration of Intelligent unattended Aggregation behaviour is solved using fuzzy reinforcement algorithm
Case.
It should be noted that the state space construction device of Intelligent unattended cluster provided in an embodiment of the present invention and aforementioned reality
The state space construction method for applying Intelligent unattended cluster described in example belongs to identical technical concept, and specific implementation process can join
According to, to the explanation of method and step, details are not described herein in previous embodiment.
It should be appreciated that the state space construction device of one of the above Intelligent unattended cluster include unit only according to this set
The logical partitioning that the function that standby device is realized carries out in practical application, can carry out the superposition or fractionation of said units.And it should
A kind of function realized of state space construction device for Intelligent unattended cluster that embodiment provides with it is provided by the above embodiment
A kind of state space construction method one-to-one correspondence of Intelligent unattended cluster, the more detailed processing stream realized for the device
Journey has been described in detail in above method embodiment, is not described in detail herein.
The state space construction method and device of Intelligent unattended cluster provided in an embodiment of the present invention, for Intelligent unattended collection
The multiple shot array characteristic of the continuity variation and ambient condition of group's oneself state, using Intelligent unattended section each in Intelligent unattended cluster
What relative distance and relative positional relationship between point and remaining Intelligent unattended node were formed has ordinal number to collection, to describe intelligent nothing
The state of each Intelligent unattended node in people's cluster, and then the state space of Intelligent unattended cluster is constructed, specify node state
With the relationship of cluster state, facilitate expression and calculating.Thus, it is possible to establish the local behavior collaboration of Intelligent unattended cluster based on this
Model, and then propose the research approach that the collaboration of Intelligent unattended Aggregation behaviour is solved using fuzzy reinforcement algorithm.
It will be understood by those skilled in the art that realizing that all or part of the steps of above-mentioned each method embodiment can pass through journey
Sequence instructs relevant hardware to complete.Program above-mentioned can be stored in a computer readable storage medium.The program exists
When execution, execution includes the steps that above-mentioned each method embodiment, and storage medium above-mentioned includes ROM, RAM, magnetic disk or light
The various media that can store program code such as disk.
In the above embodiment of the invention, it all emphasizes particularly on different fields to the description of each embodiment, does not have in some embodiment
The part of detailed description, reference can be made to the related descriptions of other embodiments.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (10)
1. a kind of state space construction method of Intelligent unattended cluster, which is characterized in that the described method includes:
According to the relative distance between Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node and with respect to position
That sets relationship formation has ordinal number to collection, describes the state at any one moment when the operation of each Intelligent unattended node；
The state at any one moment, constructs the state of the Intelligent unattended cluster when being run according to each Intelligent unattended node
Space.
2. the method according to claim 1, wherein describing any one when operation of each Intelligent unattended node
When the state at moment, i-th of Intelligent unattended node is described in the state of t moment by following equation:
N is the total number of Intelligent unattended node in Intelligent unattended cluster；
Wherein, { (d1, o1) ..., (di-1, oi-1), (di+1, oi+1) ..., (dn, on) it is to indicate in Intelligent unattended cluster i-th
Relative distance and relative positional relationship between Intelligent unattended node and remaining n-1 Intelligent unattended node have ordinal number to collection,
dj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate i-th of Intelligent unattended node and j-th of intelligence in Intelligent unattended cluster
Relative distance between the unmanned node of energy；oj=αi+αj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate Intelligent unattended collection
Relative positional relationship in group between i-th of Intelligent unattended node and j-th of Intelligent unattended node, αiIndicate i-th of intelligent nothing
The line direction of the directional velocity of people's node and i-th of Intelligent unattended node and j-th of Intelligent unattended node is along clockwise direction
Angle, αjIndicate the directional velocity and i-th of Intelligent unattended node and j-th of Intelligent unattended section of j-th of Intelligent unattended node
The angle of the line direction of point along clockwise direction.
3. according to the method described in claim 2, it is characterized in that, leading to when constructing the state space of the Intelligent unattended cluster
Following equation is crossed the state space of Intelligent unattended cluster is described:
4. method described in any one of claim 1 to 3, which is characterized in that the method also includes:
By Adaptive Fuzzy Neural-network clustering method, the state space of the Intelligent unattended cluster is clustered, is generated
State space after the Intelligent unattended cluster cluster.
5. according to the method described in claim 4, it is characterized in that, described by Adaptive Fuzzy Neural-network clustering method,
The state space of the Intelligent unattended cluster is clustered, the state space packet after generating the Intelligent unattended cluster cluster
It includes:
Determine the state and the sample state of any one Intelligent unattended node outside the sample state set currently newly obtained
Relative distance in set between the state of each Intelligent unattended node, and the sample state is determined according to each relative distance
The weight of the state of each Intelligent unattended node in set；
According to ECM clustering algorithm, by the weight of the state of each Intelligent unattended node in the sample state set to current
The state of each Intelligent unattended node is classified in the Intelligent unattended cluster, generates at least one cluster, the cluster
Parameter includes cluster centre and cluster radius；
Using the cluster centre of the cluster and cluster radius as the center of fuzzy membership function and width, and utilize ladder
Degree descent algorithm can solve the parameter in the fuzzy membership function；
State space to the fuzzy membership function de-fuzzy, after generating the Intelligent unattended cluster cluster.
6. a kind of state space construction device of Intelligent unattended cluster, which is characterized in that described device includes:
State description unit, for according between Intelligent unattended node each in Intelligent unattended cluster and remaining Intelligent unattended node
What relative distance and relative positional relationship were formed has ordinal number to collection, when describing when the operation of each Intelligent unattended node any one
The state at quarter；
Space construction unit, the state at any one moment when for being run according to each Intelligent unattended node, described in building
The state space of Intelligent unattended cluster.
7. device according to claim 6, which is characterized in that the state description unit describes each Intelligent unattended section
When point operation when the state at any one moment, the state by following equation to i-th of Intelligent unattended node in t moment is carried out
Description:
N is the total number of Intelligent unattended node in Intelligent unattended cluster；
Wherein, { (d1, o1) ..., (di-1, oi-1), (di+1, oi+1) ..., (dn, on) it is to indicate in Intelligent unattended cluster i-th
Relative distance and relative positional relationship between Intelligent unattended node and remaining n-1 Intelligent unattended node have ordinal number to collection,
dj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate i-th of Intelligent unattended node and j-th of intelligence in Intelligent unattended cluster
Relative distance between the unmanned node of energy；oj=ai+aj(j=1 ..., i-1, i+1 ..., n, and j！=i) indicate Intelligent unattended collection
Relative positional relationship in group between i-th of Intelligent unattended node and j-th of Intelligent unattended node, αiIndicate i-th of intelligent nothing
The line direction of the directional velocity of people's node and i-th of Intelligent unattended node and j-th of Intelligent unattended node is along clockwise direction
Angle, αjIndicate the directional velocity and i-th of Intelligent unattended node and j-th of Intelligent unattended section of j-th of Intelligent unattended node
The angle of the line direction of point along clockwise direction.
8. device according to claim 7, which is characterized in that the space construction unit constructs the Intelligent unattended cluster
State space when, be described by state space of the following equation to Intelligent unattended cluster:
9. the device according to any one of claim 6~8, which is characterized in that described device further include:
Space clustering unit, for passing through Adaptive Fuzzy Neural-network clustering method, to the state of the Intelligent unattended cluster
Space is clustered, the state space after generating the Intelligent unattended cluster cluster.
10. device according to claim 9, which is characterized in that the space clustering unit is specifically used for:
Determine the state and the sample state of any one Intelligent unattended node outside the sample state set currently newly obtained
Relative distance in set between the state of each Intelligent unattended node, and the sample state is determined according to each relative distance
The weight of the state of each Intelligent unattended node in set；
According to ECM clustering algorithm, by the weight of the state of each Intelligent unattended node in the sample state set to current
The state of each Intelligent unattended node is classified in the Intelligent unattended cluster, generates at least one cluster, the cluster
Parameter includes cluster centre and cluster radius；
Using the cluster centre of the cluster and cluster radius as the center of fuzzy membership function and width, and utilize ladder
Degree descent algorithm can solve the parameter in the fuzzy membership function；
State space to the fuzzy membership function de-fuzzy, after generating the Intelligent unattended cluster cluster.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910539923.7A CN110321938B (en) | 2019-06-20 | 2019-06-20 | State space construction method and device of intelligent unmanned cluster |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910539923.7A CN110321938B (en) | 2019-06-20 | 2019-06-20 | State space construction method and device of intelligent unmanned cluster |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110321938A true CN110321938A (en) | 2019-10-11 |
| CN110321938B CN110321938B (en) | 2022-10-11 |
Family
ID=68121101
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910539923.7A Active CN110321938B (en) | 2019-06-20 | 2019-06-20 | State space construction method and device of intelligent unmanned cluster |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110321938B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101825901A (en) * | 2010-03-31 | 2010-09-08 | 北京航空航天大学 | Multi-agent robot cooperative control method based on artificial physics method |
| CN102096415A (en) * | 2010-12-31 | 2011-06-15 | 重庆邮电大学 | Multi-robot formation method based on Ad-Hoc network and leader-follower algorithm |
| CN102749847A (en) * | 2012-06-26 | 2012-10-24 | 清华大学 | Cooperative landing method for multiple unmanned aerial vehicles |
| CN103197684A (en) * | 2013-04-25 | 2013-07-10 | 清华大学 | Method and system for cooperatively tracking target by unmanned aerial vehicle cluster |
| CN103631141A (en) * | 2013-12-11 | 2014-03-12 | 北京航空航天大学 | Light transmission hypothesis based intensive autonomous aerial vehicle formation control method |
| CN106295613A (en) * | 2016-08-23 | 2017-01-04 | 哈尔滨理工大学 | A kind of unmanned plane target localization method and system |
| CN108983823A (en) * | 2018-08-27 | 2018-12-11 | 安徽农业大学 | A kind of plant protection drone cluster cooperative control method |
| CN109343966A (en) * | 2018-11-01 | 2019-02-15 | 西北工业大学 | A kind of cluster organization method and device of unmanned node |
| CN109445456A (en) * | 2018-10-15 | 2019-03-08 | 清华大学 | A kind of multiple no-manned plane cluster air navigation aid |
| CN109885883A (en) * | 2019-01-21 | 2019-06-14 | 江苏大学 | A kind of control method of the unmanned vehicle transverse movement based on GK clustering algorithm model prediction |
-
2019
- 2019-06-20 CN CN201910539923.7A patent/CN110321938B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101825901A (en) * | 2010-03-31 | 2010-09-08 | 北京航空航天大学 | Multi-agent robot cooperative control method based on artificial physics method |
| CN102096415A (en) * | 2010-12-31 | 2011-06-15 | 重庆邮电大学 | Multi-robot formation method based on Ad-Hoc network and leader-follower algorithm |
| CN102749847A (en) * | 2012-06-26 | 2012-10-24 | 清华大学 | Cooperative landing method for multiple unmanned aerial vehicles |
| CN103197684A (en) * | 2013-04-25 | 2013-07-10 | 清华大学 | Method and system for cooperatively tracking target by unmanned aerial vehicle cluster |
| CN103631141A (en) * | 2013-12-11 | 2014-03-12 | 北京航空航天大学 | Light transmission hypothesis based intensive autonomous aerial vehicle formation control method |
| CN106295613A (en) * | 2016-08-23 | 2017-01-04 | 哈尔滨理工大学 | A kind of unmanned plane target localization method and system |
| CN108983823A (en) * | 2018-08-27 | 2018-12-11 | 安徽农业大学 | A kind of plant protection drone cluster cooperative control method |
| CN109445456A (en) * | 2018-10-15 | 2019-03-08 | 清华大学 | A kind of multiple no-manned plane cluster air navigation aid |
| CN109343966A (en) * | 2018-11-01 | 2019-02-15 | 西北工业大学 | A kind of cluster organization method and device of unmanned node |
| CN109885883A (en) * | 2019-01-21 | 2019-06-14 | 江苏大学 | A kind of control method of the unmanned vehicle transverse movement based on GK clustering algorithm model prediction |
Non-Patent Citations (1)
| Title |
|---|
| 沈晶等: "《基于多智能体的Option自动生成算法》", 《智能系统学报》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110321938B (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Qolomany et al. | Parameters optimization of deep learning models using particle swarm optimization | |
| Zheng et al. | Improved multi-agent deep deterministic policy gradient for path planning-based crowd simulation | |
| Wang et al. | Large-scale online multitask learning and decision making for flexible manufacturing | |
| CN110782015A (en) | Training method and device for network structure optimizer of neural network and storage medium | |
| CN107102727B (en) | Dynamic gesture learning and recognition method based on ELM neural network | |
| Wang et al. | Ant colony optimization for task allocation in multi-agent systems | |
| CN110377051A (en) | A kind of time-varying formation applied to unmanned aerial vehicle group is swarmed control method | |
| Luo et al. | Flocking in target pursuit for multi-agent systems with partial informed agents | |
| CN108710906A (en) | Real-time point cloud model sorting technique based on lightweight network LightPointNet | |
| Wei et al. | Learning motion rules from real data: Neural network for crowd simulation | |
| US20210286375A1 (en) | Systems and methods for multi-agent system control using consensus and saturation constraints | |
| Thabet et al. | Sample-efficient deep reinforcement learning with imaginary rollouts for human-robot interaction | |
| Conde et al. | Time-varying formation controllers for unmanned aerial vehicles using deep reinforcement learning | |
| Marchese | A directional diffusion algorithm on cellular automata for robot path-planning | |
| Schwager et al. | Robust adaptive coverage for robotic sensor networks | |
| Xu et al. | Local motion simulation using deep reinforcement learning | |
| Fernando | Online flocking control of UAVs with mean-field approximation | |
| Yu et al. | Distributed generation and control of persistent formation for multi-agent systems | |
| Peng et al. | Geometric scene parsing with hierarchical lstm | |
| CN112529010B (en) | Point cloud identification method based on online local feature extraction | |
| Kumaar et al. | Mobile Service Robot Path Planning using Deep Reinforcement Learning | |
| Boubertakh | Knowledge-based ant colony optimization method to design fuzzy proportional integral derivative controllers | |
| Fernández et al. | Two steps reinforcement learning | |
| CN110321938A (en) | A kind of state space construction method and device of Intelligent unattended cluster | |
| CN112770256A (en) | Node track prediction method in unmanned aerial vehicle self-organizing network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |