CN108984634A - A kind of efficient distribution subscription method under cloud environment - Google Patents

Abstract

The present invention proposes that subscription is grouped according to core attribute by efficient publish/subscribe method EIM, EIM under a kind of cloud environment first, then according to basic operation symbol (,=,) predicate all in subscription is grouped, it will finally be grouped later subscription and be directed toward the later predicate composition subscription index of grouping.It is an advantage of the invention that, a kind of efficient subscription indexing means EIM is proposed relative to existing most of dissemination methods of subscribing to, and EIM method is optimized and is extended, EIM and existing method (k-index, BE-tree, Opindex) are compared by experimental evaluation, EIM method has than existing method better performance.

Description

A kind of efficient distribution subscription method under cloud environment

Technical field

The present invention relates to information technology, can be used in providing efficient publish/subscribe matching performance, and have smaller Memory consumption, the efficient publish/subscribe method under specially a kind of cloud environment.

Background technique

Publish/subscribe in the past twenty years between had received widespread attention, be applied to online advertisement at present (online advertise), stock market (Stock market), Social Media (social media), e-commerce (e- Commerce) etc..

One distribution subscription system includes two roles: data owner and user.Data owner is issued in the form of event Data.The form of user's Boolean expression subscribes to interested event.The interaction of the two roles can pass through single machine platform Distributed platform can also be passed through.System needs the corresponding data of event for guaranteeing to provide a user successful match in real time.

In existing publish/subscribe method, a section operator is rewritten as one group of equation extracted by k-index method Predicate, therefore there are many predicate quantity, cause the memory occupied and matching efficiency not high.In BE-tree, with the increase of attribute, BE-tree generates many nodes, causes memory consumption and matching efficiency not high.Opindex uses two-stage index structure, still The computation complexity for not minimizing event matches causes matching efficiency not high；And identical predicate is remained, therefore memory It consumes larger.Inactive event matching (static event matching) method can only support static user to subscribe to.To sum up may be used Know that current distribution subscription method in time application has inefficient and high memory consumption, therefore carries out grinding for this respect Study carefully and is of practical significance very much.

Summary of the invention

The present invention proposes a kind of efficient subscription indexing means EIM (An Efficient subscription Index for publication Match).Table 1 gives an example containing 6 one subscribed to set.EIM is first according to core Subscription is grouped by attribute, then according to basic operation symbol (,=,) predicate all in subscription is grouped, finally The subscription for being grouped later is directed toward and is grouped later predicate composition subscription index.To provide efficient publish/subscribe matching Can, and there is smaller memory consumption.

Duplicate predicate is handled, and is formed the invention proposes group technology is subscribed to for above-mentioned purpose Subscribe to index.

Minimal number of core attribute is found out from subscribing in collection S, it is right, there is a core attributeIt is corresponding with s,It is simultaneously also an attribute in S.Definition is provided to core attribute below.

Define 1.(core attribute)

One group of attribute is found out from subscribing in collection SIfMeet the following conditions, then claimsIt is the core attribute for subscribing to collection S Collection:

(1),, so thatIt is an attribute in s, Ke YiyongRepresent s, i.e., one subscribed to one Core attribute represents；

(2)It is the highest property set of the frequency of occurrences in S；

(3)It is minimum number, it is impossible to find out another group of core attributeSo thatQuantity be less than。

Based on core attribute collection, subscribe to collection S and be divided into disjoint subscription group, useIt indicates, is shown below.= …

In formula={s|s s ,Expression and core attributeRelevant subscription group.

Table 1 subscribes to example

	A=2B=3
			A8C=6E2
	B4C=6E[2, 9]
			B=3
	D8E9
			B=3C4D6

In actual publish/subscribe system, multiple subscription predicates having the same are a very common phenomenons, in order to subtract All duplicate predicates are omitted in few candidate matches predicate, EIM.In other words, subscribe to collection S in each different predicate in predicate group In only will appear it is primary.If a predicate has been already present in a predicate group, it can never appear in another In a predicate group.In order to accelerate to issue matched speed, all operators are all transformed into conjunction operation that is, if one A subscription more than one predicate, then must all predicates all match the matching being just able to achieve with incoming event.Below to predicate point The method of group provides definition.

Define the grouping of 2.(predicate)

Collection S is subscribed to if existed in system, the grouping for meeting the following conditions is known as the predicate grouping of S:

(1) each predicate group collects S from subscription；

(2) according to three kinds of operators (,=,) be grouped the predicate of all S;

(3) predicates only will appear once.

It is grouped as follows shown in formula to the predicate that collection S is carried out is subscribed to:

= 。

Subscription group and predicate group that two above part is individually formed are merged together and are formed subscription index, which builds Vertical target is the efficiency in order to improve the publication of user's subscribing matching in publish/subscribe system.

It is an advantage of the invention that proposing a kind of efficient subscription index relative to existing most of dissemination methods of subscribing to Method EIM, and EIM method is optimized and is extended, by experimental evaluation by EIM and existing method (k-index, BE- Tree, Opindex) it is compared, EIM method has than existing method better performance.

Detailed description of the invention

Fig. 1 is EIM system assumption diagram of the invention；

Fig. 2 is the subscription packet diagram of table 1 of the invention；

Fig. 3 is the predicate packet diagram of table 1 of the invention；

Fig. 4 is the subscription index map of table 1 of the invention；

Fig. 5 is that table 1 of the invention executes the updated subscription index map of subscription；

Fig. 6 is the memory consumption comparison diagram that subscription quantity of the invention influences；

Fig. 7 is the memory consumption comparison diagram of value spacial influence of the invention；

Fig. 8 is index construct comparison diagram of the invention；

Fig. 9 is the event matches comparison diagram that number of attributes of the invention influences；

Figure 10 is the event matches comparison diagram that subscription quantity of the invention influences；

Figure 11 is the event matches comparison diagram subscribing to maximum capacity and influencing of the invention；

Figure 12 is the event matches comparison diagram that event maximum capacity of the invention influences；

Figure 13 is the event matches comparison diagram of value spacial influence of the invention；

Figure 14 is that the subscription that subscription quantity of the invention influences updates comparison diagram；

Figure 15 is that the subscription that maximum capacity influences of subscribing to of the invention updates comparison diagram.

Specific embodiment

Below in conjunction with attached drawing and example, the invention will be further described, referring to Fig. 1~15, the structure of method for subscribing EIM As shown in Figure 1.There are two roles in EIM: one group of user and a data owner.User subscribes to data, is the user of data. Data owner issues data, is the owner of data.User first to system submit subscribe to application (module 1), then system from The core attribute that minimum number is found out in one group of subscription is grouped (module 2) to set is subscribed to then according to core attribute.Root According to three kinds of basic operations (,=,) be grouped each predicate for subscribing to concentration, it is divided into conjunction group and group of extracting.In order to subtract Duplicate predicate (module 3) is omitted in few predicate total quantity, EIM.The predicate for being grouped later is connected with subscription, is formed and subscribes to rope Draw (module 4).Data owner can issue a corresponding event (module 5) while issuing data (module 7).EIM will be defeated The event and subscription index entered puts together and is matched (module 6), if event can match subscription, by corresponding data It is sent to user (module 7).

Fig. 2 gives table 1 and subscribes to the subscription grouping for collecting and being divided into and subscribing to column containing 2.Since B occurs 4 times in table 1, C, E Occur 3 times, A, D occur 2 times.Therefore B is put into first.Due to subscribing to,,,B containing attribute, therefore can To represent this four subscription with B.That is:====B.Secondly, C, E are put into, subscription collection S at this time removes After four through being represented are subscribed to, only leaveWith.If selecting C, can only represent, and select E, then it can represent With, therefore remaining two subscription are represented with E.That is:==E.Therefore, the corresponding core attribute collection of table 1=B, E}.Given event: (A=2) ∧ (C=6), then subscription groupWithIn subscription be not centainlyCandidate order Read, thus do not have to judge S whether withMatching.

The subscription collection of table 1 is divided into 3 predicate groups, as shown in Figure 3.Subscribe to collection according to three kinds of operators (,=,) into Row grouping, and each predicate only will appear once.Such as most at the beginning of, S finds first subscription: A=2B=3, The two predicates are sequentially placed into "=" predicate group, and are connected with S.The subscription rope that table 1 is obtained according to index establishing method is subscribed to Draw as shown in Figure 4.The first step is directed toward the predicate that the subscription includes from each increase pointer of subscribing to.Second step has for each subscription Some conjunction predicate quantity counts, and is ready for the publication matching of next step.Due to having reduced a large amount of predicate here Quantity and operation species, therefore it is effectively reduced the matched link of publication and parameter, improve matched efficiency.

Table 1 deletes subscription first, then increase and subscribe to: D=5, updated subscription index are as shown in Figure 5.It is directed toward Four predicates: { E2, C=6, B4, E9 }, wherein B4 this predicate are not subscribed to by other and are used, and { E2, C=6 } At the same time byIt uses, E9 at the same time byIt uses, therefore can only delete、B4 and corresponding pointer.Due to core category Property be { B, E }, be free of D, andIn contain only an attribute D, therefore, increase core attribute D, increase new subscription group.By In D6 this predicate node have existed, therefore are not required to increase new predicate node, directly increase fromIt is directed toward D6 pointer ?.

The present invention discusses in terms of user query efficiency, index operation of extracting two.Choose core attribute when Wait, selection be make the method for group minimum number, and Opindex choose be the method for keeping group quantity most, actually this two The search efficiency of kind method is not highest.

Assuming that the subscription quantity in system is, then the highest perfect condition of efficiency should be: the quantity of subscription group is, and every group of subscription quantity is, user query match complexity at this time is, in this case User query efficiency highest.But in fact, the algorithm complexity of EIM and Opindex is calculated by formula (5).I.e. according to Core attribute quantity || and subscription group is (with core attributeRelevant subscription group) in subscription quantity || maximum value meter It calculates.And this value is centainly to be more than or equal to's.

Therefore, in order to maximize the efficiency of user query, EIM is optimized as follows:

(1) when choosing core attribute, to make the quantity of core attribute close as far as possible。

(2) when choosing subscription for a subscription group, to make the quantity of every group of subscription close。

In actual application scenarios, it is impossible to only exist conjunction operation, must there is also extract operation.Such as: B=.The predicate operator of set needs to be rewritten as common relational operator.It is rewritten as: (the ∨ of B=3 B=6 ∨ B = 9).The way of EIM is individually to establish the subscription index extracted, and specific algorithm is similar with conjunction index.In fact, any one A subscription can split into conjunction operation and operation of extracting, and in other words, if one is subscribed to not only containing operation of extracting, but also grasp containing conjunction Make, then it can be dismantled and be respectively put into extract index and conjunction index.

Method EIM is compared with existing method Opindex, BE-tree, k-index in the present invention.All indexes All it is terminate-and-stay-resident, subscription and event has been randomly generated, wherein all attributes and operator is all random.Due to index Memory-resident is wanted, therefore first experiment is used to test the memory consumption of four kinds of methods.Parameter that there are two influence memory consumptions: Subscribe to quantity and value space.For this purpose, experimental result is as shown in Figure 6 and Figure 7.Wherein ordinate is memory consumption, and unit is G.Fig. 6 Abscissa indicate subscribe to quantity, unit is M (million), the abscissa expression value space of Fig. 7.When subscribing to quantity from 100 When ten thousand (1M) rise to 4,000 ten thousand (40M), our method EIM slowly increases with the memory for the growth consumption for subscribing to quantity It is long, and the memory of Opindex consumption is 2 times of EIM, BE-tree is 4 times of EIM, and k-index is 14 times of EIM.This be by Identical predicate can be omitted in EIM.In k-index, a section operator can be rewritten into one group of equation predicate extracted, Therefore predicate quantity is most.And Opindex and BE-tree can't omit identical predicate, therefore memory consumption also compares EIM is big.When value space rises to the memory meeting sharp increase that 12800, k-index is consumed from 50, when value space=3200 Hes When 12800, the memory size that k-index is specifically consumed can not have been measured with the memory of our 128G.In BE-tree It deposits consumption also to increase very much: increase from 1G to 10G.This is because k-index and BE-tree are used based on attribute value Reverse list, therefore when being worth space growth, index also increases.And subscription has been divided into meaning by EIM and Opindex Phrase, therefore index does not increase with the growth in value space.Meanwhile identical predicate is omitted in EIM, and simplifies index Structure, therefore memory consumption ratio Opindex is small.

The building cost of index is only related to number of attributes, therefore test result is as shown in Figure 8.Wherein, abscissa indicates Number of attributes, unit are (1,000) k；Ordinate indicates the time of building index, and unit is the second (s).When number of attributes increases from 20k When growing to 60k, the time longest of BE-tree building index, and k-index takes second place, Opindex only consumes one more than EIM The point time.This is because EIM is omitted duplicate predicate and indexes unrelated with node capacity.The index construct of BE-tree according to Rely in node capacity (node capacity), node capacity is bigger, and the index construct time is fewer, but subscribing matching speed is got over Slowly.Therefore, a compromise has been taken when we test, the node capacity of BE-tree has taken a medium value (100).k- Index needs to generate more predicates, therefore the index construct time is also more.Opindex does not omit extra predicate, therefore The time of building index is more than EIM.

Event matches are exactly the matching subscribed to new publication data in fact, and matching speed is faster, the speed of user query data It spends faster.Therefore, the performance of event matches is only the performance indicator of whole system most critical.Dependence quantity subscribes to quantity, orders It reads maximum capacity, event maximum capacity, five, value space aspect to test event matches performance, test result such as Fig. 9 is arrived Shown in Figure 13.Ordinate indicate event matches consumption time, unit be microsecond ().Abscissa respectively indicates attribute number Amount subscribes to quantity, subscribes to maximum capacity, event maximum capacity and value space.The abscissa of Fig. 9 indicates that number of attributes, unit are k.The abscissa of Figure 10 indicates subscription quantity, and unit is M.The abscissa of Figure 11 indicates to subscribe to maximum capacity.The abscissa of Figure 12 Expression event maximum capacity.The abscissa expression value space of Figure 13.During number of attributes rises to 60k from 20k, EIM Performance be slightly higher than Opindex and BE-tree, be all the growth with number of attributes, the efficiency of event matches slightly by It is cumulative big, therefore EIM is suitable for the application scenarios of big data.And k-index is inefficient when there are many subscription quantity. This is because EIM minimizes the computation complexity of event matches, therefore performance is better than Opindex, BE-tree and k-index. During subscribing to quantity from 1M to 40M, the time that when EIM match event consumes has almost no change, be all have it is best Event matches performance, be 2 times of Opindex, 4 times of BE-tree, it is more than the decades of times of k-index.K-index efficiency is most Low reason has two: 1. not being divided, but is divided according to capacity is subscribed to, therefore not high enough according to core attribute Effect；2. candidate matches collection is much larger than EIM, therefore performance is lower than EIM.BE-tree uses hierarchical agglomerate, and therefore performance is higher than k- Index but is below EIM.And the index structure ratio EIM of Opindex is complicated, therefore performance is lower than EIM.Subscribing to maximum capacity From 4 rise to 20 during, EIM have classic performance.And with the growth for subscribing to capacity, the speed of performance decline Almost it can be ignored.Meanwhile the performance of EIM is slightly better than Opindex and BE-tree and is much better than k-index.This be by Operation is divided into EIM and extracts index and operation judges process is omitted in conjunction index.It is risen in event maximum capacity from 20 During 100, the performance of EIM is apparently higher than Opindex, BE-tree and k-index.And with event maximum capacity Increase, the performance of EIM is almost without any decline.This is because EIM simplifies index structure.It is risen in value space from 50 During 12800, EIM has classic performance.And with the growth for subscribing to capacity, the speed of performance decline almost may be used To ignore.Meanwhile the performance of EIM is substantially better than Opindex, BE-tree and k-index.This is because EIM is minimized The computation complexity of event matches.

It is related to subscription quantity and subscription maximum capacity to subscribe to update, therefore test result is as shown in Figure 14 and Figure 15.Its In, ordinate indicates renewal time, and unit is the second (s).The abscissa of Figure 14 indicates subscription quantity, and the abscissa expression of Figure 15 is ordered Read maximum capacity.K-index needs to update entire subscription group when update, BE-tree needs to update the subscription group being polymerize, Opindex needs to update reversely insertion column and related subscription group, and EIM only needs to update a small amount of correlation and subscribes to.Therefore, EIM Performance be better than Opindex, BE-tree and k-index.The performance of EIM is slightly better than Opindex and BE-tree, highly significant Be better than k-index.This is because the subscription of EIM, which updates operation, simplifies upgating object, therefore performance is by subscription maximum capacity Influence it is little.

Comprehensive all test datas, it can be deduced that draw a conclusion:

(1) comprehensive performance of EIM is substantially better than existing three kinds of methods: Opindex, BE-tree and k-index.

(2) due to all having very outstanding performance in face of various parameters, EIM can be widely used in various applications Scene, such as big data, distributed computing, e-commerce, cell phone platform, cloud computing etc..

The present invention considers the efficient matchings problem in publish/subscribe, and the memory consumption that existing method has is higher, has Subscribing matching performance not enough efficiently, have can not support dynamic subscribe to.For this purpose, the invention proposes a kind of efficient subscribe to index Method EIM will be grouped later subscription with later predicate is grouped and merge the new subscription index of composition.Subscription is provided simultaneously more New algorithm is for supporting dynamic to subscribe to.To provide efficient dynamic release/subscribing matching performance, and there is smaller memory Consumption.

Claims (1)

1. a kind of efficient publish/subscribe method under cloud environment, efficient publish/subscribe method is EIM, and EIM is first according to core Subscription is grouped by attribute, then according to basic operation symbol (,=,) predicate all in subscription is grouped, finally The subscription for being grouped later is directed toward and is grouped later predicate composition subscription index；It is specific to subscribe to group technology, to duplicate meaning Word is handled, and forms subscription index；Minimal number of core attribute is found out from subscribing in collection S, it is right, all There is a core attributeIt is corresponding with s,It is simultaneously also an attribute in S；Definition is provided to core attribute below；

Define 1.(core attribute)；One group of attribute is found out from subscribing in collection SIfMeet the following conditions, then claimsIt is to order Read the core attribute collection of collection S:

(1),, so thatIt is an attribute in s, Ke YiyongRepresent s, i.e., one subscribed to one Core attribute represents；

(2)It is the highest property set of the frequency of occurrences in S；

(3)It is minimum number, it is impossible to find out another group of core attributeSo thatQuantity be less than；

Based on core attribute collection, subscribe to collection S and be divided into disjoint subscription group, useIt indicates, is shown below；= … ；

In formula={s|s s ,Expression and core attributeRelevant subscription group；

Define the grouping of 2.(predicate)；

Collection S is subscribed to if existed in system, the grouping for meeting the following conditions is known as the predicate grouping of S:

(1) each predicate group collects S from subscription；

(2) according to three kinds of operators (,=,) be grouped the predicate of all S;

(3) predicates only will appear once；

It is grouped as follows shown in formula to the predicate that collection S is carried out is subscribed to:

= ；

Subscription group and predicate group that two above part is individually formed are merged together and are formed subscription index, which builds Vertical target is the efficiency in order to improve the publication of user's subscribing matching in publish/subscribe system.