CN109302300A - Data distributing method and device, data processing method and server - Google Patents

Data distributing method and device, data processing method and server Download PDF

Info

Publication number
CN109302300A
CN109302300A CN201710612889.2A CN201710612889A CN109302300A CN 109302300 A CN109302300 A CN 109302300A CN 201710612889 A CN201710612889 A CN 201710612889A CN 109302300 A CN109302300 A CN 109302300A
Authority
CN
China
Prior art keywords
event
data
event identifier
count value
identifier
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710612889.2A
Other languages
Chinese (zh)
Other versions
CN109302300B (en
Inventor
马冲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201710612889.2A priority Critical patent/CN109302300B/en
Publication of CN109302300A publication Critical patent/CN109302300A/en
Application granted granted Critical
Publication of CN109302300B publication Critical patent/CN109302300B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0631Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H04L41/064Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving time analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0633Lists, e.g. purchase orders, compilation or processing
    • G06Q30/0635Processing of requisition or of purchase orders
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0631Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/16Threshold monitoring

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

This application provides data distributing method and devices, data processing method and server, wherein, data processing method is applied in any server being connected with data distribution device, data processing method includes: the event identifier for receiving data distribution device and sending, and obtains the corresponding duration parameters of event identifier;Statistical phenomeon identifies total count value of the corresponding event in the duration parameters;Multiple event identifiers and its corresponding count value are preserved in server;Judge whether total count value is more than default processing threshold value, if it does, then ignoring corresponding to event, the part operation of not reproducible processing;If be no more than, execute corresponding to event, the part operation of not reproducible processing, and according to the time of reception of event identifier, the count value for the event that adds up.Using the embodiment of the present application, the mistake of user's operation generation can be also executed, user experience is promoted to avoid in the case where event data generates duplicate situation.

Description

Data distributing method and device, data processing method and server
Technical field
This application involves internet data processing technology field, in particular to a kind of data distributing method and device, with And a kind of data processing method and server.
Background technique
In the present of e-commerce rapid development, more and more users carry out various online transactions using internet, or Person transmits information etc..And for the operation that user generates via terminal on the internet, corresponding server can be sent to and held Row.For example, user logs in the operation of a mailbox, user name and login password can be sent to mailbox server via terminal, Mailbox server can generate the corresponding event of the operation and event data, include in event data and user's operation Relevant data.For another example lower single operation of the user in an online shopping mall to certain product, user also can incite somebody to action oneself via terminal The information such as oneself user name, posting address or the product that places an order are sent to the server of online shopping mall, generate phase by server As soon as the lower single event answered, the event data of the lower single event include the information of user name, address and product etc..
It but in practical applications, can due to network delay or equipment fault or user repeat submission etc. The phenomenon that same lower single operation is repeatedly sent to the server of online shopping mall can be will cause, in this case, will cause It repeats to send.
How this event data for repeating to send is handled, technical problem referred to as urgently to be resolved.
Summary of the invention
Inventor has found in the course of the research, the prior art can not counterweight recur the event data accurate judgement sent, according to The application one aspect, if can be after server receives user's operation, by the event data of generation according to each event number It is counted according to corresponding event identifier, is had submitted repeatedly if the corresponding event of certain event identifier is repeated in a short time, Then no longer respond the corresponding user's operation of the event, so that it may which whether the corresponding event of one event data of accurate judgement is weight Multiple event, and then can determine to whether the corresponding user's operation of the event needs to respond.
Based on this, this application provides a kind of data distributing method and a kind of data processing methods, each to be adopted as A event data generate relative to event identifier, and by event data according to of the processing whether duplicate server of event Number, to be respectively allocated in each server, and each server then counts some event identifier of oneself storage in one section of thing Whether number in part is more than default processing threshold value, and those of the corresponding not reproducible processing of the event is ignored as if being more than Operation, the event data that not only ensure that user's operation generates will not be repeatedly processed through vaporization, condensation and absorption, such as will not repeat and withhold Integration operation etc. is operated or reduces, also the user experience is improved.
Present invention also provides a kind of data distribution device and servers, to guarantee the reality of the above method in practice Existing and application.
To solve the above-mentioned problems, this application discloses a kind of data processing method, the data processing method includes:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;
Judge whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event , the part operation of not reproducible processing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to described The time of reception of event identifier, the count value for the event that adds up.
Wherein, with the one-to-one multiple memory spaces of multiple timeslices difference, the interval duration of each timeslice is identical, The memory space includes timestamp and fragmentation count value, when the timestamp is used to indicate the interval since the timestamp The corresponding event of the event identifier received in length, needs to be added in the memory space;It is described according to the event identifier The time of reception, the count value for the event that adds up, comprising:
The time of reception of the event identifier is calculated divided by the quotient of the interval duration, is stabbed as to storage time, with And the remainder of the number of the time of reception and the memory space is calculated, it is indexed as to memory space;
Judgement is described to index the original time stamp that corresponding memory space includes to memory space, with described when storing Between stab it is whether identical, if it is, the corresponding fragmentation count value of the original time stamp that adds up;
If it is not, then emptying the corresponding memory space of the original time stamp, the original time stamp is replaced with described It is stabbed to storage time, and cumulative described stabs corresponding, the event fragmentation count value to storage time.
Wherein, total count value of the corresponding event of the statistics event identifier in the duration parameters, comprising:
Obtain the time of reception of the event identifier;
The timestamp that the quotient and the quotient for calculating the time of reception and the interval duration include with each memory space Between difference, and using each difference as the query argument of each memory space;
Judge whether the query argument of each memory space is less than the duration parameters, if it is, obtaining the storage Fragmentation count value that space includes simultaneously adds up, and obtains the total count value of the event identifier.
The embodiment of the present application also provides a kind of data distributing methods, this method comprises:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
It is wherein, described to generate the corresponding event identifier of the event data according to pre-set mark create-rule, Include:
Obtain the type identification and/or operated object mark of user identifier, operation that the event data includes;
The user identifier, the type identification of operation and/or operated object mark are combined, the event identifier is obtained.
Wherein, the number according to server, the corresponding event data of the event identifier is distributed to the multiple In server, comprising:
For each event data, corresponding event identifier is subjected to Hash operation, obtains the cryptographic Hash of event identifier;
Number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value;
The event identifier is sent in the server of the index value mark.
The embodiment of the present application also provides a kind of server, the server includes:
Event identifier receiving unit for receiving event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic unit, for counting total count value of the corresponding event of the event identifier in the duration parameters;
Judging unit, for judging whether the total count value is more than default processing threshold value;
First processing units, for the result of the judging unit be more than in the case where, it is right to ignore event institute The part operation of processing answer, not reproducible;
The second processing unit, for the result of the judging unit be no more than in the case where, execute the event institute The part operation of corresponding, not reproducible processing;
Summing elements, for according to the time of reception of the event identifier, tiring out after triggering described the second processing unit Add the count value of the event.
Wherein, with the one-to-one multiple memory spaces of multiple timeslices difference, the interval duration of each timeslice is identical, The memory space includes timestamp and fragmentation count value, when the timestamp is used to indicate the interval since the timestamp The corresponding event of the event identifier received in length, needs to be added in the memory space;The summing elements include:
Second computation subunit, the time of reception for calculating the event identifier are made divided by the quotient of the interval duration To be stabbed to storage time, and, the remainder of the number of the time of reception and the memory space is calculated, as sky to be stored Between index;
Second judgment sub-unit, for judge it is described when memory space index that corresponding memory space includes it is original when Between stab, with it is described to storage time stab it is whether identical;
Second cumulative subelement, for the result of the judgment sub-unit be in the case where, when adding up described original Between stab corresponding fragmentation count value;
Subelement is emptied, for emptying the original time stamp pair in the case where second judgment sub-unit is no The memory space answered replaces with the original time stamp described to storage time stamp;
Third adds up subelement, described stabs corresponding, the event fragmentation count value to storage time for cumulative.
Wherein, the statistic unit includes:
Second obtains subelement, for obtaining the time of reception of the event identifier;
First computation subunit, for calculating the time of reception and the quotient for being spaced duration and the quotient and respectively depositing The difference between timestamp that storage space includes, and using each difference as the query argument of each memory space;
First judgment sub-unit, for judging whether the query argument of each memory space is less than the duration parameters;
First cumulative subelement, for obtaining the memory space in the case where the result of the judgment sub-unit, which is, is Including fragmentation count value and add up, obtain the total count value of the event identifier.
The embodiment of the present application also provides a kind of data distribution device, which includes:
Event data receiving unit is used for receiving event data, and the event data includes: user is in foreground system The related data of operation;
Generation unit, for generating the corresponding event mark of the event data according to pre-set mark create-rule Know;
Allocation unit distributes the corresponding event data of the event identifier to described for the number according to server In multiple servers.
Wherein, the generation unit includes:
First obtains subelement, for obtaining the type identification of user identifier, operation that the event data includes And/or operated object mark;
Combination subelement is obtained for combining the type identification and/or operated object mark of the user identifier, operation The event identifier.
Wherein, the allocation unit includes:
Hash operation subelement, for corresponding event identifier being carried out Hash operation, is obtained for each event data To the cryptographic Hash of event identifier;
Modulo operation subelement obtains rope for number of the cryptographic Hash to the server to be carried out modulo operation Draw value;
Transmission sub-unit, for the event identifier to be sent in the server of the index value mark.
The embodiment of the present application also provides a kind of computer-readable medium, which has and makes to calculate Machine execute for data distribution program, the processing the following steps are included:
Receiving step, receiving event data, the event data includes: correlation of the user in the operation of foreground system Data;
Generation step generates the corresponding event mark of the event data according to pre-set mark create-rule Know;
Allocation step distributes the corresponding event data of the event identifier to described more according to the number of server In a server.
The embodiment of the present application also provides another computer-readable medium, which has and makes to count Calculation machine execution be used for, the processing the following steps are included:
Receiving step receives event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic procedure counts total count value of the corresponding event of the event identifier in the duration parameters;
Judgment step judges whether the total count value is more than default processing threshold value;
First processing step, for ignoring corresponding to the event in the case where result that judgment step is more than , the part operation of not reproducible processing;
Second processing step is right in the case where the result that judgment step is no more than, executing the event institute The part operation of processing answer, not reproducible;
Accumulation step, for according to the time of reception of the event identifier, adding up described after second processing step The count value of event.
The embodiment of the present application also provides a kind of computer equipment, which includes: processor, memory, net Network interface and bus system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing logical between the computer equipment and at least one other computer equipment Letter connection;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
The embodiment of the present application also provides another computer equipment, the computer equipment include: processor, memory, Network interface and bus system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing logical between the computer equipment and at least one other computer equipment Letter connection;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;It is protected in the server There are multiple event identifiers and its corresponding count value;
Judge whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event , the part operation of not reproducible processing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to described The time of reception of event identifier, the count value for the event that adds up.
The embodiment of the present application also provides a kind of user interface interaction equipment, comprising:
First interactive module, for forwarding the first triggering command to processor after receiving first triggering command, So as to processor receiving event data, the event data includes: related data of the user in the operation of foreground system;
Second interactive module, for referring to after receiving the second triggering command, and to processor forwarding first triggering It enables, so that processor is according to pre-set mark create-rule, generates the corresponding event identifier of the event data;
Third interactive module, for forwarding the third triggering to refer to after receiving third triggering command, and to processor It enables, the number so as to processor according to server distributes the corresponding event data of the event identifier to the multiple service In device.
The embodiment of the present application also provides another user interface interaction equipment, comprising:
4th interactive module, for forwarding the 4th triggering command to processor after receiving the 4th triggering command, So that processor receives event identifier, and obtain the corresponding duration parameters of the event identifier;
5th interactive module forwards the 5th triggering command for receiving the 5th triggering command, and to processor, with Just processor counts total count value of the corresponding event of the event identifier in the duration parameters;It is protected in the server There are multiple event identifiers and its corresponding count value;
6th interactive module forwards the 6th triggering command for receiving the 6th triggering command, and to processor, with Just processor judges whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event , the part operation of not reproducible processing;If be no more than, execute corresponding to the event, the portion of not reproducible processing Divide operation, and according to the time of reception of the event identifier, the count value for the event that adds up.
The embodiment of the present application also provides a kind of data processing methods, comprising:
First calculate equipment generate event identifier, wherein the event identifier include at least or based on following data it One: User ID, timestamp, operation code and operation object ID;
First, which calculates equipment, calculates the equipment transmission event identifier to second, wherein the second calculating equipment is based on The event identifier carries out deduplication operation.
Compared with prior art, the application includes following advantages:
In the embodiment of the present application, data distribution device can generate each event number according to preset mark create-rule According to corresponding event identifier, and when distributing each event identifier to server, the number according to server is allocated, Guarantee that the same event identifier can be assigned to being handled on the same server, so that server statistics event identifier Be not in when count value mistake or careless omission, also, because each event identifier distribute to each server be it is random, Therefore, data skew problem of the event identifier between each server is also avoided to a certain extent.
Further, after each server receives event identifier, can will a period of time in store the event identifier Corresponding fragmentation count value adds up, to obtain the corresponding total quantity of the event identifier, then by the total quantity and in advance If processing threshold value is judged, to just no longer respond the event institute of the event identifier when being more than default processing threshold value Corresponding user's operation also avoids the mistake that user's operation generation can be also executed in the case where event data generates duplicate situation Accidentally, simultaneously as avoiding the repetition of user being withholdd or reduced the relevant operations such as integral, also the user experience is improved.
Further, since server internal can also mark off multiple memory spaces, each memory space is stored The fragmentation count value of the event identifier received in a certain section of event, so as to rapidly find each memory space storage Fragmentation count value add up, obtain the total quantity of event, ensure that data processing speed and accuracy.
Certainly, any product for implementing the application does not necessarily require achieving all the advantages described above at the same time.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, required in being described below to embodiment The attached drawing used is briefly described, it should be apparent that, the drawings in the following description are only some examples of the present application, For those of ordinary skill in the art, without any creative labor, it can also obtain according to these attached drawings Obtain other attached drawings.
Fig. 1 is the illustrative diagram of the application scenarios of the application in practical applications;
Fig. 2 is the exemplary process diagram of the data distributing method embodiment of the application;
Fig. 3 is the exemplary process diagram of the data processing method embodiment of the application;
Fig. 4 is the exemplary block diagram of the data distribution device embodiment of the application;
Fig. 5 is the exemplary block diagram of the data processing equipment embodiment of the application;
Fig. 6 is a kind of exemplary block diagram of computer equipment of the application;
Fig. 7 is the exemplary block diagram of another computer equipment of the application;
Fig. 8 a~Fig. 8 b is a kind of interface schematic diagram of user interface interaction equipment of the application;
Fig. 9 a~Fig. 9 b is the interface schematic diagram of another user interface interaction equipment of the application.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete Site preparation description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on Embodiment in the application, it is obtained by those of ordinary skill in the art without making creative efforts it is all its His embodiment, shall fall in the protection scope of this application.
Refering to what is shown in Fig. 1, being the exemplary scene schematic diagram of the embodiment of the present application in practical applications.Wherein, it uses Family produces operation in foreground system 101, for example, lower single operation, register, alternatively, the edit operation etc. to product, A referred to as event.Event data is the relevant data content of the operation, for example, the user name to place an order, order number are stepped on Record password, etc..The event that foreground system 101 is generated in response to the operation of user, can be sent to number for each event data According to distributor 102, each event data is allocated by data distribution device 102, and the event identifier of each event is sent out It send into each server being connected with data distribution device 102.In Fig. 1, the number of server can be set to N number of, and N is Integer greater than 1.In server 103, shown in embodiment as shown in Figure 3, server 103 can be to distributing to oneself Event identifier is counted, and judges whether the total count value (i.e. quantity) of the corresponding event of event identifier does not meet threshold value, and root It is judged that result operates to determine execution or ignore the part of operation corresponding with the event, not reproducible, thus The duplicate removal processing for realizing each event, guarantee the same event will not repeat it is corresponding, cause damages to user certain A little operations, for example, withholing or deducting the operation such as integral.
Below with reference to application scenarios shown in FIG. 1, data distributing method and data processing method to the application carry out detailed It is thin to introduce.With reference to Fig. 2, a kind of flow chart of data distributing method embodiment of the application is shown, the present embodiment can be applied to figure On data distribution device shown in 1, the present embodiment can with the following steps are included:
Step 201: receiving event data, the event data may include: the correlation of operation of the user in foreground system Data.
For user after foreground system generates operation, the corresponding event data of the operation is sent to data by foreground system Distributor.Wherein, event data may include the related data that user operates in foreground system.For example, user is at the terminal After browsing product details, the lower single operation of product A is had submitted to foreground system by terminal, then foreground system is under this Single operation generates a lower single event, may include: user name, O/No., lower single object in the event data of lower single event Mark and posting address etc., the event data of lower single event is sent to data distribution device again by foreground system.
Step 202: according to pre-set mark create-rule, generating the corresponding event identifier of the event data.
After data distribution device receives event data, can according to the mark create-rule pre-set, come for Each event identifier generates event identifier corresponding with each event data.Specifically, data distribution device can be according to setting in advance The mark create-rule set, to generate the corresponding event identifier of each event data, for example, those skilled in the art set in advance Mark create-rule is set are as follows: each event data generates unique corresponding event identifier according to different principle, can incite somebody to action The contents such as time, user name, O/No. are combined, to generate unique corresponding event identifier for each event data.
In a kind of possible embodiment, step 202 specifically can first obtain the user identifier for including in event data (can be user name), operation type identification and/or operated object mark, recombinant user identifier, operation type identification And/or operated object mark etc., obtain the corresponding event identifier of each event data.Wherein, the type identification of operation is for indicating The action type of the corresponding event of event data logs in for example, placing an order, the action types such as modification, or deletion.Operation object Mark is for uniquely indicating the operation object of the corresponding event of event data, for example, commodity 156789, i.e., unique corresponding Commodity represented by 156789.
Specifically, the event identifier that user identifier, the type identification of operation and operated object mark obtain after being combined It can be shaped like: event identifier=User ID _ operation type identification _ operated object mark ID.For example, 12345_ deduction of points _ deduction of points The type, _ commodity 1234, etc. alternatively, 12345_ places an order.Wherein, User ID " 12345 " is used for one user of unique identification, behaviour The type identification " deduction of points " of work or " placing an order " are used to indicate that the type of user's operation to be, operated object mark " deduction of points class Type " or " commodity 1234 " are used for the targeted operation object of one user's operation of unique identification.Implemented according to the application one Example, further, " deduction of points " corresponding digital code 001, " deduction of points type " correspond to digital code 7, then event identifier can be 12345_001_7。
Certainly, specific data above-mentioned are used for the purpose of those skilled in the art understand that the application facilitates that is illustrated to show Example property data, should not be construed as the restriction of the embodiment of the present application.
Step 203: according to the number of server, the corresponding event data of the event identifier being distributed to the multiple In server.
It can be each event point after data distribution device is the corresponding each event identifier of generation of each event data Most its that server distributed is sent with corresponding server, and by the corresponding event identifier of each event.Specifically, Data distribution device can distribute each event identifier according to step A1~step A3 as follows:
Step A1: for each event data, corresponding event identifier is subjected to Hash operation, obtains event identifier Cryptographic Hash.
Firstly, for event identifier obtained in step 202, Hash operation is carried out to event identifier respectively, for example, can To carry out Hash operation to each event identifier, obtain the Hash of each event identifier using hash algorithms such as CRC32, MD5 Value.
Step A2: number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value.
In order to guarantee to lead to the problem of data skew between each server, also, the same event identifier can divide It is assigned in the same server, modulo operation is carried out to number of the cryptographic Hash of each event identifier to server in this step, Obtain that each event identifier is corresponding, index value of server.For example, it is assumed that server has three: server 1,2 and of server Server 3, then the index value that event identifier respectively may obtain 3 progress modulo operations is 0,1 or 2, then will be indexed The event identifier that value is 0 is distributed to server 1, will be obtained the event identifier that index value is 1 and is distributed to server 2, and will obtain The event identifier that index value is 2 is distributed to server 3.
Step A3: the event identifier is sent in the server of the index value mark.
Then, event identifier is respectively sent in that server corresponding with index value by data distribution device again, example Such as, all event identifiers that index value is 0 will be obtained and be all sent to server 1, all things of the index value for 1 will be obtained Part mark is all sent to server 2, and, all event identifiers that index value is 2 will be obtained and be all sent to server 3.
As it can be seen that in the embodiment of the present application, data distribution device can generate each thing according to preset mark create-rule Number of packages according to corresponding event identifier, and when distributing each event identifier to server, divided by the number according to server Match, guarantees that the same event identifier can be assigned to being handled on the same server, so that server statistics event identifier Count value when be not in mistake or careless omission, also, because each event identifier to distribute to each server be random , therefore, data skew problem of the event identifier between each server is also avoided to a certain extent.
In the embodiment of the present application, after event identifier is sent to each server by data distribution device, in each server In, the number that the corresponding event of event identifier is submitted by user in single timeslice is counted for convenience, and each server can divide Multiple one-to-one memory spaces and timeslice are not set.Wherein, the interval duration of each timeslice is identical, each memory space Timestamp and fragmentation count value are respectively included, which is used to indicate to receive in the interval duration since the timestamp The corresponding event of event identifier, need to be added in memory space belonging to the timestamp.
For example, 3 memory spaces store:store1, store2 and store3 are arranged, respectively for server The size of store can not be identical, and the interval duration interval of the corresponding timeslice of each store is identical, and interval can be with The quantity of Millisecond is taken, such as interval is 2 milliseconds.Each store includes a timestamp and a fragmentation count value, The value ts of timestamp record can be used to indicate that the event in this period of time of ts+interval since ts, corresponding Event identifier should be added to the fragmentation count value that memory space belonging to the timestamp includes.The fragmentation count value can lead to Mapping table is crossed to realize, that is, each memory space is corresponding, the fragmentation count value of event identifier that receives can be stored in one It opens in mapping table, event identifier is saved in the mapping table and its starts point into this period of time of ts+interval in ts Section count value.
It is assumed that there are 3 memory spaces store1, store2 and store3,3 timeslices are respectively corresponded: " 2017/5/ 9 00:00:01 ", " 2017/5/9 00:00:02 " and " 2017/5/9 00:00:03 ", i.e., store1 is saved " 2017/ The count value of the event identifier entered in the 5/9 00:00:01 " period, and so on, store3 is saved " 2017/5/9 The count value of the event identifier entered in the 00:00:03 " period, i.e., each memory space store 1 second data, i.e., Interval=1000 (millisecond).Assuming that this 3 timeslices corresponding time of reception are as follows: 1494259201000 (millis Second), 1494259202000 (milliseconds) and 1494259203000 (milliseconds).Then the corresponding timestamp of memory space store is just It is the quotient of the time of reception and interval duration, the i.e. corresponding timestamp ts1=1494259201000/ of memory space store1 The corresponding timestamp ts2=1494259202000/interval=of interval=1494259201, memory space store2 The corresponding ts3=1494259203000/interval=1494259203 of 1494259202, memory space store3.
In the embodiment of the present application, n memory space in the server can be recycled.Still with 3 memory spaces For respectively corresponding 3 seconds timeslices, store1 stores the fragmentation count value of the event identifier received in first second, The fragmentation count value for the event identifier that store2 storage receives in second second, and what store3 was received in the storage third second The fragmentation count value of event identifier, and so on, when the 4th second, stored the 4th second by first memory space store1 The fragmentation count value of the event identifier received is stored the 5th second when the 5th second by second memory space store2 Fragmentation count value of the event identifier received, etc..
With reference to Fig. 3, a kind of flow chart of data processing method embodiment of the application, the processing method application are shown In any server 103 being connected with data distribution device, the present embodiment may comprise steps of 301~step 306:
Step 301: receiving event identifier, and obtain the corresponding duration parameters of the event identifier.
After each event identifier is sent to each server according to index value by data distribution device, for each It is available to arrive the corresponding duration parameters of the event identifier after receiving event identifier for server, so that server comes Count count value of the corresponding event of the event identifier in the time range that the duration parameters are illustrated.Wherein, the duration parameters It can be configured and be stored in each server by those skilled in the art in advance, it, can for the event of each type The time range of all events of the type is indicated so that a duration parameters are arranged.For example, for this seed type that places an order Event, the count value in 2 seconds can be inquired, then for the event of lower single type, duration parameters are exactly 2 seconds.Certainly, ability Field technique personnel may be that duration parameters preset other numerical value.
Step 302: counting total count value of the corresponding event of the event identifier in the duration parameters.
Because identical event identifier will be sent to the same server, institute by the distribution of data distribution device With, it is only necessary to the corresponding total count value of the event identifier that the server statistics receive oneself, without considering other services Device.
Specifically, memory space is needed with to determine according to each memory space corresponding timestamp for including in server Count the fragmentation count value recorded in which memory space, then by the fragmentation count value in each memory space to be counted Carry out the cumulative total count value that event identifier can be obtained.A kind of embodiment of step 302 may include step B1~step B4:
Step B1: the time of reception of the event identifier is obtained.
Firstly, determining the time of reception of event identifier A to be counted.It is assumed that event identifier A is 9 days 00 May in 2017: Server 1 receives at the time of 00:03, then corresponding time of reception time=1494259203000.
Step B2: calculating the time of reception and the quotient for being spaced duration and the quotient and each memory space includes Difference between timestamp, and using each difference as the query argument of each memory space.
In this step, the time of reception " 2017/5/9 00:00:03 " of calculating and the quotient of interval duration are 1494259203000/interval=1494259203, then by the quotient " 1494259203 " respectively with store1, store2 and The corresponding timestamp 1494259201,1494259202 and 1494259203 of store3 subtracts each other, and respectively obtains three differences are as follows: 2,1 and 0, i.e., the query argument that the query argument of store1 is 2, store2 is 1, and, the query argument of store3 is 0.
Step B3: judging whether the query argument of each memory space is less than the duration parameters, if it is, into Enter step B4.
For above-mentioned 3 memory spaces, judge whether its corresponding query argument is less than the value of duration parameters respectively, i.e., Whether the corresponding query argument of store1, store2 and store3 is judged less than 2, and judgement obtains looking into for store2 and store3 Parameter is ask less than 2, and the query argument of store1 is equal to 2.
Step B4: it obtains the fragmentation count value that the memory space includes and adds up, obtain the total of the event identifier Count value.
Then it is directed to store2 and store3, A pairs of event identifier will just recorded in the mapping table in the two memory spaces The fragmentation count value answered adds up, and obtains the total count value of event identifier A.For example, event identifier in the mapping table of store2 The fragmentation count value that the fragmentation count value of A is event identifier A in the mapping table of 2, store3 is 1, then event identifier A is in duration Total count value in parameter 2 seconds is 2+1=3.
Step 303: judging whether the total count value is more than default processing threshold value, if it does, then entering step 304; If be less than, 305 are entered step.
Next, it is determined that whether the total count value that step 302 obtains is more than default processing threshold value, for example, threshold can will be handled Value is set in advance as 2, and the total count value that event identifier A is calculated has been more than default processing threshold value, then enters step 304.
Step 304: ignoring corresponding to the event, the part operation of not reproducible processing.
In practical applications, having the corresponding operation of some events is not reproducible processing, for example, user is to same Payment request when a order is paid the bill will be right if repetition has submitted payment request due to webpage delay etc. User, which executes, repeats operation of withholing, and this repetitive operation will bring very bad influence to user experience.Therefore, in this reality It applies in example, for the corresponding operation of event of this type, for example, the operation of user integral is reduced, or operation of withholing, with And the disable operation etc. to user account, corresponding processing threshold value can be preset, when the event mark of discovery event The total count value of knowledge has been more than default processing threshold value, can directly be ignored corresponding to event, the part of not reproducible processing Operation, for example, withholing or reducing the operation such as integral.
Step 305: executing corresponding to the event, the part operation of not reproducible processing, subsequently enter step 306.
And if the corresponding total count value of event identifier is less than default processing threshold value, illustrate that the event can be executed The part operation of the not reproducible processing of part that is corresponding, being related to user's personal information, because being less than default processing threshold Value, then execute when the corresponding part of the event operates and illustrate to repeat.
Step 306: according to the time of reception of the event identifier, the count value for the event that adds up.
Then, according still further to the time of reception of event identifier, the count value for the event that adds up, i.e., by corresponding point of event identifier Section count value adds 1 in memory space corresponding with the time of reception.Specifically, step 306 may include step C1~step C4:
Step C1: the time of reception of the event identifier is calculated divided by the quotient of the interval duration, as to storage time Stamp, and, the remainder of the number of the time of reception and the memory space is calculated, is indexed as to memory space.
Firstly, when receiving event identifier, time of reception of event identifier is calculated divided by the quotient of interval duration, It is stabbed as to storage time, i.e. ts=time/interval;And the number of memory space described in the calculating time of reception is remaining Number, indexes, i.e. index=time/n as to memory space, wherein n is the number of memory space.It is still interior with 3 for server For a memory space, it is assumed that obtained index=0, then it represents that need the event identifier in first memory space Fragmentation count value in store1 adds 1, and so on, it is assumed that obtained index=2, then it represents that need the event identifier Fragmentation count value in memory space store3 adds 1.
Step C2: judgement is described to index the original time stamp that corresponding memory space includes to memory space, with it is described to Whether storage time stamp is identical, if it is, entering step C3;If it is not, then entering step C4.
Next, it is determined that original time stamp included by the corresponding memory space of index obtained in step C1, with step Whether stabbing to storage time for being actually calculated in C1 be identical, if identical, C3 is entered step, by the memory space The corresponding fragmentation count value of the event identifier saved in mapping table adds 1.For example, it is assumed that obtained index is zero, i.e., first Memory space store1 is to memory space, then timestamp ts1 included by store1 is original time stamp, and step C1 is counted If obtained timestamp is also equal to ts1, C3 is thened follow the steps, if being not equal to ts1, enters step C4.
Step C3: the corresponding fragmentation count value of the original time stamp that adds up.
The fragmentation count of the event identifier in mapping table in corresponding memory space will be indexed in this step to memory space Value plus 1 is indicated in time reception to the corresponding event of the primary event identifier, so that subsequent needs judge whether to execute When the part operation of corresponding, the not reproducible execution of event, added up each fragmentation count value to obtain total count value.
Step C4: emptying the corresponding memory space of the original time stamp, by the original time stamp replace with it is described to Storage time stamp, and cumulative described corresponding, the event fragmentation count value is stabbed to storage time.
If it is not the same, then illustrate it is expired wait store index index, then will index be corresponding deposits wait store index Store up space in mapping table reset, and by the time stamp setting of the memory space after clearing be step C1 in be calculated when Between stab ts, the corresponding fragmentation count value of the event identifier is added 1, and fragmentation count value is saved to the memory space In mapping table.
As it can be seen that in the embodiment of the present application, it, can be by storage in a period of time after each server receives event identifier The corresponding fragmentation count value of the event identifier add up, to obtain the corresponding total quantity of the event identifier, then should It amounts to quantity to be judged with default processing threshold value, to just no longer respond the event when being more than default processing threshold value User's operation corresponding to the event of mark also avoids also execute event in the case where event data generates duplicate situation The error in data that corresponding user's operation generates, simultaneously as avoiding the repetition of user being withholdd or reduced the phases such as integral Operation is closed, also the user experience is improved.
Further, since server internal can also mark off multiple memory spaces, each memory space is stored The fragmentation count value of the event identifier received in a certain section of event, so as to rapidly find each memory space storage Fragmentation count value add up, obtain the total quantity of event, ensure that data processing speed and accuracy.
For the aforementioned method embodiment, for simple description, therefore, it is stated as a series of action combinations, but Be those skilled in the art should understand that, the application is not limited by the described action sequence because according to the application, Some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know that, in specification Described embodiment belongs to preferred embodiment, necessary to related actions and modules not necessarily the application.
It is corresponding with method provided by a kind of data distributing method embodiment of above-mentioned the application, referring to fig. 4, the application A kind of data distribution device embodiment is additionally provided, in the present embodiment, which is connected with multiple servers, The data distribution device may include:
Event data receiving unit 401 is used for receiving event data, and the event data includes: user is in foreground system Operation related data.
Generation unit 402, for generating the corresponding thing of the event data according to pre-set mark create-rule Part mark.
Wherein, the generation unit 402 can specifically include:
First obtains subelement, for obtaining the type identification of user identifier, operation that the event data includes And/or operated object mark;With combination subelement, for combining type identification and/or the operation of the user identifier, operation Object identity obtains the event identifier.
Allocation unit 403, for the number according to the server, by the corresponding event data point of the event identifier It is assigned in the multiple server.
Wherein, the allocation unit 403 can specifically include:
Hash operation subelement, for corresponding event identifier being carried out Hash operation, is obtained for each event data To the cryptographic Hash of event identifier;Modulo operation subelement, for number of the cryptographic Hash to the server to be carried out modulus Operation obtains index value;With transmission sub-unit, for the event identifier to be sent to the server of the index value mark In.
It is corresponding with method provided by a kind of data processing method embodiment of above-mentioned the application, refering to what is shown in Fig. 5, this Application additionally provides a kind of server example, and in the present embodiment, which is connected with data distribution device, the service Device may include:
Event identifier receiving unit 501, the event identifier sent for receiving the data distribution device, and obtain institute State the corresponding duration parameters of event identifier.
Statistic unit 502, for counting tale of the corresponding event of the event identifier in the duration parameters Value;Multiple event identifiers and its corresponding count value are preserved in the server.
Wherein, the statistic unit 502 can specifically include:
Second obtains subelement, for obtaining the time of reception of the event identifier;First computation subunit, based on The difference between the timestamp that the time of reception includes with the quotient for being spaced duration and the quotient and each memory space is calculated, And using each difference as the query argument of each memory space;First judgment sub-unit, for judging each memory space Query argument whether be less than the duration parameters;First cumulative subelement, for the result in first judgment sub-unit In the case where being, obtains the fragmentation count value that the memory space includes and add up, obtain the total of the event identifier Count value.
Judging unit 503, for judging whether the total count value is more than default processing threshold value.
First processing units 504, for the result of the judging unit be more than in the case where, ignore the event The part operation of corresponding, not reproducible processing.
The second processing unit 505, for the result of the judging unit be no more than in the case where, execute the thing Corresponding to part, the part operation of not reproducible processing.
Summing elements 506, for after triggering described the second processing unit, according to the time of reception of the event identifier, The count value for the event that adds up.
It wherein, include multiple one-to-one memory spaces and timeslice in the server, when the interval of each timeslice Length is identical, and the memory space includes timestamp and fragmentation count value, and the timestamp is for indicating since the timestamp The corresponding event of event identifier received in the duration of interval, needs to be added in the memory space;The summing elements 506 It can specifically include:
Second computation subunit, the time of reception for calculating the event identifier are made divided by the quotient of the interval duration To be stabbed to storage time, and, the remainder of the number of the time of reception and the memory space is calculated, as sky to be stored Between index;With, the second judgment sub-unit, for judge it is described to memory space index that corresponding memory space includes it is original Timestamp, with it is described to storage time stab it is whether identical;Second cumulative subelement, in second judgment sub-unit It as a result is the corresponding fragmentation count value of the original time stamp that adds up in the case where being;Subelement is emptied, for described the In the case that two judgment sub-units are no, the corresponding memory space of the original time stamp is emptied, the original time stamp is replaced It is changed to described to storage time stamp;Third adds up subelement, for it is cumulative it is described stab to storage time it is corresponding, the event Fragmentation count value.
The embodiment of the present application also provides a kind of computer-readable medium, which has and makes to calculate Machine executes the program for data distribution, which may comprise steps of:
Receiving step, receiving event data, the event data includes: correlation of the user in the operation of foreground system Data;
Generation step generates the corresponding event mark of the event data according to pre-set mark create-rule Know;And
Allocation step distributes the corresponding event data of the event identifier to described more according to the number of server In a server.
The embodiment of the present application also provides another computer-readable medium, which has and makes to count The execution of calculation machine is used for, which may comprise steps of:
Receiving step receives event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic procedure counts total count value of the corresponding event of the event identifier in the duration parameters;
Judgment step judges whether the total count value is more than default processing threshold value;
First processing step, for ignoring corresponding to the event in the case where result that judgment step is more than , the part operation of not reproducible processing;
Second processing step is right in the case where the result that judgment step is no more than, executing the event institute The part operation of processing answer, not reproducible;And
Accumulation step, for according to the time of reception of the event identifier, adding up described after second processing step The count value of event.
Fig. 6 is a kind of hardware structural diagram of computer equipment 600 in the embodiment of the present application.Computer equipment 600 can For realizing data distribution, such as it can be client.I.e. computer equipment 600 can be used for executing provides in above-described embodiment Method.In the present embodiment, computer equipment 600 includes: processor 601, memory 602, network interface 603 and total linear system System 604.
The bus system 604, for each hardware component for calculating equipment 600 to be coupled.
The network interface 603, for realizing equipment 600 and at least one other communication calculated between equipment is calculated Connection, can be used internet, wide area network, local network, the modes such as Metropolitan Area Network (MAN).
The memory 602, for storing program instruction and/or data.
The processor 601, for reading the instruction and/or data that store in memory 602, the following operation of execution:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;According to pre- The mark create-rule being first arranged generates the corresponding event identifier of the event data;It, will be described according to the number of server The corresponding event data of event identifier is distributed into the multiple server.
Wherein, described according to pre-set mark create-rule, generate the corresponding event mark of the event data Know, may include: the user identifier for obtaining the event data and including, the type identification and/or operation object mark of operation Know;The user identifier, the type identification of operation and/or operated object mark are combined, the event identifier is obtained.
Wherein, the number according to server, the corresponding event data of the event identifier is distributed to the multiple In server, may include:
For each event data, corresponding event identifier is subjected to Hash operation, obtains the cryptographic Hash of event identifier; Number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value;The event identifier is sent to institute In the server for stating index value mark.
Fig. 7 is the hardware structural diagram of another computer equipment 700 in the embodiment of the present application.Computer equipment 700 It can be used for realizing that data are distributed, such as can be client.I.e. computer equipment 700 can be used for executing in above-described embodiment and mention The method of confession.In the present embodiment, computer equipment 700 includes: processor 701, memory 702, network interface 703 and total Linear system system 704.
The bus system 704, for each hardware component for calculating equipment 700 to be coupled.
The network interface 703, for realizing equipment 700 and at least one other communication calculated between equipment is calculated Connection, can be used internet, wide area network, local network, the modes such as Metropolitan Area Network (MAN).
The memory 702, for storing program instruction and/or data.
The processor 701, for reading the instruction and/or data that store in memory 702, the following operation of execution:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;
Judge whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event , the part operation of not reproducible processing;If be no more than, execute corresponding to the event, the portion of not reproducible processing Divide operation, and according to the time of reception of the event identifier, the count value for the event that adds up.
It wherein, include multiple one-to-one memory spaces and timeslice in the server, when the interval of each timeslice Length is identical, and the memory space includes timestamp and fragmentation count value, and the timestamp is for indicating since the timestamp The corresponding event of event identifier received in the duration of interval, needs to be added in the memory space;It is described according to the event The time of reception of mark, the count value for the event that adds up may include:
The time of reception of the event identifier is calculated divided by the quotient of the interval duration, is stabbed as to storage time, with And the remainder of the number of the time of reception and the memory space is calculated, it is indexed as to memory space;Judgement it is described to Memory space indexes the original time stamp that corresponding memory space includes, with it is described to storage time stab it is whether identical, if It is the corresponding fragmentation count value of the original time stamp that then adds up;If it is not, then emptying, the original time stamp is corresponding to be deposited Store up space, by the original time stamp replace with it is described stabbed to storage time, and it is cumulative it is described to storage time stab it is corresponding, The fragmentation count value of the event.
Wherein, total count value of the corresponding event of the statistics event identifier in the duration parameters, can wrap It includes:
Obtain the time of reception of the event identifier;The quotient of the time of reception and the interval duration are calculated, and should The difference between timestamp that quotient and each memory space include, and using each difference as the query argument of each memory space; Judge whether the query argument of each memory space is less than the duration parameters, if it is, obtaining the memory space and including Fragmentation count value and add up, obtain the total count value of the event identifier.
In practical applications, the data processing method of the embodiment of the present application first can generate thing by the first computer equipment Part mark, wherein the event identifier includes at least or based on one of following data: User ID, timestamp, operation code, operation Object ID;Then the first computer equipment sends the event identifier to second computer equipment, wherein described second calculates Machine equipment is based on the event identifier and carries out deduplication operation.
Wherein, referring to Fig. 8 a~Fig. 8 b, present invention also provides a kind of user interface interaction equipment, which can To include:
First interactive module, for forwarding the first triggering command to processor after receiving first triggering command, So as to processor receiving event data, the event data includes: related data of the user in the operation of foreground system.
After the control (refer to Fig. 8 a) that user clicks " receiving event data " by interactive interface, the of interactive device One interactive module receives first triggering command, and then, the first interactive module forwards first triggering command to processor, to locate Reason device can receive foreground system transmission, event event data, and the event data may include: user in foreground system The related data of the operation of system.
Second interactive module forwards the first triggering command for receiving the second triggering command, and to processor, with Just processor is received according to pre-set mark create-rule, generates the corresponding event identifier of the event data.
After user clicks the control (referring to Fig. 8 a) of " generate event identifier " by interactive interface, the of interactive device Two interactive modules receive the second triggering command, then, the second interactive module to processor forward the second triggering command, so as to according to According to pre-set mark create-rule, the corresponding event identifier of the event data is generated.
Third interactive module forwards the third triggering command for receiving third triggering command, and to processor, with Just number of the processor according to server, the corresponding event data of the event identifier is distributed into the multiple server (referring to Fig. 8 b).
After user clicks the control (referring to Fig. 8 a) of " display allocation result " by interactive interface, the of interactive device Three interactive modules receive third triggering command, then, third interactive module to processor forward third triggering command, so as to according to According to the number of the server, the corresponding event data of the event identifier is distributed into the multiple server, and is received The allocation result that processor is sent is shown.
If function described in the present embodiment method is realized in the form of SFU software functional unit and as independent product pin It sells or in use, can store in a storage medium readable by a compute device.Based on this understanding, the application is implemented The part of example the part that contributes to existing technology or the technical solution can be embodied in the form of software products, The software product is stored in a storage medium, including some instructions are used so that a calculating equipment (can be personal meter Calculation machine, server, mobile computing device or network equipment etc.) execute each embodiment the method for the application whole or Part steps.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), Random access memory (RAM, Random Access Memory), magnetic or disk etc. be various to can store program code Medium.
Wherein, referring to Fig. 9 a~Fig. 9 b, present invention also provides another user interface interaction equipment, the interactive devices May include:
4th interactive module, for forwarding the 4th triggering command to processor after receiving the 4th triggering command, So that processor receives event identifier, and obtain the corresponding duration parameters of the event identifier.
After user clicks the control (referring to Fig. 9 a) of " receive event identifier " by interactive interface, the of interactive device Four interactive modules receive the 4th triggering command, and then, the 4th interactive module forwards the 4th triggering command to processor, to locate Reason device can receive the event identifier that the data distribution device is sent, and obtain the corresponding duration parameters of the event identifier.
5th interactive module forwards the 5th triggering command for receiving the 5th triggering command, and to processor, with Just it receives processor and counts total count value of the corresponding event of the event identifier in the duration parameters;The server In preserve multiple event identifiers and its corresponding count value.
After user clicks the control (referring to Fig. 9 a) of " statistical counting value " by interactive interface, the 5th of interactive device the Interactive module receives the 5th triggering command, and then, the 5th interactive module forwards the 5th triggering command to processor, so as to foundation Pre-set mark create-rule generates the corresponding event identifier of the event data.
6th interactive module forwards the 6th triggering command for receiving the 6th triggering command, and to processor, with Just processor judges whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event , the part operation of not reproducible processing;If be no more than, execute corresponding to the event, the portion of not reproducible processing Divide operation, and according to the time of reception of the event identifier, the count value (referring to Fig. 9 b) for the event that adds up.
After user clicks the control (referring to Fig. 9 a) of " display processing result " by interactive interface, the of interactive device Six interactive modules receive the 6th triggering command, then, the 6th interactive module to processor forward the 6th triggering command, so as to according to According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server, and receives processing The allocation result that device is sent is shown.
If function described in the present embodiment method is realized in the form of SFU software functional unit and as independent product pin It sells or in use, can store in a storage medium readable by a compute device.Based on this understanding, the application is implemented The part of example the part that contributes to existing technology or the technical solution can be embodied in the form of software products, The software product is stored in a storage medium, including some instructions are used so that a calculating equipment (can be personal meter Calculation machine, server, mobile computing device or network equipment etc.) execute each embodiment the method for the application whole or Part steps.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), Random access memory (RAM, Random Access Memory), magnetic or disk etc. be various to can store program code Medium.
It should be noted that all the embodiments in this specification are described in a progressive manner, each embodiment weight Point explanation is the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For device class embodiment, since it is basically similar to the method embodiment, so being described relatively simple, related place Illustrate referring to the part of embodiment of the method.
Finally, it is to be noted that, herein, relational terms such as first and second and the like are used merely to Distinguish one entity or operation from another entity or operation, without necessarily requiring or implying these entities or There are any actual relationship or orders between operation.Moreover, the terms "include", "comprise" or its any other Variant is intended to non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not Only include those elements, but also including other elements that are not explicitly listed, or further includes for this process, side Method, article or the intrinsic element of equipment.In the absence of more restrictions, being limited by sentence "including a ..." Element, it is not excluded that there is also other identical elements in the process, method, article or apparatus that includes the element.
Data distributing method and device provided herein, data processing method and server have been carried out in detail above Thin to introduce, specific examples are used herein to illustrate the principle and implementation manner of the present application, and above embodiments are said It is bright to be merely used to help understand the present processes and its core concept;At the same time, for those skilled in the art, according to According to the thought of the application, there will be changes in the specific implementation manner and application range, in conclusion in this specification Hold the limitation that should not be construed as to the application.

Claims (15)

1. a kind of data processing method, which is characterized in that the data processing method includes:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;
Judge whether the total count value is more than default processing threshold value, if it does, then ignore corresponding to the event, can not The part operation of reprocessing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to the event mark The time of reception of knowledge, the count value for the event that adds up.
2. the method according to claim 1, wherein with multiple timeslices, one-to-one multiple storages are empty respectively Between, the interval duration of each timeslice is identical, and the memory space includes timestamp and fragmentation count value, and the timestamp is used for table Show the corresponding event of event identifier received in the interval duration since the timestamp, needs to be added to the memory space In;The time of reception according to the event identifier, the count value for the event that adds up, comprising:
The time of reception of the event identifier is calculated divided by the quotient of the interval duration, is stabbed as to storage time, and, it calculates The remainder of the number of the time of reception and the memory space, indexes as to memory space;
Judgement is described to index the original time stamp that corresponding memory space includes to memory space, is to storage time stamp with described It is no identical, if it is, the corresponding fragmentation count value of the original time stamp that adds up;
If it is not, then emptying the corresponding memory space of the original time stamp, the original time stamp is replaced with described wait deposit Timestamp is stored up, and cumulative described stabs corresponding, the event fragmentation count value to storage time.
3. according to the method described in claim 2, it is characterized in that, the corresponding event of the statistics event identifier is described Total count value in duration parameters, comprising:
Obtain the time of reception of the event identifier;
It calculates between the timestamp that the time of reception includes with the quotient for being spaced duration and the quotient and each memory space Difference, and using each difference as the query argument of each memory space;
Judge whether the query argument of each memory space is less than the duration parameters, if it is, obtaining the memory space Including fragmentation count value and add up, obtain the total count value of the event identifier.
4. a kind of data distributing method, which is characterized in that this method comprises:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
5. according to the method described in claim 4, it is characterized in that, described according to pre-set mark create-rule, generation The corresponding event identifier of the event data, comprising:
Obtain the type identification and/or operated object mark of user identifier, operation that the event data includes;
The user identifier, the type identification of operation and/or operated object mark are combined, the event identifier is obtained.
6. according to the method described in claim 4, it is characterized in that, the number according to server, by the event identifier Corresponding event data is distributed into the multiple server, comprising:
For each event data, corresponding event identifier is subjected to Hash operation, obtains the cryptographic Hash of event identifier;
Number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value;
The event identifier is sent in the server of the index value mark.
7. a kind of server, which is characterized in that the server includes:
Event identifier receiving unit for receiving event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic unit, for counting total count value of the corresponding event of the event identifier in the duration parameters;
Judging unit, for judging whether the total count value is more than default processing threshold value;
First processing units, for the result of the judging unit be more than in the case where, ignore corresponding to the event, The part operation of not reproducible processing;
The second processing unit, for the result of the judging unit be no more than in the case where, execute corresponding to the event , the part operation of not reproducible processing;
Summing elements, for according to the time of reception of the event identifier, adding up described after triggering described the second processing unit The count value of event.
8. a kind of data distribution device, which is characterized in that the data distribution device includes:
Event data receiving unit is used for receiving event data, and the event data includes: operation of the user in foreground system Related data;
Generation unit, for generating the corresponding event identifier of the event data according to pre-set mark create-rule;
Allocation unit distributes the corresponding event data of the event identifier to the multiple for the number according to server In server.
9. a kind of computer-readable medium, which has the journey for executing computer for data distribution Sequence, the processing the following steps are included:
Receiving step, receiving event data, the event data includes: related data of the user in the operation of foreground system;
Generation step generates the corresponding event identifier of the event data according to pre-set mark create-rule;
Allocation step distributes the corresponding event data of the event identifier to the multiple clothes according to the number of server It is engaged in device.
10. a kind of computer-readable medium, which, which has, is used for computer execution, the processing include with Lower step:
Receiving step receives event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic procedure counts total count value of the corresponding event of the event identifier in the duration parameters;
Judgment step judges whether the total count value is more than default processing threshold value;
First processing step, for ignoring corresponding to the event, no in the case where the result that judgment step is more than The part operation of repeatable processing;
Second processing step, in the case where the result that judgment step is no more than, executing corresponding to the event, The part operation of not reproducible processing;
Accumulation step, for after second processing step, according to the time of reception of the event identifier, to add up the event Count value.
11. a kind of computer equipment, which is characterized in that the computer equipment includes: processor, memory, network interface and total Linear system system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing the communication link between the computer equipment and at least one other computer equipment It connects;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
12. a kind of computer equipment, which is characterized in that the computer equipment includes: processor, memory, network interface and total Linear system system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing the communication link between the computer equipment and at least one other computer equipment It connects;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;It is preserved in the server more A event identifier and its corresponding count value;
Judge whether the total count value is more than default processing threshold value, if it does, then ignore corresponding to the event, can not The part operation of reprocessing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to the event mark The time of reception of knowledge, the count value for the event that adds up.
13. a kind of user interface interaction equipment characterized by comprising
First interactive module, for the first triggering command being forwarded to processor, to locate after receiving first triggering command Device receiving event data is managed, the event data includes: related data of the user in the operation of foreground system;
Second interactive module, for forwarding the first triggering command after receiving the second triggering command, and to processor, so as to Processor generates the corresponding event identifier of the event data according to pre-set mark create-rule;
Third interactive module, for forwarding the third triggering command after receiving third triggering command, and to processor, so as to Number of the processor according to server, the corresponding event data of the event identifier is distributed into the multiple server.
14. a kind of user interface interaction equipment characterized by comprising
4th interactive module, for the 4th triggering command being forwarded to processor, to locate after receiving the 4th triggering command It manages device and receives event identifier, and obtain the corresponding duration parameters of the event identifier;
5th interactive module forwards the 5th triggering command for receiving the 5th triggering command, and to processor, to handle Device counts total count value of the corresponding event of the event identifier in the duration parameters;It is preserved in the server multiple Event identifier and its corresponding count value;
6th interactive module forwards the 6th triggering command for receiving the 6th triggering command, and to processor, to handle Device judges whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event, can not weighing The part operation handled again;If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and According to the time of reception of the event identifier, the count value for the event that adds up.
15. a kind of data processing method characterized by comprising
First, which calculates equipment, generates event identifier, wherein the event identifier includes at least or based on one of following data: user ID, timestamp, operation code, operation object ID;
First, which calculates equipment, calculates the equipment transmission event identifier to second, wherein described second calculates equipment based on described Event identifier carries out deduplication operation.
CN201710612889.2A 2017-07-25 2017-07-25 Data distribution and processing method, system and computer readable recording medium Active CN109302300B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710612889.2A CN109302300B (en) 2017-07-25 2017-07-25 Data distribution and processing method, system and computer readable recording medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710612889.2A CN109302300B (en) 2017-07-25 2017-07-25 Data distribution and processing method, system and computer readable recording medium

Publications (2)

Publication Number Publication Date
CN109302300A true CN109302300A (en) 2019-02-01
CN109302300B CN109302300B (en) 2022-03-15

Family

ID=65167366

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710612889.2A Active CN109302300B (en) 2017-07-25 2017-07-25 Data distribution and processing method, system and computer readable recording medium

Country Status (1)

Country Link
CN (1) CN109302300B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110442466A (en) * 2019-06-14 2019-11-12 平安科技(深圳)有限公司 Prevent request repeated accesses method, apparatus, computer equipment and storage medium
CN112131267A (en) * 2020-08-14 2020-12-25 北京达佳互联信息技术有限公司 Counting processing method, device, server and counting processing system
CN112260902A (en) * 2020-10-23 2021-01-22 深圳前海微众银行股份有限公司 Network equipment monitoring method, device, equipment and storage medium
CN112929379A (en) * 2021-02-22 2021-06-08 深圳供电局有限公司 Intelligent recorder remote operation and maintenance instruction defense method and system
CN113139097A (en) * 2020-01-19 2021-07-20 阿里巴巴集团控股有限公司 Data writing method, data reading method, device and equipment

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20010067511A (en) * 2001-02-02 2001-07-13 허경진 최병무 A method for reducing a telecommunication fee using a secret number and an intrinsic number of goods
CN1762141A (en) * 2003-11-10 2006-04-19 Eath株式会社 Compilation system
US20090070336A1 (en) * 2007-09-07 2009-03-12 Sap Ag Method and system for managing transmitted requests
CN101009586B (en) * 2006-01-27 2010-09-08 华为技术有限公司 Processing method for alarm filtering condition in the alarming system
JP4677686B2 (en) * 2001-06-15 2011-04-27 株式会社ニコン Interface device and recording medium recording program
CN102541918A (en) * 2010-12-30 2012-07-04 阿里巴巴集团控股有限公司 Method and equipment for identifying repeated information
CN102663590A (en) * 2012-02-29 2012-09-12 向亚峰 System and method of commodity anti-counterfeiting authentication based on restriction of authentication frequency
CN103164511A (en) * 2013-02-21 2013-06-19 烽火通信科技股份有限公司 Method for storage event log to automatically filter repeated jitter data
CN103297468A (en) * 2012-02-29 2013-09-11 华为技术有限公司 Operation method for group resources and group server
FR2993685A1 (en) * 2012-07-17 2014-01-24 France Telecom Data processing machine i.e. networked server, monitoring method, involves repeating collection of history of events, choosing type of event, and assignment of calculation value, and obtaining variation of value with regard to time
CN103678471A (en) * 2012-09-24 2014-03-26 国际商业机器公司 Method and device for partitioning search space for distributed crawling
CN104077701A (en) * 2014-06-09 2014-10-01 中国建设银行股份有限公司 Task processing method and device used for e-business platform
CN104508700A (en) * 2012-07-27 2015-04-08 谷歌公司 Determining a correlation between presentation of a content item and a transaction by a user at a point of sale terminal
CN104618432A (en) * 2014-12-30 2015-05-13 北京红马传媒文化发展有限公司 Event sending and receiving handling method and system
CN104769555A (en) * 2012-06-18 2015-07-08 艾克特菲欧有限公司 Enhanced data management virtualization system
US20160055200A1 (en) * 2008-08-18 2016-02-25 Guidance Software, Inc. Scalable deduplication system and method
CN105468699A (en) * 2015-11-18 2016-04-06 珠海多玩信息技术有限公司 Duplicate removal data statistics method and equipment
CN106296246A (en) * 2015-05-26 2017-01-04 腾讯科技(深圳)有限公司 Method for processing business and device
CN106899666A (en) * 2017-02-21 2017-06-27 阿里巴巴集团控股有限公司 A kind of data processing method and device for service identification

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20010067511A (en) * 2001-02-02 2001-07-13 허경진 최병무 A method for reducing a telecommunication fee using a secret number and an intrinsic number of goods
JP4677686B2 (en) * 2001-06-15 2011-04-27 株式会社ニコン Interface device and recording medium recording program
CN1762141A (en) * 2003-11-10 2006-04-19 Eath株式会社 Compilation system
CN101009586B (en) * 2006-01-27 2010-09-08 华为技术有限公司 Processing method for alarm filtering condition in the alarming system
US20090070336A1 (en) * 2007-09-07 2009-03-12 Sap Ag Method and system for managing transmitted requests
US20160055200A1 (en) * 2008-08-18 2016-02-25 Guidance Software, Inc. Scalable deduplication system and method
CN102541918A (en) * 2010-12-30 2012-07-04 阿里巴巴集团控股有限公司 Method and equipment for identifying repeated information
CN103297468A (en) * 2012-02-29 2013-09-11 华为技术有限公司 Operation method for group resources and group server
CN102663590A (en) * 2012-02-29 2012-09-12 向亚峰 System and method of commodity anti-counterfeiting authentication based on restriction of authentication frequency
CN104769555A (en) * 2012-06-18 2015-07-08 艾克特菲欧有限公司 Enhanced data management virtualization system
FR2993685A1 (en) * 2012-07-17 2014-01-24 France Telecom Data processing machine i.e. networked server, monitoring method, involves repeating collection of history of events, choosing type of event, and assignment of calculation value, and obtaining variation of value with regard to time
CN104508700A (en) * 2012-07-27 2015-04-08 谷歌公司 Determining a correlation between presentation of a content item and a transaction by a user at a point of sale terminal
CN103678471A (en) * 2012-09-24 2014-03-26 国际商业机器公司 Method and device for partitioning search space for distributed crawling
CN103164511A (en) * 2013-02-21 2013-06-19 烽火通信科技股份有限公司 Method for storage event log to automatically filter repeated jitter data
CN104077701A (en) * 2014-06-09 2014-10-01 中国建设银行股份有限公司 Task processing method and device used for e-business platform
CN104618432A (en) * 2014-12-30 2015-05-13 北京红马传媒文化发展有限公司 Event sending and receiving handling method and system
CN106296246A (en) * 2015-05-26 2017-01-04 腾讯科技(深圳)有限公司 Method for processing business and device
CN105468699A (en) * 2015-11-18 2016-04-06 珠海多玩信息技术有限公司 Duplicate removal data statistics method and equipment
CN106899666A (en) * 2017-02-21 2017-06-27 阿里巴巴集团控股有限公司 A kind of data processing method and device for service identification

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110442466A (en) * 2019-06-14 2019-11-12 平安科技(深圳)有限公司 Prevent request repeated accesses method, apparatus, computer equipment and storage medium
CN110442466B (en) * 2019-06-14 2024-05-07 平安科技(深圳)有限公司 Method, device, computer equipment and storage medium for preventing repeated access request
CN113139097A (en) * 2020-01-19 2021-07-20 阿里巴巴集团控股有限公司 Data writing method, data reading method, device and equipment
CN112131267A (en) * 2020-08-14 2020-12-25 北京达佳互联信息技术有限公司 Counting processing method, device, server and counting processing system
CN112131267B (en) * 2020-08-14 2023-10-03 北京达佳互联信息技术有限公司 Count processing method, device, server and count processing system
CN112260902A (en) * 2020-10-23 2021-01-22 深圳前海微众银行股份有限公司 Network equipment monitoring method, device, equipment and storage medium
CN112929379A (en) * 2021-02-22 2021-06-08 深圳供电局有限公司 Intelligent recorder remote operation and maintenance instruction defense method and system

Also Published As

Publication number Publication date
CN109302300B (en) 2022-03-15

Similar Documents

Publication Publication Date Title
CN109302300A (en) Data distributing method and device, data processing method and server
Luu et al. {SmartPool}: Practical decentralized pooled mining
CN105100162B (en) Virtual objects sending method and device, method of reseptance and device, system
WO2016131365A1 (en) Information processing method, client, server and computer readable storage medium
KR102282763B1 (en) Service implementation method and device
CN108615151B (en) Data processing method, block chain server and node equipment
CN106470133A (en) System pressure method of testing and device
CN106411777A (en) Method and system for processing high concurrent data
US10587650B2 (en) Communications security
CN107360006A (en) A kind of resource billing method and device
EP3879477A1 (en) Electronic certificate-based interaction method and apparatus, and electronic device
CN106649637B (en) Method and system for processing red packet
JP2018507475A (en) Service implementation method, payment method, and apparatus
CN110377854A (en) User access activity information monitoring method and device, computer equipment
CN110365712A (en) A kind of defence method and system of distributed denial of service attack
CN107786628A (en) Business numbering distribution method, device, computer equipment and storage medium
CN111367621A (en) Intelligent contract timing processing method, block chain node and storage medium
CN106302111A (en) A kind of information processing method, terminal and server
CN104184603B (en) A kind of User Status statistical method, apparatus and system
CN116846980B (en) Integral issuing and using method, system and equipment
CN111741067B (en) Block chain-based tableware circulation information management method and device and block chain system
CN104462116B (en) Data selection method and device
CN108959047A (en) A kind of method for testing pressure and device based on business scenario
CN105184559A (en) System and method for payment
CN105100072B (en) A kind of network node monitoring method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant