CN109302300A - Data distributing method and device, data processing method and server - Google Patents
Data distributing method and device, data processing method and server Download PDFInfo
- Publication number
- CN109302300A CN109302300A CN201710612889.2A CN201710612889A CN109302300A CN 109302300 A CN109302300 A CN 109302300A CN 201710612889 A CN201710612889 A CN 201710612889A CN 109302300 A CN109302300 A CN 109302300A
- Authority
- CN
- China
- Prior art keywords
- event
- data
- event identifier
- count value
- identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
- H04L41/064—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving time analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0633—Lists, e.g. purchase orders, compilation or processing
- G06Q30/0635—Processing of requisition or of purchase orders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/16—Threshold monitoring
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Information Transfer Between Computers (AREA)
Abstract
This application provides data distributing method and devices, data processing method and server, wherein, data processing method is applied in any server being connected with data distribution device, data processing method includes: the event identifier for receiving data distribution device and sending, and obtains the corresponding duration parameters of event identifier;Statistical phenomeon identifies total count value of the corresponding event in the duration parameters;Multiple event identifiers and its corresponding count value are preserved in server;Judge whether total count value is more than default processing threshold value, if it does, then ignoring corresponding to event, the part operation of not reproducible processing;If be no more than, execute corresponding to event, the part operation of not reproducible processing, and according to the time of reception of event identifier, the count value for the event that adds up.Using the embodiment of the present application, the mistake of user's operation generation can be also executed, user experience is promoted to avoid in the case where event data generates duplicate situation.
Description
Technical field
This application involves internet data processing technology field, in particular to a kind of data distributing method and device, with
And a kind of data processing method and server.
Background technique
In the present of e-commerce rapid development, more and more users carry out various online transactions using internet, or
Person transmits information etc..And for the operation that user generates via terminal on the internet, corresponding server can be sent to and held
Row.For example, user logs in the operation of a mailbox, user name and login password can be sent to mailbox server via terminal,
Mailbox server can generate the corresponding event of the operation and event data, include in event data and user's operation
Relevant data.For another example lower single operation of the user in an online shopping mall to certain product, user also can incite somebody to action oneself via terminal
The information such as oneself user name, posting address or the product that places an order are sent to the server of online shopping mall, generate phase by server
As soon as the lower single event answered, the event data of the lower single event include the information of user name, address and product etc..
It but in practical applications, can due to network delay or equipment fault or user repeat submission etc.
The phenomenon that same lower single operation is repeatedly sent to the server of online shopping mall can be will cause, in this case, will cause
It repeats to send.
How this event data for repeating to send is handled, technical problem referred to as urgently to be resolved.
Summary of the invention
Inventor has found in the course of the research, the prior art can not counterweight recur the event data accurate judgement sent, according to
The application one aspect, if can be after server receives user's operation, by the event data of generation according to each event number
It is counted according to corresponding event identifier, is had submitted repeatedly if the corresponding event of certain event identifier is repeated in a short time,
Then no longer respond the corresponding user's operation of the event, so that it may which whether the corresponding event of one event data of accurate judgement is weight
Multiple event, and then can determine to whether the corresponding user's operation of the event needs to respond.
Based on this, this application provides a kind of data distributing method and a kind of data processing methods, each to be adopted as
A event data generate relative to event identifier, and by event data according to of the processing whether duplicate server of event
Number, to be respectively allocated in each server, and each server then counts some event identifier of oneself storage in one section of thing
Whether number in part is more than default processing threshold value, and those of the corresponding not reproducible processing of the event is ignored as if being more than
Operation, the event data that not only ensure that user's operation generates will not be repeatedly processed through vaporization, condensation and absorption, such as will not repeat and withhold
Integration operation etc. is operated or reduces, also the user experience is improved.
Present invention also provides a kind of data distribution device and servers, to guarantee the reality of the above method in practice
Existing and application.
To solve the above-mentioned problems, this application discloses a kind of data processing method, the data processing method includes:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;
Judge whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event
, the part operation of not reproducible processing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to described
The time of reception of event identifier, the count value for the event that adds up.
Wherein, with the one-to-one multiple memory spaces of multiple timeslices difference, the interval duration of each timeslice is identical,
The memory space includes timestamp and fragmentation count value, when the timestamp is used to indicate the interval since the timestamp
The corresponding event of the event identifier received in length, needs to be added in the memory space;It is described according to the event identifier
The time of reception, the count value for the event that adds up, comprising:
The time of reception of the event identifier is calculated divided by the quotient of the interval duration, is stabbed as to storage time, with
And the remainder of the number of the time of reception and the memory space is calculated, it is indexed as to memory space;
Judgement is described to index the original time stamp that corresponding memory space includes to memory space, with described when storing
Between stab it is whether identical, if it is, the corresponding fragmentation count value of the original time stamp that adds up;
If it is not, then emptying the corresponding memory space of the original time stamp, the original time stamp is replaced with described
It is stabbed to storage time, and cumulative described stabs corresponding, the event fragmentation count value to storage time.
Wherein, total count value of the corresponding event of the statistics event identifier in the duration parameters, comprising:
Obtain the time of reception of the event identifier;
The timestamp that the quotient and the quotient for calculating the time of reception and the interval duration include with each memory space
Between difference, and using each difference as the query argument of each memory space;
Judge whether the query argument of each memory space is less than the duration parameters, if it is, obtaining the storage
Fragmentation count value that space includes simultaneously adds up, and obtains the total count value of the event identifier.
The embodiment of the present application also provides a kind of data distributing methods, this method comprises:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
It is wherein, described to generate the corresponding event identifier of the event data according to pre-set mark create-rule,
Include:
Obtain the type identification and/or operated object mark of user identifier, operation that the event data includes;
The user identifier, the type identification of operation and/or operated object mark are combined, the event identifier is obtained.
Wherein, the number according to server, the corresponding event data of the event identifier is distributed to the multiple
In server, comprising:
For each event data, corresponding event identifier is subjected to Hash operation, obtains the cryptographic Hash of event identifier;
Number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value;
The event identifier is sent in the server of the index value mark.
The embodiment of the present application also provides a kind of server, the server includes:
Event identifier receiving unit for receiving event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic unit, for counting total count value of the corresponding event of the event identifier in the duration parameters;
Judging unit, for judging whether the total count value is more than default processing threshold value;
First processing units, for the result of the judging unit be more than in the case where, it is right to ignore event institute
The part operation of processing answer, not reproducible;
The second processing unit, for the result of the judging unit be no more than in the case where, execute the event institute
The part operation of corresponding, not reproducible processing;
Summing elements, for according to the time of reception of the event identifier, tiring out after triggering described the second processing unit
Add the count value of the event.
Wherein, with the one-to-one multiple memory spaces of multiple timeslices difference, the interval duration of each timeslice is identical,
The memory space includes timestamp and fragmentation count value, when the timestamp is used to indicate the interval since the timestamp
The corresponding event of the event identifier received in length, needs to be added in the memory space;The summing elements include:
Second computation subunit, the time of reception for calculating the event identifier are made divided by the quotient of the interval duration
To be stabbed to storage time, and, the remainder of the number of the time of reception and the memory space is calculated, as sky to be stored
Between index;
Second judgment sub-unit, for judge it is described when memory space index that corresponding memory space includes it is original when
Between stab, with it is described to storage time stab it is whether identical;
Second cumulative subelement, for the result of the judgment sub-unit be in the case where, when adding up described original
Between stab corresponding fragmentation count value;
Subelement is emptied, for emptying the original time stamp pair in the case where second judgment sub-unit is no
The memory space answered replaces with the original time stamp described to storage time stamp;
Third adds up subelement, described stabs corresponding, the event fragmentation count value to storage time for cumulative.
Wherein, the statistic unit includes:
Second obtains subelement, for obtaining the time of reception of the event identifier;
First computation subunit, for calculating the time of reception and the quotient for being spaced duration and the quotient and respectively depositing
The difference between timestamp that storage space includes, and using each difference as the query argument of each memory space;
First judgment sub-unit, for judging whether the query argument of each memory space is less than the duration parameters;
First cumulative subelement, for obtaining the memory space in the case where the result of the judgment sub-unit, which is, is
Including fragmentation count value and add up, obtain the total count value of the event identifier.
The embodiment of the present application also provides a kind of data distribution device, which includes:
Event data receiving unit is used for receiving event data, and the event data includes: user is in foreground system
The related data of operation;
Generation unit, for generating the corresponding event mark of the event data according to pre-set mark create-rule
Know;
Allocation unit distributes the corresponding event data of the event identifier to described for the number according to server
In multiple servers.
Wherein, the generation unit includes:
First obtains subelement, for obtaining the type identification of user identifier, operation that the event data includes
And/or operated object mark;
Combination subelement is obtained for combining the type identification and/or operated object mark of the user identifier, operation
The event identifier.
Wherein, the allocation unit includes:
Hash operation subelement, for corresponding event identifier being carried out Hash operation, is obtained for each event data
To the cryptographic Hash of event identifier;
Modulo operation subelement obtains rope for number of the cryptographic Hash to the server to be carried out modulo operation
Draw value;
Transmission sub-unit, for the event identifier to be sent in the server of the index value mark.
The embodiment of the present application also provides a kind of computer-readable medium, which has and makes to calculate
Machine execute for data distribution program, the processing the following steps are included:
Receiving step, receiving event data, the event data includes: correlation of the user in the operation of foreground system
Data;
Generation step generates the corresponding event mark of the event data according to pre-set mark create-rule
Know;
Allocation step distributes the corresponding event data of the event identifier to described more according to the number of server
In a server.
The embodiment of the present application also provides another computer-readable medium, which has and makes to count
Calculation machine execution be used for, the processing the following steps are included:
Receiving step receives event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic procedure counts total count value of the corresponding event of the event identifier in the duration parameters;
Judgment step judges whether the total count value is more than default processing threshold value;
First processing step, for ignoring corresponding to the event in the case where result that judgment step is more than
, the part operation of not reproducible processing;
Second processing step is right in the case where the result that judgment step is no more than, executing the event institute
The part operation of processing answer, not reproducible;
Accumulation step, for according to the time of reception of the event identifier, adding up described after second processing step
The count value of event.
The embodiment of the present application also provides a kind of computer equipment, which includes: processor, memory, net
Network interface and bus system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing logical between the computer equipment and at least one other computer equipment
Letter connection;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
The embodiment of the present application also provides another computer equipment, the computer equipment include: processor, memory,
Network interface and bus system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing logical between the computer equipment and at least one other computer equipment
Letter connection;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;It is protected in the server
There are multiple event identifiers and its corresponding count value;
Judge whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event
, the part operation of not reproducible processing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to described
The time of reception of event identifier, the count value for the event that adds up.
The embodiment of the present application also provides a kind of user interface interaction equipment, comprising:
First interactive module, for forwarding the first triggering command to processor after receiving first triggering command,
So as to processor receiving event data, the event data includes: related data of the user in the operation of foreground system;
Second interactive module, for referring to after receiving the second triggering command, and to processor forwarding first triggering
It enables, so that processor is according to pre-set mark create-rule, generates the corresponding event identifier of the event data;
Third interactive module, for forwarding the third triggering to refer to after receiving third triggering command, and to processor
It enables, the number so as to processor according to server distributes the corresponding event data of the event identifier to the multiple service
In device.
The embodiment of the present application also provides another user interface interaction equipment, comprising:
4th interactive module, for forwarding the 4th triggering command to processor after receiving the 4th triggering command,
So that processor receives event identifier, and obtain the corresponding duration parameters of the event identifier;
5th interactive module forwards the 5th triggering command for receiving the 5th triggering command, and to processor, with
Just processor counts total count value of the corresponding event of the event identifier in the duration parameters;It is protected in the server
There are multiple event identifiers and its corresponding count value;
6th interactive module forwards the 6th triggering command for receiving the 6th triggering command, and to processor, with
Just processor judges whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event
, the part operation of not reproducible processing;If be no more than, execute corresponding to the event, the portion of not reproducible processing
Divide operation, and according to the time of reception of the event identifier, the count value for the event that adds up.
The embodiment of the present application also provides a kind of data processing methods, comprising:
First calculate equipment generate event identifier, wherein the event identifier include at least or based on following data it
One: User ID, timestamp, operation code and operation object ID;
First, which calculates equipment, calculates the equipment transmission event identifier to second, wherein the second calculating equipment is based on
The event identifier carries out deduplication operation.
Compared with prior art, the application includes following advantages:
In the embodiment of the present application, data distribution device can generate each event number according to preset mark create-rule
According to corresponding event identifier, and when distributing each event identifier to server, the number according to server is allocated,
Guarantee that the same event identifier can be assigned to being handled on the same server, so that server statistics event identifier
Be not in when count value mistake or careless omission, also, because each event identifier distribute to each server be it is random,
Therefore, data skew problem of the event identifier between each server is also avoided to a certain extent.
Further, after each server receives event identifier, can will a period of time in store the event identifier
Corresponding fragmentation count value adds up, to obtain the corresponding total quantity of the event identifier, then by the total quantity and in advance
If processing threshold value is judged, to just no longer respond the event institute of the event identifier when being more than default processing threshold value
Corresponding user's operation also avoids the mistake that user's operation generation can be also executed in the case where event data generates duplicate situation
Accidentally, simultaneously as avoiding the repetition of user being withholdd or reduced the relevant operations such as integral, also the user experience is improved.
Further, since server internal can also mark off multiple memory spaces, each memory space is stored
The fragmentation count value of the event identifier received in a certain section of event, so as to rapidly find each memory space storage
Fragmentation count value add up, obtain the total quantity of event, ensure that data processing speed and accuracy.
Certainly, any product for implementing the application does not necessarily require achieving all the advantages described above at the same time.
Detailed description of the invention
In order to more clearly explain the technical solutions in the embodiments of the present application, required in being described below to embodiment
The attached drawing used is briefly described, it should be apparent that, the drawings in the following description are only some examples of the present application,
For those of ordinary skill in the art, without any creative labor, it can also obtain according to these attached drawings
Obtain other attached drawings.
Fig. 1 is the illustrative diagram of the application scenarios of the application in practical applications;
Fig. 2 is the exemplary process diagram of the data distributing method embodiment of the application;
Fig. 3 is the exemplary process diagram of the data processing method embodiment of the application;
Fig. 4 is the exemplary block diagram of the data distribution device embodiment of the application;
Fig. 5 is the exemplary block diagram of the data processing equipment embodiment of the application;
Fig. 6 is a kind of exemplary block diagram of computer equipment of the application;
Fig. 7 is the exemplary block diagram of another computer equipment of the application;
Fig. 8 a~Fig. 8 b is a kind of interface schematic diagram of user interface interaction equipment of the application;
Fig. 9 a~Fig. 9 b is the interface schematic diagram of another user interface interaction equipment of the application.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.It is based on
Embodiment in the application, it is obtained by those of ordinary skill in the art without making creative efforts it is all its
His embodiment, shall fall in the protection scope of this application.
Refering to what is shown in Fig. 1, being the exemplary scene schematic diagram of the embodiment of the present application in practical applications.Wherein, it uses
Family produces operation in foreground system 101, for example, lower single operation, register, alternatively, the edit operation etc. to product,
A referred to as event.Event data is the relevant data content of the operation, for example, the user name to place an order, order number are stepped on
Record password, etc..The event that foreground system 101 is generated in response to the operation of user, can be sent to number for each event data
According to distributor 102, each event data is allocated by data distribution device 102, and the event identifier of each event is sent out
It send into each server being connected with data distribution device 102.In Fig. 1, the number of server can be set to N number of, and N is
Integer greater than 1.In server 103, shown in embodiment as shown in Figure 3, server 103 can be to distributing to oneself
Event identifier is counted, and judges whether the total count value (i.e. quantity) of the corresponding event of event identifier does not meet threshold value, and root
It is judged that result operates to determine execution or ignore the part of operation corresponding with the event, not reproducible, thus
The duplicate removal processing for realizing each event, guarantee the same event will not repeat it is corresponding, cause damages to user certain
A little operations, for example, withholing or deducting the operation such as integral.
Below with reference to application scenarios shown in FIG. 1, data distributing method and data processing method to the application carry out detailed
It is thin to introduce.With reference to Fig. 2, a kind of flow chart of data distributing method embodiment of the application is shown, the present embodiment can be applied to figure
On data distribution device shown in 1, the present embodiment can with the following steps are included:
Step 201: receiving event data, the event data may include: the correlation of operation of the user in foreground system
Data.
For user after foreground system generates operation, the corresponding event data of the operation is sent to data by foreground system
Distributor.Wherein, event data may include the related data that user operates in foreground system.For example, user is at the terminal
After browsing product details, the lower single operation of product A is had submitted to foreground system by terminal, then foreground system is under this
Single operation generates a lower single event, may include: user name, O/No., lower single object in the event data of lower single event
Mark and posting address etc., the event data of lower single event is sent to data distribution device again by foreground system.
Step 202: according to pre-set mark create-rule, generating the corresponding event identifier of the event data.
After data distribution device receives event data, can according to the mark create-rule pre-set, come for
Each event identifier generates event identifier corresponding with each event data.Specifically, data distribution device can be according to setting in advance
The mark create-rule set, to generate the corresponding event identifier of each event data, for example, those skilled in the art set in advance
Mark create-rule is set are as follows: each event data generates unique corresponding event identifier according to different principle, can incite somebody to action
The contents such as time, user name, O/No. are combined, to generate unique corresponding event identifier for each event data.
In a kind of possible embodiment, step 202 specifically can first obtain the user identifier for including in event data
(can be user name), operation type identification and/or operated object mark, recombinant user identifier, operation type identification
And/or operated object mark etc., obtain the corresponding event identifier of each event data.Wherein, the type identification of operation is for indicating
The action type of the corresponding event of event data logs in for example, placing an order, the action types such as modification, or deletion.Operation object
Mark is for uniquely indicating the operation object of the corresponding event of event data, for example, commodity 156789, i.e., unique corresponding
Commodity represented by 156789.
Specifically, the event identifier that user identifier, the type identification of operation and operated object mark obtain after being combined
It can be shaped like: event identifier=User ID _ operation type identification _ operated object mark ID.For example, 12345_ deduction of points _ deduction of points
The type, _ commodity 1234, etc. alternatively, 12345_ places an order.Wherein, User ID " 12345 " is used for one user of unique identification, behaviour
The type identification " deduction of points " of work or " placing an order " are used to indicate that the type of user's operation to be, operated object mark " deduction of points class
Type " or " commodity 1234 " are used for the targeted operation object of one user's operation of unique identification.Implemented according to the application one
Example, further, " deduction of points " corresponding digital code 001, " deduction of points type " correspond to digital code 7, then event identifier can be
12345_001_7。
Certainly, specific data above-mentioned are used for the purpose of those skilled in the art understand that the application facilitates that is illustrated to show
Example property data, should not be construed as the restriction of the embodiment of the present application.
Step 203: according to the number of server, the corresponding event data of the event identifier being distributed to the multiple
In server.
It can be each event point after data distribution device is the corresponding each event identifier of generation of each event data
Most its that server distributed is sent with corresponding server, and by the corresponding event identifier of each event.Specifically,
Data distribution device can distribute each event identifier according to step A1~step A3 as follows:
Step A1: for each event data, corresponding event identifier is subjected to Hash operation, obtains event identifier
Cryptographic Hash.
Firstly, for event identifier obtained in step 202, Hash operation is carried out to event identifier respectively, for example, can
To carry out Hash operation to each event identifier, obtain the Hash of each event identifier using hash algorithms such as CRC32, MD5
Value.
Step A2: number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value.
In order to guarantee to lead to the problem of data skew between each server, also, the same event identifier can divide
It is assigned in the same server, modulo operation is carried out to number of the cryptographic Hash of each event identifier to server in this step,
Obtain that each event identifier is corresponding, index value of server.For example, it is assumed that server has three: server 1,2 and of server
Server 3, then the index value that event identifier respectively may obtain 3 progress modulo operations is 0,1 or 2, then will be indexed
The event identifier that value is 0 is distributed to server 1, will be obtained the event identifier that index value is 1 and is distributed to server 2, and will obtain
The event identifier that index value is 2 is distributed to server 3.
Step A3: the event identifier is sent in the server of the index value mark.
Then, event identifier is respectively sent in that server corresponding with index value by data distribution device again, example
Such as, all event identifiers that index value is 0 will be obtained and be all sent to server 1, all things of the index value for 1 will be obtained
Part mark is all sent to server 2, and, all event identifiers that index value is 2 will be obtained and be all sent to server 3.
As it can be seen that in the embodiment of the present application, data distribution device can generate each thing according to preset mark create-rule
Number of packages according to corresponding event identifier, and when distributing each event identifier to server, divided by the number according to server
Match, guarantees that the same event identifier can be assigned to being handled on the same server, so that server statistics event identifier
Count value when be not in mistake or careless omission, also, because each event identifier to distribute to each server be random
, therefore, data skew problem of the event identifier between each server is also avoided to a certain extent.
In the embodiment of the present application, after event identifier is sent to each server by data distribution device, in each server
In, the number that the corresponding event of event identifier is submitted by user in single timeslice is counted for convenience, and each server can divide
Multiple one-to-one memory spaces and timeslice are not set.Wherein, the interval duration of each timeslice is identical, each memory space
Timestamp and fragmentation count value are respectively included, which is used to indicate to receive in the interval duration since the timestamp
The corresponding event of event identifier, need to be added in memory space belonging to the timestamp.
For example, 3 memory spaces store:store1, store2 and store3 are arranged, respectively for server
The size of store can not be identical, and the interval duration interval of the corresponding timeslice of each store is identical, and interval can be with
The quantity of Millisecond is taken, such as interval is 2 milliseconds.Each store includes a timestamp and a fragmentation count value,
The value ts of timestamp record can be used to indicate that the event in this period of time of ts+interval since ts, corresponding
Event identifier should be added to the fragmentation count value that memory space belonging to the timestamp includes.The fragmentation count value can lead to
Mapping table is crossed to realize, that is, each memory space is corresponding, the fragmentation count value of event identifier that receives can be stored in one
It opens in mapping table, event identifier is saved in the mapping table and its starts point into this period of time of ts+interval in ts
Section count value.
It is assumed that there are 3 memory spaces store1, store2 and store3,3 timeslices are respectively corresponded: " 2017/5/
9 00:00:01 ", " 2017/5/9 00:00:02 " and " 2017/5/9 00:00:03 ", i.e., store1 is saved " 2017/
The count value of the event identifier entered in the 5/9 00:00:01 " period, and so on, store3 is saved " 2017/5/9
The count value of the event identifier entered in the 00:00:03 " period, i.e., each memory space store 1 second data, i.e.,
Interval=1000 (millisecond).Assuming that this 3 timeslices corresponding time of reception are as follows: 1494259201000 (millis
Second), 1494259202000 (milliseconds) and 1494259203000 (milliseconds).Then the corresponding timestamp of memory space store is just
It is the quotient of the time of reception and interval duration, the i.e. corresponding timestamp ts1=1494259201000/ of memory space store1
The corresponding timestamp ts2=1494259202000/interval=of interval=1494259201, memory space store2
The corresponding ts3=1494259203000/interval=1494259203 of 1494259202, memory space store3.
In the embodiment of the present application, n memory space in the server can be recycled.Still with 3 memory spaces
For respectively corresponding 3 seconds timeslices, store1 stores the fragmentation count value of the event identifier received in first second,
The fragmentation count value for the event identifier that store2 storage receives in second second, and what store3 was received in the storage third second
The fragmentation count value of event identifier, and so on, when the 4th second, stored the 4th second by first memory space store1
The fragmentation count value of the event identifier received is stored the 5th second when the 5th second by second memory space store2
Fragmentation count value of the event identifier received, etc..
With reference to Fig. 3, a kind of flow chart of data processing method embodiment of the application, the processing method application are shown
In any server 103 being connected with data distribution device, the present embodiment may comprise steps of 301~step 306:
Step 301: receiving event identifier, and obtain the corresponding duration parameters of the event identifier.
After each event identifier is sent to each server according to index value by data distribution device, for each
It is available to arrive the corresponding duration parameters of the event identifier after receiving event identifier for server, so that server comes
Count count value of the corresponding event of the event identifier in the time range that the duration parameters are illustrated.Wherein, the duration parameters
It can be configured and be stored in each server by those skilled in the art in advance, it, can for the event of each type
The time range of all events of the type is indicated so that a duration parameters are arranged.For example, for this seed type that places an order
Event, the count value in 2 seconds can be inquired, then for the event of lower single type, duration parameters are exactly 2 seconds.Certainly, ability
Field technique personnel may be that duration parameters preset other numerical value.
Step 302: counting total count value of the corresponding event of the event identifier in the duration parameters.
Because identical event identifier will be sent to the same server, institute by the distribution of data distribution device
With, it is only necessary to the corresponding total count value of the event identifier that the server statistics receive oneself, without considering other services
Device.
Specifically, memory space is needed with to determine according to each memory space corresponding timestamp for including in server
Count the fragmentation count value recorded in which memory space, then by the fragmentation count value in each memory space to be counted
Carry out the cumulative total count value that event identifier can be obtained.A kind of embodiment of step 302 may include step B1~step
B4:
Step B1: the time of reception of the event identifier is obtained.
Firstly, determining the time of reception of event identifier A to be counted.It is assumed that event identifier A is 9 days 00 May in 2017:
Server 1 receives at the time of 00:03, then corresponding time of reception time=1494259203000.
Step B2: calculating the time of reception and the quotient for being spaced duration and the quotient and each memory space includes
Difference between timestamp, and using each difference as the query argument of each memory space.
In this step, the time of reception " 2017/5/9 00:00:03 " of calculating and the quotient of interval duration are
1494259203000/interval=1494259203, then by the quotient " 1494259203 " respectively with store1, store2 and
The corresponding timestamp 1494259201,1494259202 and 1494259203 of store3 subtracts each other, and respectively obtains three differences are as follows:
2,1 and 0, i.e., the query argument that the query argument of store1 is 2, store2 is 1, and, the query argument of store3 is 0.
Step B3: judging whether the query argument of each memory space is less than the duration parameters, if it is, into
Enter step B4.
For above-mentioned 3 memory spaces, judge whether its corresponding query argument is less than the value of duration parameters respectively, i.e.,
Whether the corresponding query argument of store1, store2 and store3 is judged less than 2, and judgement obtains looking into for store2 and store3
Parameter is ask less than 2, and the query argument of store1 is equal to 2.
Step B4: it obtains the fragmentation count value that the memory space includes and adds up, obtain the total of the event identifier
Count value.
Then it is directed to store2 and store3, A pairs of event identifier will just recorded in the mapping table in the two memory spaces
The fragmentation count value answered adds up, and obtains the total count value of event identifier A.For example, event identifier in the mapping table of store2
The fragmentation count value that the fragmentation count value of A is event identifier A in the mapping table of 2, store3 is 1, then event identifier A is in duration
Total count value in parameter 2 seconds is 2+1=3.
Step 303: judging whether the total count value is more than default processing threshold value, if it does, then entering step 304;
If be less than, 305 are entered step.
Next, it is determined that whether the total count value that step 302 obtains is more than default processing threshold value, for example, threshold can will be handled
Value is set in advance as 2, and the total count value that event identifier A is calculated has been more than default processing threshold value, then enters step 304.
Step 304: ignoring corresponding to the event, the part operation of not reproducible processing.
In practical applications, having the corresponding operation of some events is not reproducible processing, for example, user is to same
Payment request when a order is paid the bill will be right if repetition has submitted payment request due to webpage delay etc.
User, which executes, repeats operation of withholing, and this repetitive operation will bring very bad influence to user experience.Therefore, in this reality
It applies in example, for the corresponding operation of event of this type, for example, the operation of user integral is reduced, or operation of withholing, with
And the disable operation etc. to user account, corresponding processing threshold value can be preset, when the event mark of discovery event
The total count value of knowledge has been more than default processing threshold value, can directly be ignored corresponding to event, the part of not reproducible processing
Operation, for example, withholing or reducing the operation such as integral.
Step 305: executing corresponding to the event, the part operation of not reproducible processing, subsequently enter step 306.
And if the corresponding total count value of event identifier is less than default processing threshold value, illustrate that the event can be executed
The part operation of the not reproducible processing of part that is corresponding, being related to user's personal information, because being less than default processing threshold
Value, then execute when the corresponding part of the event operates and illustrate to repeat.
Step 306: according to the time of reception of the event identifier, the count value for the event that adds up.
Then, according still further to the time of reception of event identifier, the count value for the event that adds up, i.e., by corresponding point of event identifier
Section count value adds 1 in memory space corresponding with the time of reception.Specifically, step 306 may include step C1~step
C4:
Step C1: the time of reception of the event identifier is calculated divided by the quotient of the interval duration, as to storage time
Stamp, and, the remainder of the number of the time of reception and the memory space is calculated, is indexed as to memory space.
Firstly, when receiving event identifier, time of reception of event identifier is calculated divided by the quotient of interval duration,
It is stabbed as to storage time, i.e. ts=time/interval;And the number of memory space described in the calculating time of reception is remaining
Number, indexes, i.e. index=time/n as to memory space, wherein n is the number of memory space.It is still interior with 3 for server
For a memory space, it is assumed that obtained index=0, then it represents that need the event identifier in first memory space
Fragmentation count value in store1 adds 1, and so on, it is assumed that obtained index=2, then it represents that need the event identifier
Fragmentation count value in memory space store3 adds 1.
Step C2: judgement is described to index the original time stamp that corresponding memory space includes to memory space, with it is described to
Whether storage time stamp is identical, if it is, entering step C3;If it is not, then entering step C4.
Next, it is determined that original time stamp included by the corresponding memory space of index obtained in step C1, with step
Whether stabbing to storage time for being actually calculated in C1 be identical, if identical, C3 is entered step, by the memory space
The corresponding fragmentation count value of the event identifier saved in mapping table adds 1.For example, it is assumed that obtained index is zero, i.e., first
Memory space store1 is to memory space, then timestamp ts1 included by store1 is original time stamp, and step C1 is counted
If obtained timestamp is also equal to ts1, C3 is thened follow the steps, if being not equal to ts1, enters step C4.
Step C3: the corresponding fragmentation count value of the original time stamp that adds up.
The fragmentation count of the event identifier in mapping table in corresponding memory space will be indexed in this step to memory space
Value plus 1 is indicated in time reception to the corresponding event of the primary event identifier, so that subsequent needs judge whether to execute
When the part operation of corresponding, the not reproducible execution of event, added up each fragmentation count value to obtain total count value.
Step C4: emptying the corresponding memory space of the original time stamp, by the original time stamp replace with it is described to
Storage time stamp, and cumulative described corresponding, the event fragmentation count value is stabbed to storage time.
If it is not the same, then illustrate it is expired wait store index index, then will index be corresponding deposits wait store index
Store up space in mapping table reset, and by the time stamp setting of the memory space after clearing be step C1 in be calculated when
Between stab ts, the corresponding fragmentation count value of the event identifier is added 1, and fragmentation count value is saved to the memory space
In mapping table.
As it can be seen that in the embodiment of the present application, it, can be by storage in a period of time after each server receives event identifier
The corresponding fragmentation count value of the event identifier add up, to obtain the corresponding total quantity of the event identifier, then should
It amounts to quantity to be judged with default processing threshold value, to just no longer respond the event when being more than default processing threshold value
User's operation corresponding to the event of mark also avoids also execute event in the case where event data generates duplicate situation
The error in data that corresponding user's operation generates, simultaneously as avoiding the repetition of user being withholdd or reduced the phases such as integral
Operation is closed, also the user experience is improved.
Further, since server internal can also mark off multiple memory spaces, each memory space is stored
The fragmentation count value of the event identifier received in a certain section of event, so as to rapidly find each memory space storage
Fragmentation count value add up, obtain the total quantity of event, ensure that data processing speed and accuracy.
For the aforementioned method embodiment, for simple description, therefore, it is stated as a series of action combinations, but
Be those skilled in the art should understand that, the application is not limited by the described action sequence because according to the application,
Some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know that, in specification
Described embodiment belongs to preferred embodiment, necessary to related actions and modules not necessarily the application.
It is corresponding with method provided by a kind of data distributing method embodiment of above-mentioned the application, referring to fig. 4, the application
A kind of data distribution device embodiment is additionally provided, in the present embodiment, which is connected with multiple servers,
The data distribution device may include:
Event data receiving unit 401 is used for receiving event data, and the event data includes: user is in foreground system
Operation related data.
Generation unit 402, for generating the corresponding thing of the event data according to pre-set mark create-rule
Part mark.
Wherein, the generation unit 402 can specifically include:
First obtains subelement, for obtaining the type identification of user identifier, operation that the event data includes
And/or operated object mark;With combination subelement, for combining type identification and/or the operation of the user identifier, operation
Object identity obtains the event identifier.
Allocation unit 403, for the number according to the server, by the corresponding event data point of the event identifier
It is assigned in the multiple server.
Wherein, the allocation unit 403 can specifically include:
Hash operation subelement, for corresponding event identifier being carried out Hash operation, is obtained for each event data
To the cryptographic Hash of event identifier;Modulo operation subelement, for number of the cryptographic Hash to the server to be carried out modulus
Operation obtains index value;With transmission sub-unit, for the event identifier to be sent to the server of the index value mark
In.
It is corresponding with method provided by a kind of data processing method embodiment of above-mentioned the application, refering to what is shown in Fig. 5, this
Application additionally provides a kind of server example, and in the present embodiment, which is connected with data distribution device, the service
Device may include:
Event identifier receiving unit 501, the event identifier sent for receiving the data distribution device, and obtain institute
State the corresponding duration parameters of event identifier.
Statistic unit 502, for counting tale of the corresponding event of the event identifier in the duration parameters
Value;Multiple event identifiers and its corresponding count value are preserved in the server.
Wherein, the statistic unit 502 can specifically include:
Second obtains subelement, for obtaining the time of reception of the event identifier;First computation subunit, based on
The difference between the timestamp that the time of reception includes with the quotient for being spaced duration and the quotient and each memory space is calculated,
And using each difference as the query argument of each memory space;First judgment sub-unit, for judging each memory space
Query argument whether be less than the duration parameters;First cumulative subelement, for the result in first judgment sub-unit
In the case where being, obtains the fragmentation count value that the memory space includes and add up, obtain the total of the event identifier
Count value.
Judging unit 503, for judging whether the total count value is more than default processing threshold value.
First processing units 504, for the result of the judging unit be more than in the case where, ignore the event
The part operation of corresponding, not reproducible processing.
The second processing unit 505, for the result of the judging unit be no more than in the case where, execute the thing
Corresponding to part, the part operation of not reproducible processing.
Summing elements 506, for after triggering described the second processing unit, according to the time of reception of the event identifier,
The count value for the event that adds up.
It wherein, include multiple one-to-one memory spaces and timeslice in the server, when the interval of each timeslice
Length is identical, and the memory space includes timestamp and fragmentation count value, and the timestamp is for indicating since the timestamp
The corresponding event of event identifier received in the duration of interval, needs to be added in the memory space;The summing elements 506
It can specifically include:
Second computation subunit, the time of reception for calculating the event identifier are made divided by the quotient of the interval duration
To be stabbed to storage time, and, the remainder of the number of the time of reception and the memory space is calculated, as sky to be stored
Between index;With, the second judgment sub-unit, for judge it is described to memory space index that corresponding memory space includes it is original
Timestamp, with it is described to storage time stab it is whether identical;Second cumulative subelement, in second judgment sub-unit
It as a result is the corresponding fragmentation count value of the original time stamp that adds up in the case where being;Subelement is emptied, for described the
In the case that two judgment sub-units are no, the corresponding memory space of the original time stamp is emptied, the original time stamp is replaced
It is changed to described to storage time stamp;Third adds up subelement, for it is cumulative it is described stab to storage time it is corresponding, the event
Fragmentation count value.
The embodiment of the present application also provides a kind of computer-readable medium, which has and makes to calculate
Machine executes the program for data distribution, which may comprise steps of:
Receiving step, receiving event data, the event data includes: correlation of the user in the operation of foreground system
Data;
Generation step generates the corresponding event mark of the event data according to pre-set mark create-rule
Know;And
Allocation step distributes the corresponding event data of the event identifier to described more according to the number of server
In a server.
The embodiment of the present application also provides another computer-readable medium, which has and makes to count
The execution of calculation machine is used for, which may comprise steps of:
Receiving step receives event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic procedure counts total count value of the corresponding event of the event identifier in the duration parameters;
Judgment step judges whether the total count value is more than default processing threshold value;
First processing step, for ignoring corresponding to the event in the case where result that judgment step is more than
, the part operation of not reproducible processing;
Second processing step is right in the case where the result that judgment step is no more than, executing the event institute
The part operation of processing answer, not reproducible;And
Accumulation step, for according to the time of reception of the event identifier, adding up described after second processing step
The count value of event.
Fig. 6 is a kind of hardware structural diagram of computer equipment 600 in the embodiment of the present application.Computer equipment 600 can
For realizing data distribution, such as it can be client.I.e. computer equipment 600 can be used for executing provides in above-described embodiment
Method.In the present embodiment, computer equipment 600 includes: processor 601, memory 602, network interface 603 and total linear system
System 604.
The bus system 604, for each hardware component for calculating equipment 600 to be coupled.
The network interface 603, for realizing equipment 600 and at least one other communication calculated between equipment is calculated
Connection, can be used internet, wide area network, local network, the modes such as Metropolitan Area Network (MAN).
The memory 602, for storing program instruction and/or data.
The processor 601, for reading the instruction and/or data that store in memory 602, the following operation of execution:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;According to pre-
The mark create-rule being first arranged generates the corresponding event identifier of the event data;It, will be described according to the number of server
The corresponding event data of event identifier is distributed into the multiple server.
Wherein, described according to pre-set mark create-rule, generate the corresponding event mark of the event data
Know, may include: the user identifier for obtaining the event data and including, the type identification and/or operation object mark of operation
Know;The user identifier, the type identification of operation and/or operated object mark are combined, the event identifier is obtained.
Wherein, the number according to server, the corresponding event data of the event identifier is distributed to the multiple
In server, may include:
For each event data, corresponding event identifier is subjected to Hash operation, obtains the cryptographic Hash of event identifier;
Number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value;The event identifier is sent to institute
In the server for stating index value mark.
Fig. 7 is the hardware structural diagram of another computer equipment 700 in the embodiment of the present application.Computer equipment 700
It can be used for realizing that data are distributed, such as can be client.I.e. computer equipment 700 can be used for executing in above-described embodiment and mention
The method of confession.In the present embodiment, computer equipment 700 includes: processor 701, memory 702, network interface 703 and total
Linear system system 704.
The bus system 704, for each hardware component for calculating equipment 700 to be coupled.
The network interface 703, for realizing equipment 700 and at least one other communication calculated between equipment is calculated
Connection, can be used internet, wide area network, local network, the modes such as Metropolitan Area Network (MAN).
The memory 702, for storing program instruction and/or data.
The processor 701, for reading the instruction and/or data that store in memory 702, the following operation of execution:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;
Judge whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event
, the part operation of not reproducible processing;If be no more than, execute corresponding to the event, the portion of not reproducible processing
Divide operation, and according to the time of reception of the event identifier, the count value for the event that adds up.
It wherein, include multiple one-to-one memory spaces and timeslice in the server, when the interval of each timeslice
Length is identical, and the memory space includes timestamp and fragmentation count value, and the timestamp is for indicating since the timestamp
The corresponding event of event identifier received in the duration of interval, needs to be added in the memory space;It is described according to the event
The time of reception of mark, the count value for the event that adds up may include:
The time of reception of the event identifier is calculated divided by the quotient of the interval duration, is stabbed as to storage time, with
And the remainder of the number of the time of reception and the memory space is calculated, it is indexed as to memory space;Judgement it is described to
Memory space indexes the original time stamp that corresponding memory space includes, with it is described to storage time stab it is whether identical, if
It is the corresponding fragmentation count value of the original time stamp that then adds up;If it is not, then emptying, the original time stamp is corresponding to be deposited
Store up space, by the original time stamp replace with it is described stabbed to storage time, and it is cumulative it is described to storage time stab it is corresponding,
The fragmentation count value of the event.
Wherein, total count value of the corresponding event of the statistics event identifier in the duration parameters, can wrap
It includes:
Obtain the time of reception of the event identifier;The quotient of the time of reception and the interval duration are calculated, and should
The difference between timestamp that quotient and each memory space include, and using each difference as the query argument of each memory space;
Judge whether the query argument of each memory space is less than the duration parameters, if it is, obtaining the memory space and including
Fragmentation count value and add up, obtain the total count value of the event identifier.
In practical applications, the data processing method of the embodiment of the present application first can generate thing by the first computer equipment
Part mark, wherein the event identifier includes at least or based on one of following data: User ID, timestamp, operation code, operation
Object ID;Then the first computer equipment sends the event identifier to second computer equipment, wherein described second calculates
Machine equipment is based on the event identifier and carries out deduplication operation.
Wherein, referring to Fig. 8 a~Fig. 8 b, present invention also provides a kind of user interface interaction equipment, which can
To include:
First interactive module, for forwarding the first triggering command to processor after receiving first triggering command,
So as to processor receiving event data, the event data includes: related data of the user in the operation of foreground system.
After the control (refer to Fig. 8 a) that user clicks " receiving event data " by interactive interface, the of interactive device
One interactive module receives first triggering command, and then, the first interactive module forwards first triggering command to processor, to locate
Reason device can receive foreground system transmission, event event data, and the event data may include: user in foreground system
The related data of the operation of system.
Second interactive module forwards the first triggering command for receiving the second triggering command, and to processor, with
Just processor is received according to pre-set mark create-rule, generates the corresponding event identifier of the event data.
After user clicks the control (referring to Fig. 8 a) of " generate event identifier " by interactive interface, the of interactive device
Two interactive modules receive the second triggering command, then, the second interactive module to processor forward the second triggering command, so as to according to
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated.
Third interactive module forwards the third triggering command for receiving third triggering command, and to processor, with
Just number of the processor according to server, the corresponding event data of the event identifier is distributed into the multiple server
(referring to Fig. 8 b).
After user clicks the control (referring to Fig. 8 a) of " display allocation result " by interactive interface, the of interactive device
Three interactive modules receive third triggering command, then, third interactive module to processor forward third triggering command, so as to according to
According to the number of the server, the corresponding event data of the event identifier is distributed into the multiple server, and is received
The allocation result that processor is sent is shown.
If function described in the present embodiment method is realized in the form of SFU software functional unit and as independent product pin
It sells or in use, can store in a storage medium readable by a compute device.Based on this understanding, the application is implemented
The part of example the part that contributes to existing technology or the technical solution can be embodied in the form of software products,
The software product is stored in a storage medium, including some instructions are used so that a calculating equipment (can be personal meter
Calculation machine, server, mobile computing device or network equipment etc.) execute each embodiment the method for the application whole or
Part steps.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory),
Random access memory (RAM, Random Access Memory), magnetic or disk etc. be various to can store program code
Medium.
Wherein, referring to Fig. 9 a~Fig. 9 b, present invention also provides another user interface interaction equipment, the interactive devices
May include:
4th interactive module, for forwarding the 4th triggering command to processor after receiving the 4th triggering command,
So that processor receives event identifier, and obtain the corresponding duration parameters of the event identifier.
After user clicks the control (referring to Fig. 9 a) of " receive event identifier " by interactive interface, the of interactive device
Four interactive modules receive the 4th triggering command, and then, the 4th interactive module forwards the 4th triggering command to processor, to locate
Reason device can receive the event identifier that the data distribution device is sent, and obtain the corresponding duration parameters of the event identifier.
5th interactive module forwards the 5th triggering command for receiving the 5th triggering command, and to processor, with
Just it receives processor and counts total count value of the corresponding event of the event identifier in the duration parameters;The server
In preserve multiple event identifiers and its corresponding count value.
After user clicks the control (referring to Fig. 9 a) of " statistical counting value " by interactive interface, the 5th of interactive device the
Interactive module receives the 5th triggering command, and then, the 5th interactive module forwards the 5th triggering command to processor, so as to foundation
Pre-set mark create-rule generates the corresponding event identifier of the event data.
6th interactive module forwards the 6th triggering command for receiving the 6th triggering command, and to processor, with
Just processor judges whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event
, the part operation of not reproducible processing;If be no more than, execute corresponding to the event, the portion of not reproducible processing
Divide operation, and according to the time of reception of the event identifier, the count value (referring to Fig. 9 b) for the event that adds up.
After user clicks the control (referring to Fig. 9 a) of " display processing result " by interactive interface, the of interactive device
Six interactive modules receive the 6th triggering command, then, the 6th interactive module to processor forward the 6th triggering command, so as to according to
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server, and receives processing
The allocation result that device is sent is shown.
If function described in the present embodiment method is realized in the form of SFU software functional unit and as independent product pin
It sells or in use, can store in a storage medium readable by a compute device.Based on this understanding, the application is implemented
The part of example the part that contributes to existing technology or the technical solution can be embodied in the form of software products,
The software product is stored in a storage medium, including some instructions are used so that a calculating equipment (can be personal meter
Calculation machine, server, mobile computing device or network equipment etc.) execute each embodiment the method for the application whole or
Part steps.And storage medium above-mentioned include: USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory),
Random access memory (RAM, Random Access Memory), magnetic or disk etc. be various to can store program code
Medium.
It should be noted that all the embodiments in this specification are described in a progressive manner, each embodiment weight
Point explanation is the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other.
For device class embodiment, since it is basically similar to the method embodiment, so being described relatively simple, related place
Illustrate referring to the part of embodiment of the method.
Finally, it is to be noted that, herein, relational terms such as first and second and the like are used merely to
Distinguish one entity or operation from another entity or operation, without necessarily requiring or implying these entities or
There are any actual relationship or orders between operation.Moreover, the terms "include", "comprise" or its any other
Variant is intended to non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not
Only include those elements, but also including other elements that are not explicitly listed, or further includes for this process, side
Method, article or the intrinsic element of equipment.In the absence of more restrictions, being limited by sentence "including a ..."
Element, it is not excluded that there is also other identical elements in the process, method, article or apparatus that includes the element.
Data distributing method and device provided herein, data processing method and server have been carried out in detail above
Thin to introduce, specific examples are used herein to illustrate the principle and implementation manner of the present application, and above embodiments are said
It is bright to be merely used to help understand the present processes and its core concept;At the same time, for those skilled in the art, according to
According to the thought of the application, there will be changes in the specific implementation manner and application range, in conclusion in this specification
Hold the limitation that should not be construed as to the application.
Claims (15)
1. a kind of data processing method, which is characterized in that the data processing method includes:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;
Judge whether the total count value is more than default processing threshold value, if it does, then ignore corresponding to the event, can not
The part operation of reprocessing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to the event mark
The time of reception of knowledge, the count value for the event that adds up.
2. the method according to claim 1, wherein with multiple timeslices, one-to-one multiple storages are empty respectively
Between, the interval duration of each timeslice is identical, and the memory space includes timestamp and fragmentation count value, and the timestamp is used for table
Show the corresponding event of event identifier received in the interval duration since the timestamp, needs to be added to the memory space
In;The time of reception according to the event identifier, the count value for the event that adds up, comprising:
The time of reception of the event identifier is calculated divided by the quotient of the interval duration, is stabbed as to storage time, and, it calculates
The remainder of the number of the time of reception and the memory space, indexes as to memory space;
Judgement is described to index the original time stamp that corresponding memory space includes to memory space, is to storage time stamp with described
It is no identical, if it is, the corresponding fragmentation count value of the original time stamp that adds up;
If it is not, then emptying the corresponding memory space of the original time stamp, the original time stamp is replaced with described wait deposit
Timestamp is stored up, and cumulative described stabs corresponding, the event fragmentation count value to storage time.
3. according to the method described in claim 2, it is characterized in that, the corresponding event of the statistics event identifier is described
Total count value in duration parameters, comprising:
Obtain the time of reception of the event identifier;
It calculates between the timestamp that the time of reception includes with the quotient for being spaced duration and the quotient and each memory space
Difference, and using each difference as the query argument of each memory space;
Judge whether the query argument of each memory space is less than the duration parameters, if it is, obtaining the memory space
Including fragmentation count value and add up, obtain the total count value of the event identifier.
4. a kind of data distributing method, which is characterized in that this method comprises:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
5. according to the method described in claim 4, it is characterized in that, described according to pre-set mark create-rule, generation
The corresponding event identifier of the event data, comprising:
Obtain the type identification and/or operated object mark of user identifier, operation that the event data includes;
The user identifier, the type identification of operation and/or operated object mark are combined, the event identifier is obtained.
6. according to the method described in claim 4, it is characterized in that, the number according to server, by the event identifier
Corresponding event data is distributed into the multiple server, comprising:
For each event data, corresponding event identifier is subjected to Hash operation, obtains the cryptographic Hash of event identifier;
Number of the cryptographic Hash to the server is subjected to modulo operation, obtains index value;
The event identifier is sent in the server of the index value mark.
7. a kind of server, which is characterized in that the server includes:
Event identifier receiving unit for receiving event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic unit, for counting total count value of the corresponding event of the event identifier in the duration parameters;
Judging unit, for judging whether the total count value is more than default processing threshold value;
First processing units, for the result of the judging unit be more than in the case where, ignore corresponding to the event,
The part operation of not reproducible processing;
The second processing unit, for the result of the judging unit be no more than in the case where, execute corresponding to the event
, the part operation of not reproducible processing;
Summing elements, for according to the time of reception of the event identifier, adding up described after triggering described the second processing unit
The count value of event.
8. a kind of data distribution device, which is characterized in that the data distribution device includes:
Event data receiving unit is used for receiving event data, and the event data includes: operation of the user in foreground system
Related data;
Generation unit, for generating the corresponding event identifier of the event data according to pre-set mark create-rule;
Allocation unit distributes the corresponding event data of the event identifier to the multiple for the number according to server
In server.
9. a kind of computer-readable medium, which has the journey for executing computer for data distribution
Sequence, the processing the following steps are included:
Receiving step, receiving event data, the event data includes: related data of the user in the operation of foreground system;
Generation step generates the corresponding event identifier of the event data according to pre-set mark create-rule;
Allocation step distributes the corresponding event data of the event identifier to the multiple clothes according to the number of server
It is engaged in device.
10. a kind of computer-readable medium, which, which has, is used for computer execution, the processing include with
Lower step:
Receiving step receives event identifier, and obtains the corresponding duration parameters of the event identifier;
Statistic procedure counts total count value of the corresponding event of the event identifier in the duration parameters;
Judgment step judges whether the total count value is more than default processing threshold value;
First processing step, for ignoring corresponding to the event, no in the case where the result that judgment step is more than
The part operation of repeatable processing;
Second processing step, in the case where the result that judgment step is no more than, executing corresponding to the event,
The part operation of not reproducible processing;
Accumulation step, for after second processing step, according to the time of reception of the event identifier, to add up the event
Count value.
11. a kind of computer equipment, which is characterized in that the computer equipment includes: processor, memory, network interface and total
Linear system system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing the communication link between the computer equipment and at least one other computer equipment
It connects;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Receiving event data, the event data includes: related data of the user in the operation of foreground system;
According to pre-set mark create-rule, the corresponding event identifier of the event data is generated;
According to the number of server, the corresponding event data of the event identifier is distributed into the multiple server.
12. a kind of computer equipment, which is characterized in that the computer equipment includes: processor, memory, network interface and total
Linear system system;
The bus system, for each hardware component of the computer equipment to be coupled;
The network interface, for realizing the communication link between the computer equipment and at least one other computer equipment
It connects;
The memory, for storing program instruction and/or data;
The processor, for reading the instruction and/or data that store in the memory, the following operation of execution:
Event identifier is received, and obtains the corresponding duration parameters of the event identifier;
Count total count value of the corresponding event of the event identifier in the duration parameters;It is preserved in the server more
A event identifier and its corresponding count value;
Judge whether the total count value is more than default processing threshold value, if it does, then ignore corresponding to the event, can not
The part operation of reprocessing;
If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and according to the event mark
The time of reception of knowledge, the count value for the event that adds up.
13. a kind of user interface interaction equipment characterized by comprising
First interactive module, for the first triggering command being forwarded to processor, to locate after receiving first triggering command
Device receiving event data is managed, the event data includes: related data of the user in the operation of foreground system;
Second interactive module, for forwarding the first triggering command after receiving the second triggering command, and to processor, so as to
Processor generates the corresponding event identifier of the event data according to pre-set mark create-rule;
Third interactive module, for forwarding the third triggering command after receiving third triggering command, and to processor, so as to
Number of the processor according to server, the corresponding event data of the event identifier is distributed into the multiple server.
14. a kind of user interface interaction equipment characterized by comprising
4th interactive module, for the 4th triggering command being forwarded to processor, to locate after receiving the 4th triggering command
It manages device and receives event identifier, and obtain the corresponding duration parameters of the event identifier;
5th interactive module forwards the 5th triggering command for receiving the 5th triggering command, and to processor, to handle
Device counts total count value of the corresponding event of the event identifier in the duration parameters;It is preserved in the server multiple
Event identifier and its corresponding count value;
6th interactive module forwards the 6th triggering command for receiving the 6th triggering command, and to processor, to handle
Device judges whether the total count value is more than default processing threshold value, if it does, then ignoring corresponding to the event, can not weighing
The part operation handled again;If be no more than, execute corresponding to the event, the part operation of not reproducible processing, and
According to the time of reception of the event identifier, the count value for the event that adds up.
15. a kind of data processing method characterized by comprising
First, which calculates equipment, generates event identifier, wherein the event identifier includes at least or based on one of following data: user
ID, timestamp, operation code, operation object ID;
First, which calculates equipment, calculates the equipment transmission event identifier to second, wherein described second calculates equipment based on described
Event identifier carries out deduplication operation.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710612889.2A CN109302300B (en) | 2017-07-25 | 2017-07-25 | Data distribution and processing method, system and computer readable recording medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201710612889.2A CN109302300B (en) | 2017-07-25 | 2017-07-25 | Data distribution and processing method, system and computer readable recording medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109302300A true CN109302300A (en) | 2019-02-01 |
CN109302300B CN109302300B (en) | 2022-03-15 |
Family
ID=65167366
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201710612889.2A Active CN109302300B (en) | 2017-07-25 | 2017-07-25 | Data distribution and processing method, system and computer readable recording medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109302300B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110442466A (en) * | 2019-06-14 | 2019-11-12 | 平安科技(深圳)有限公司 | Prevent request repeated accesses method, apparatus, computer equipment and storage medium |
CN112131267A (en) * | 2020-08-14 | 2020-12-25 | 北京达佳互联信息技术有限公司 | Counting processing method, device, server and counting processing system |
CN112260902A (en) * | 2020-10-23 | 2021-01-22 | 深圳前海微众银行股份有限公司 | Network equipment monitoring method, device, equipment and storage medium |
CN112929379A (en) * | 2021-02-22 | 2021-06-08 | 深圳供电局有限公司 | Intelligent recorder remote operation and maintenance instruction defense method and system |
CN113139097A (en) * | 2020-01-19 | 2021-07-20 | 阿里巴巴集团控股有限公司 | Data writing method, data reading method, device and equipment |
Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20010067511A (en) * | 2001-02-02 | 2001-07-13 | 허경진 최병무 | A method for reducing a telecommunication fee using a secret number and an intrinsic number of goods |
CN1762141A (en) * | 2003-11-10 | 2006-04-19 | Eath株式会社 | Compilation system |
US20090070336A1 (en) * | 2007-09-07 | 2009-03-12 | Sap Ag | Method and system for managing transmitted requests |
CN101009586B (en) * | 2006-01-27 | 2010-09-08 | 华为技术有限公司 | Processing method for alarm filtering condition in the alarming system |
JP4677686B2 (en) * | 2001-06-15 | 2011-04-27 | 株式会社ニコン | Interface device and recording medium recording program |
CN102541918A (en) * | 2010-12-30 | 2012-07-04 | 阿里巴巴集团控股有限公司 | Method and equipment for identifying repeated information |
CN102663590A (en) * | 2012-02-29 | 2012-09-12 | 向亚峰 | System and method of commodity anti-counterfeiting authentication based on restriction of authentication frequency |
CN103164511A (en) * | 2013-02-21 | 2013-06-19 | 烽火通信科技股份有限公司 | Method for storage event log to automatically filter repeated jitter data |
CN103297468A (en) * | 2012-02-29 | 2013-09-11 | 华为技术有限公司 | Operation method for group resources and group server |
FR2993685A1 (en) * | 2012-07-17 | 2014-01-24 | France Telecom | Data processing machine i.e. networked server, monitoring method, involves repeating collection of history of events, choosing type of event, and assignment of calculation value, and obtaining variation of value with regard to time |
CN103678471A (en) * | 2012-09-24 | 2014-03-26 | 国际商业机器公司 | Method and device for partitioning search space for distributed crawling |
CN104077701A (en) * | 2014-06-09 | 2014-10-01 | 中国建设银行股份有限公司 | Task processing method and device used for e-business platform |
CN104508700A (en) * | 2012-07-27 | 2015-04-08 | 谷歌公司 | Determining a correlation between presentation of a content item and a transaction by a user at a point of sale terminal |
CN104618432A (en) * | 2014-12-30 | 2015-05-13 | 北京红马传媒文化发展有限公司 | Event sending and receiving handling method and system |
CN104769555A (en) * | 2012-06-18 | 2015-07-08 | 艾克特菲欧有限公司 | Enhanced data management virtualization system |
US20160055200A1 (en) * | 2008-08-18 | 2016-02-25 | Guidance Software, Inc. | Scalable deduplication system and method |
CN105468699A (en) * | 2015-11-18 | 2016-04-06 | 珠海多玩信息技术有限公司 | Duplicate removal data statistics method and equipment |
CN106296246A (en) * | 2015-05-26 | 2017-01-04 | 腾讯科技(深圳)有限公司 | Method for processing business and device |
CN106899666A (en) * | 2017-02-21 | 2017-06-27 | 阿里巴巴集团控股有限公司 | A kind of data processing method and device for service identification |
-
2017
- 2017-07-25 CN CN201710612889.2A patent/CN109302300B/en active Active
Patent Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20010067511A (en) * | 2001-02-02 | 2001-07-13 | 허경진 최병무 | A method for reducing a telecommunication fee using a secret number and an intrinsic number of goods |
JP4677686B2 (en) * | 2001-06-15 | 2011-04-27 | 株式会社ニコン | Interface device and recording medium recording program |
CN1762141A (en) * | 2003-11-10 | 2006-04-19 | Eath株式会社 | Compilation system |
CN101009586B (en) * | 2006-01-27 | 2010-09-08 | 华为技术有限公司 | Processing method for alarm filtering condition in the alarming system |
US20090070336A1 (en) * | 2007-09-07 | 2009-03-12 | Sap Ag | Method and system for managing transmitted requests |
US20160055200A1 (en) * | 2008-08-18 | 2016-02-25 | Guidance Software, Inc. | Scalable deduplication system and method |
CN102541918A (en) * | 2010-12-30 | 2012-07-04 | 阿里巴巴集团控股有限公司 | Method and equipment for identifying repeated information |
CN103297468A (en) * | 2012-02-29 | 2013-09-11 | 华为技术有限公司 | Operation method for group resources and group server |
CN102663590A (en) * | 2012-02-29 | 2012-09-12 | 向亚峰 | System and method of commodity anti-counterfeiting authentication based on restriction of authentication frequency |
CN104769555A (en) * | 2012-06-18 | 2015-07-08 | 艾克特菲欧有限公司 | Enhanced data management virtualization system |
FR2993685A1 (en) * | 2012-07-17 | 2014-01-24 | France Telecom | Data processing machine i.e. networked server, monitoring method, involves repeating collection of history of events, choosing type of event, and assignment of calculation value, and obtaining variation of value with regard to time |
CN104508700A (en) * | 2012-07-27 | 2015-04-08 | 谷歌公司 | Determining a correlation between presentation of a content item and a transaction by a user at a point of sale terminal |
CN103678471A (en) * | 2012-09-24 | 2014-03-26 | 国际商业机器公司 | Method and device for partitioning search space for distributed crawling |
CN103164511A (en) * | 2013-02-21 | 2013-06-19 | 烽火通信科技股份有限公司 | Method for storage event log to automatically filter repeated jitter data |
CN104077701A (en) * | 2014-06-09 | 2014-10-01 | 中国建设银行股份有限公司 | Task processing method and device used for e-business platform |
CN104618432A (en) * | 2014-12-30 | 2015-05-13 | 北京红马传媒文化发展有限公司 | Event sending and receiving handling method and system |
CN106296246A (en) * | 2015-05-26 | 2017-01-04 | 腾讯科技(深圳)有限公司 | Method for processing business and device |
CN105468699A (en) * | 2015-11-18 | 2016-04-06 | 珠海多玩信息技术有限公司 | Duplicate removal data statistics method and equipment |
CN106899666A (en) * | 2017-02-21 | 2017-06-27 | 阿里巴巴集团控股有限公司 | A kind of data processing method and device for service identification |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110442466A (en) * | 2019-06-14 | 2019-11-12 | 平安科技(深圳)有限公司 | Prevent request repeated accesses method, apparatus, computer equipment and storage medium |
CN110442466B (en) * | 2019-06-14 | 2024-05-07 | 平安科技(深圳)有限公司 | Method, device, computer equipment and storage medium for preventing repeated access request |
CN113139097A (en) * | 2020-01-19 | 2021-07-20 | 阿里巴巴集团控股有限公司 | Data writing method, data reading method, device and equipment |
CN112131267A (en) * | 2020-08-14 | 2020-12-25 | 北京达佳互联信息技术有限公司 | Counting processing method, device, server and counting processing system |
CN112131267B (en) * | 2020-08-14 | 2023-10-03 | 北京达佳互联信息技术有限公司 | Count processing method, device, server and count processing system |
CN112260902A (en) * | 2020-10-23 | 2021-01-22 | 深圳前海微众银行股份有限公司 | Network equipment monitoring method, device, equipment and storage medium |
CN112929379A (en) * | 2021-02-22 | 2021-06-08 | 深圳供电局有限公司 | Intelligent recorder remote operation and maintenance instruction defense method and system |
Also Published As
Publication number | Publication date |
---|---|
CN109302300B (en) | 2022-03-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109302300A (en) | Data distributing method and device, data processing method and server | |
Luu et al. | {SmartPool}: Practical decentralized pooled mining | |
CN105100162B (en) | Virtual objects sending method and device, method of reseptance and device, system | |
WO2016131365A1 (en) | Information processing method, client, server and computer readable storage medium | |
KR102282763B1 (en) | Service implementation method and device | |
CN108615151B (en) | Data processing method, block chain server and node equipment | |
CN106470133A (en) | System pressure method of testing and device | |
CN106411777A (en) | Method and system for processing high concurrent data | |
US10587650B2 (en) | Communications security | |
CN107360006A (en) | A kind of resource billing method and device | |
EP3879477A1 (en) | Electronic certificate-based interaction method and apparatus, and electronic device | |
CN106649637B (en) | Method and system for processing red packet | |
JP2018507475A (en) | Service implementation method, payment method, and apparatus | |
CN110377854A (en) | User access activity information monitoring method and device, computer equipment | |
CN110365712A (en) | A kind of defence method and system of distributed denial of service attack | |
CN107786628A (en) | Business numbering distribution method, device, computer equipment and storage medium | |
CN111367621A (en) | Intelligent contract timing processing method, block chain node and storage medium | |
CN106302111A (en) | A kind of information processing method, terminal and server | |
CN104184603B (en) | A kind of User Status statistical method, apparatus and system | |
CN116846980B (en) | Integral issuing and using method, system and equipment | |
CN111741067B (en) | Block chain-based tableware circulation information management method and device and block chain system | |
CN104462116B (en) | Data selection method and device | |
CN108959047A (en) | A kind of method for testing pressure and device based on business scenario | |
CN105184559A (en) | System and method for payment | |
CN105100072B (en) | A kind of network node monitoring method and device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |