CN108449231A - A kind of filter method of transaction data, device and realization device - Google Patents
A kind of filter method of transaction data, device and realization device Download PDFInfo
- Publication number
- CN108449231A CN108449231A CN201810216684.7A CN201810216684A CN108449231A CN 108449231 A CN108449231 A CN 108449231A CN 201810216684 A CN201810216684 A CN 201810216684A CN 108449231 A CN108449231 A CN 108449231A
- Authority
- CN
- China
- Prior art keywords
- transaction
- feature
- transaction data
- filtering rule
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/028—Capturing of monitoring data by filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/026—Capturing of monitoring data using flow identification
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of filter method of transaction data, device and realization devices；Wherein, this method includes：By in a plurality of filtering rule stream, the identical transaction feature of characteristic value merges；Transaction feature after merging shares same matching identification；Transaction data is matched with a plurality of filtering rule stream one by one；When being matched with first filtering rule stream, wherein each transaction feature is matched, corresponding matching identification is added；When being matched with other filtering rule streams, the transaction feature there is no matching identification is matched, adds corresponding matching identification；If transaction data and the equal successful match of each transaction feature in current filter regular flow, the identity of current filter regular flow is carried in transaction data.The present invention reduces the matching times between transaction feature and rule, the computation complexity being filtered to transaction data is reduced, reduces the calculating time of CPU, improves the treatment effeciency and performance of transaction data filtering.
Description
Technical field
The present invention relates to field of computer technology, more particularly, to a kind of filter method of transaction data, device and realization
Device.
Background technology
By being decoded to the transaction data captured on network, the transaction feature of the transaction, such as IP can be extracted
(Internet Protocol, procotol) address, port, type of transaction, transaction return code etc.；The spy of these transaction features
Value indicative is the attribute of transaction itself, because all identical two transactions of all characteristic values may be not present, but can be had between merchandising
There is the identical characteristic value in part, as source IP address is identical, type of transaction is mutually same.It can mistake by one or more transaction features
The transaction for filtering out one or more, the transaction filtered out can be used for calculating the statistical natures such as trading volume, success rate.
Include one or more in the transaction flow in general, the rule of the characteristic value of one group of transaction feature is defined as transaction flow
The characteristic value of a transaction data；Since data statistics demand is larger, usually transaction data is carried out using a plurality of transaction flow simultaneously
Filtering；In filter process, the transaction feature in every transaction data needs to carry out with the character pair value in every transaction flow
It matches one by one, therefore, to complete above-mentioned filtration duty, CPU needs to repeat a large amount of matching operation, and with transaction data amount
Or the increase of transaction flow quantity, the operand of CPU are even more to be multiplied, and cause the process performance of filter data relatively low.
Invention content
In view of this, the purpose of the present invention is to provide a kind of filter method of transaction data, device and realization device, with
The computation complexity being filtered to transaction data is reduced, the calculating time of CPU is reduced, improves the processing effect of transaction data filtering
Rate and process performance.
In a first aspect, an embodiment of the present invention provides a kind of filter method of transaction data, this method includes：It obtains a plurality of
Filtering rule stream；Every filtering rule stream includes one or more transaction features and the corresponding characteristic value of transaction feature；It will be a plurality of
In filtering rule stream, the identical transaction feature of characteristic value merges；Transaction feature after merging shares same matching identification；It obtains
Take transaction data to be matched；Transaction data is matched with a plurality of filtering rule stream one by one；Transaction data with first
When filtering rule stream is matched, each transaction feature in first filtering rule stream is matched, adds corresponding
With mark；Transaction data with the filtering rule stream in addition to first filtering rule stream when being matched, to filtering rule stream
In matched there is no the transaction feature of matching identification, add corresponding matching identification；If transaction data and current filter
The equal successful match of each transaction feature in regular flow carries the identity of current filter rule in transaction data.
With reference to first aspect, an embodiment of the present invention provides the first possible embodiments of first aspect, wherein on
The step of being matched transaction data with a plurality of filtering rule stream one by one is stated, including：Processing is decoded to transaction data, is obtained
To the transaction feature and the corresponding characteristic value of transaction feature in transaction data；Transaction feature includes at least the IP that transaction data occurs
It is a variety of in address, port address, type of transaction, transaction return code；By transaction feature and the corresponding characteristic value of transaction feature by
One is matched with a plurality of filtering rule stream.
The possible embodiment of with reference to first aspect the first, an embodiment of the present invention provides second of first aspect
Possible embodiment, wherein above-mentioned to advise transaction feature and the corresponding characteristic value of the transaction feature with a plurality of filtering one by one
The step of then stream is matched, including：Transaction feature and the corresponding characteristic value of transaction feature are carried out with current filter regular flow
When matching, the transaction feature and characteristic value in the current filter regular flow are obtained；According to the transaction in current filter regular flow
Feature and characteristic value compare transaction feature and characteristic value in transaction data；If same transaction feature, transaction data corresponds to
Characteristic value characteristic value corresponding with current filter regular flow it is identical, then transaction feature successful match.
Second of possible embodiment with reference to first aspect, an embodiment of the present invention provides the third of first aspect
Possible embodiment, wherein when transaction feature successful match, matching identification is " 1 "；When it fails to match for transaction feature,
Matching identification is " 0 ".
With reference to first aspect, an embodiment of the present invention provides the 4th kind of possible embodiments of first aspect, wherein on
The method of stating further includes：Bit is allocated in advance for each transaction feature in a plurality of filtering rule stream；In the matching process, will add
The matching identification added is inserted into corresponding bit.
The 4th kind of possible embodiment with reference to first aspect, an embodiment of the present invention provides the 5th kind of first aspect
Possible embodiment, wherein above-mentioned by step that in a plurality of filtering rule stream, the identical transaction feature of characteristic value merges
Suddenly, including：By in a plurality of filtering rule stream, the bit of the identical transaction feature distribution of characteristic value merges.
With reference to first aspect, an embodiment of the present invention provides the 6th kind of possible embodiments of first aspect, wherein on
If transaction data and the equal successful match of each transaction feature in current filter regular flow are stated, by the body of current filter rule
Part mark carries the step in transaction data, including：Field is added in transaction data；By the identity of current filter regular flow
In mark filling field.
Second aspect, the embodiment of the present invention also provide a kind of filter device of transaction data, including：Filtering rule stream obtains
Module, for obtaining a plurality of filtering rule stream；Every filtering rule stream includes one or more transaction features and transaction feature pair
The characteristic value answered；Merge transaction feature module, for by a plurality of filtering rule stream, the identical transaction feature of characteristic value to be closed
And；Transaction feature after merging shares same matching identification；Transaction data acquisition module, for obtaining number of deals to be matched
According to；Matching module, for matching transaction data with a plurality of filtering rule stream one by one；Transaction data is filtered with first
When regular flow is matched, each transaction feature in first filtering rule stream is matched, adds corresponding matching mark
Know；Transaction data when being matched with the filtering rule stream in addition to first filtering rule stream, in filtering rule stream not
There are the transaction features of matching identification to be matched, and corresponding matching identification is added；Mark carries module, if being used for above-mentioned friendship
Easy data and the equal successful match of each transaction feature in current filter regular flow, the identity of current filter rule is taken
Band is in above-mentioned transaction data.
In conjunction with second aspect, an embodiment of the present invention provides the first possible embodiments of second aspect, wherein on
Matching module is stated to be additionally operable to：Processing is decoded to transaction data, obtains the transaction feature in transaction data and transaction feature pair
The characteristic value answered；Transaction feature includes at least IP address, port address, type of transaction, the transaction return code that transaction data occurs
In it is a variety of；Transaction feature and the corresponding characteristic value of transaction feature are matched with a plurality of filtering rule stream one by one.
The third aspect, the embodiment of the present invention also provide a kind of filtering realization device of transaction data, including memory and place
Manage device, wherein memory is held for storing one or more computer instruction, one or more computer instruction by processor
Row, with above-mentioned transaction data filter method.
The embodiment of the present invention brings following advantageous effect：
An embodiment of the present invention provides a kind of filter method of transaction data, device and realization devices, and a plurality of filtering is advised
The identical transaction feature of characteristic value merges in then flowing；Transaction feature after merging shares same matching identification；It again will transaction
Data are matched with a plurality of filtering rule stream one by one；The transaction data is right when being matched with first filtering rule stream
Each transaction feature in first filtering rule stream is matched, and according to matching result, adds corresponding matching identification；The friendship
Easy data are not present when being matched with the filtering rule stream in addition to first filtering rule stream with the filtering rule stream
The transaction feature of matching identification is matched, and adds corresponding matching identification；If transaction data and current filter regular flow
In the equal successful match of each transaction feature, then by the identity of current filter rule carry in transaction data；The party
Formula reduces the matching times between transaction feature and rule, reduces the computation complexity being filtered to transaction data, subtracts
The calculating time for having lacked CPU improves the treatment effeciency and performance of transaction data filtering.
Other features and advantages of the present invention will illustrate in the following description, alternatively, Partial Feature and advantage can be with
Deduce from specification or unambiguously determine, or by implement the present invention above-mentioned technology it can be learnt that.
To enable the above objects, features and advantages of the present invention to be clearer and more comprehensible, better embodiment cited below particularly, and match
Appended attached drawing is closed, is described in detail below.
Description of the drawings
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art
Embodiment or attached drawing needed to be used in the description of the prior art are briefly described, it should be apparent that, in being described below
Attached drawing is some embodiments of the present invention, for those of ordinary skill in the art, before not making the creative labor
It puts, other drawings may also be obtained based on these drawings.
Fig. 1 is a kind of flow chart of the filter method of transaction data provided in an embodiment of the present invention；
Fig. 2 is the flow chart of the filter method of another transaction data provided in an embodiment of the present invention；
Fig. 3 is a kind of structural schematic diagram of the filter device of transaction data provided in an embodiment of the present invention；
Fig. 4 is a kind of structural schematic diagram of the filtering realization device of transaction data provided in an embodiment of the present invention.
Specific implementation mode
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached drawing to the present invention
Technical solution be clearly and completely described, it is clear that described embodiments are some of the embodiments of the present invention, rather than
Whole embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art are not making creative work premise
Lower obtained every other embodiment, shall fall within the protection scope of the present invention.
The rule of the characteristic value of one group of transaction feature is defined as transaction flow.When in stream rule each characteristic value with it is true
When each attribute of transaction data exactly matches, then say that this transaction data belongs to current transaction flow.Each transaction flow has uniquely
Stream ID (identity, identity), the rule by the way that stream is arranged filters transaction data.When transaction data and stream rule
Timing is then stamped stream ID label to this transaction and is marked.According to it is defined above it is found that a stream possibly filter out out one or
More transactions, a transaction may also belong to one or more transaction flows.
Include multiple matching rules in one transaction flow, there may be identical matching rules in cocurrent flow.It is handed in filtering
When easy data, by each transaction feature rule in each transaction flow with the actual value of some attribute field in every transaction into
Row matches one by one, and matching every time will obtain the transaction data on backstage, and time complexity is O (n2).In view of actual transaction
Data volume is very huge, and trading rules are deeply increasingly refined also with transaction monitoring demand, are gradually increased.Work as data volume
When more than a certain range, calculation amount, which crosses senior general, causes process performance prominent question occur.
2000 transaction flows as defined in system, a transaction flow include 5 filtering rules, then need to carry out altogether
2000*5=10000 matching.Therefore, a relatively simple filtration duty just needs carrying out up to ten thousand transaction ratios from the background
To inquiry；If the transaction flow rule of filtration duty is complex, excessive operand can lead to a transaction being skillfully constructed
Stream definition, and be difficult to be generalized under larger data amount scene based on the transaction monitoring tool that this definition design is realized and apply.
And according to the experience in practical application, between many transaction flows, there is a large amount of identical matching rules in fact, such as
The identical ports IP or identical type of transaction etc..With original matching algorithm, need repeatedly to obtain from transaction data same
The data of a field waste the calculating time of CPU, reduce working efficiency.Based on this, a kind of friendship provided in an embodiment of the present invention
Filter method, device and the realization device of easy data can be applied to processing for the processing and data statistics to transaction data
Journey.
For ease of understanding the present embodiment, the filtering first to a kind of transaction data disclosed in the embodiment of the present invention
Method describes in detail.
A kind of flow chart of the filter method of transaction data shown in Figure 1, this method include：
Step S100 obtains a plurality of filtering rule stream；Every filtering rule stream includes one or more transaction features and friendship
The corresponding characteristic value of easy feature；
The transaction data each merchandised has respective transaction feature, such as IP (Internet Protocol, network association
View) address, port, type of transaction, transaction return code etc..According to filtering rule stream, one group of relationship trading can be filtered,
To realize the statistics to certain parameter transactions.For example, filtering rule stream A includes IP address (character pair value is X), transaction class
Type (character pair value is Y) and type of transaction (character pair value is Z) totally three transaction features；It is set according to the filtering rule stream pair
The transaction data determined in range is filtered, can obtain that IP address is X, type of transaction is Y and type of transaction is Z one or
A plurality of transaction data, these transaction data are satisfied by above-mentioned filtering rule stream, and setting can be counted according to these transaction data
Parameter transaction.
In actual implementation, to the transaction data of same range, the statistics of multiple parameters may be carried out, therefore be directed to
The transaction data can carry out filtration treatment according to a plurality of filtering rule stream, which is equivalent to above-mentioned transaction flow.
Step S102, by a plurality of filtering rule stream, the identical transaction feature of characteristic value merges；Transaction after merging
Feature shares same matching identification；
It is the transaction feature addition matching of the particular characteristic value in some filtering rule stream when being filtered to transaction data
After mark, the transaction feature of the same characteristic features value in other filtering rule streams can be total to without being matched again with transaction data
Enjoy the matching identification.For example, the transaction feature in a plurality of filtering rule stream, can be arranged in the form of bitmap or list, when working as
Preceding transaction data is matched with the transaction feature m in a certain filtering rule stream when finishing, then adds matching identification；Through the above way
After merging, if filtering rule stream A all has transaction feature m with filtering rule stream B and characteristic value is identical, same transaction
After data are matched with the transaction feature m in filtering rule stream A, due to matching identification and addition, then no longer advised with filtering
The transaction feature m then flowed in B is matched.
By in a plurality of filtering rule stream, the identical transaction feature of characteristic value merges, and avoids in filter process to identical
The transaction feature of characteristic value carries out repeated matching；Also, the transaction feature after merging shares same matching identification, saves matching
The storage space of mark.
Step S104 obtains transaction data to be matched；
Step S106 matches transaction data with a plurality of filtering rule stream one by one；Transaction data with first mistake
When filter regular flow is matched, each transaction feature in first filtering rule stream is matched, corresponding matching is added
Mark；Transaction data with the filtering rule stream in addition to first filtering rule stream when being matched, in filtering rule stream
There is no the transaction features of matching identification to be matched, and corresponding matching identification is added；
Step S108 will if transaction data and the equal successful match of each transaction feature in current filter regular flow
The identity of current filter rule carries in transaction data.
An embodiment of the present invention provides a kind of filter methods of transaction data, and characteristic value in a plurality of filtering rule stream is identical
Transaction feature merge；Transaction feature after merging shares same matching identification；Again by transaction data one by one with a plurality of mistake
Filter regular flow is matched；The transaction data with first filtering rule stream when being matched, to first filtering rule stream
In each transaction feature matched, according to matching result, add corresponding matching identification；The transaction data with except first
It is special with transaction that matching identification is not present in the filtering rule stream when filtering rule stream except filtering rule stream is matched
Sign is matched, and adds corresponding matching identification；If transaction data and each transaction in current filter regular flow are special
Equal successful match is levied, then is carried the identity of current filter rule in transaction data；The method reduce transaction features
Matching times between rule reduce the computation complexity being filtered to transaction data, when reducing the calculating of CPU
Between, improve the treatment effeciency and performance of transaction data filtering.
The flow chart of the filter method of another transaction data shown in Figure 2；This method is referred to as transaction
The process that data label, when strictly all rules exactly match it is upper when, to this transaction increase a digit group type field streams
(data flow), respective value are the identity ID of each matched filtering rule stream.
This method specifically comprises the following steps：
Step S200 obtains a plurality of filtering rule stream, is allocated in advance for each transaction feature in a plurality of filtering rule stream
Bit；
In the present embodiment, transaction data is handled by bitmap technology.Bitmap is a kind of data structure, that is, utilizes two
The position of system describes certain state, in the case of being suitable for known to existing state, handles to larger data.
Above-mentioned filtering rule stream is also referred to as transaction flow；Include one or more transaction features in each transaction flow.With
For two transaction flows of stream1 and stream2,1 and stream 2 are hereinafter referred to as flowed.
Stream 1 has following rule：IP=A, Port=B, Trantype=C, Retcode=D；
Stream 2 has following rule：IP=A, Port=B, Trantype=E, Retcode=F；
Wherein IP is internet protocol address, and Port is port, and Trantype is type of transaction, and Retcode is that transaction returns
Code, above-mentioned rule can be as shown in Table 1：
Table 1
| Stream ID | IP | Port | Transtype | Retcode |
| stream1 | A | B | C | D |
| stream2 | A | B | E | F |
For behind each transaction feature distributing bit position of stream 1 and stream 2, stream 1 and 2 corresponding bitmaps of stream are as shown in table 2：
Table 2
| Stream ID | IP | Port | Transtype | Retcode |
| stream1 | 0 | 0 | 0 | 0 |
| stream2 | 0 | 0 | 0 | 0 |
Step S202, by a plurality of filtering rule stream, the bit of the identical transaction feature distribution of characteristic value merges；
Above-mentioned steps 202, also referred to as by phase cocurrent flow compatible rule merging present in different transaction flows.By above-mentioned stream 1 and stream 2
The amalgamation result that the identical transaction feature of characteristic value merges is as shown in table 3：
Table 3
After corresponding bit merges, the bitmap of stream 1 and stream 2 is as shown in table 4：
Table 4
Therefore, when being matched with stream 1 and stream 2 to the characteristic value of a transaction, to IP and Port the two attributes
Matching need to only carry out once, and due to having been marked on corresponding position when matching for the second time, so there is no need to match again again.
Step S204 is decoded processing to transaction data, obtains the transaction feature in transaction data and transaction feature pair
The characteristic value answered；Transaction feature includes at least IP address, port address, type of transaction, the transaction return code that transaction data occurs
In it is a variety of；
For example, after by being decoded to transaction data m, obtained transaction feature is IP=A, Port=B, Trantype
=C, Retcode=D, referring to table 5：
Table 5
| IP | Port | Transtype | Retcode |
| A | B | C | D |
Step S206, transaction feature and the corresponding characteristic value of transaction feature is carried out with a plurality of filtering rule stream one by one
Match；
Above-mentioned steps 206 can specifically be realized by following manner：
(1) when being matched transaction feature and the corresponding characteristic value of transaction feature with current filter regular flow, institute is obtained
State the transaction feature and characteristic value in current filter regular flow；
(2) according to the transaction feature and characteristic value in current filter regular flow, compare transaction feature in transaction data and
Characteristic value；If same transaction feature, the corresponding characteristic value of transaction data characteristic value phase corresponding with current filter regular flow
Together, then transaction feature successful match.
Specifically, in the matching process, the matching identification of addition is inserted into corresponding bit, when transaction feature
When with success, matching identification is " 1 "；When it fails to match for transaction feature, matching identification is " 0 ".
The process that matching identification is inserted to corresponding bit is also referred to as bitmap label.By above-mentioned transaction with stream 1 and stream 2 into
After row matching, obtained bitmap is as shown in table 6：
Table 6
Step 208, if transaction data and the equal successful match of each transaction feature in current filter regular flow, are being handed over
Field is added in easy data, it will be in the identity filling field of current filter regular flow.
Above-mentioned steps 208 are specifically realized by following manner：
(1) each bit of filtering rule stream is compared；
The step is also referred to as result comparison, i.e., the result that transaction flow is corresponded to bitmap is compared, if the position of stream rule
Each in figure is 1, then it represents that the All Activity feature currently merchandised is satisfied by the filter condition of stream rule, that is, judges to hand over
Easily belong to current stream.It is learnt by matched bitmap according to above-mentioned, above-mentioned IP=A, Port=B, Trantype=C, Retcode
The transaction of=D belongs to stream 1, but is not belonging to stream 2.
(2) if each bit of above-mentioned filtering rule stream is 1, to add field in transaction data, will work as
In the identity filling field of preceding filtering rule stream.
The step is also referred to as flow label, i.e., by the flow label of successful match in transaction.Specific practice is increased to the transaction
Add a streams field, is added the value of stream ID as the value of the field.Streams is array type, multiple when having
When the stream rule of transaction flow matches transaction feature simultaneously, the label of all streams can be stamped to currently merchandising.It is handed over will pass through
Flexible configuration of transaction feature is easily flowed, the transaction of concern is filtered out, the transaction for meeting condition is monitored later or statistical
Analysis.
Therefore, the label of stream 1 is added in the field of above-mentioned and 1 successful match of stream transaction, as shown in table 7：
Table 7
| IP | Port | Transtype | Retcode | streams |
| A | B | C | D | stream1 |
In the example mentioned in above step, a transaction matches two stream rules, needs to compare 6 times altogether.If
Matching 8 times are then needed using original match scheme, are reduced 2 times, can be reduced 20 times if it is 10 transactions, 100 transactions are then
It can reduce 200 times, and so on.One simple filtering scene is the process performance for improving 25%.
When the algorithm copes with big data quantity, multi-field, there are when the case where identical filtering rule in more transaction flows, such as one
The extreme scene of a comparison：If a total of 2000 transaction flows, each stream has 5 rules and all streams rule is the same；It does not close
And when the transaction feature of same characteristic features value, the bitmap of each transaction flow is as shown in table 8：
Table 8
| Stream ID | IP | Port | Transtype | Retcode | Channel |
| stream1 | 0 | 0 | 0 | 0 | 0 |
| stream2 | 0 | 0 | 0 | 0 | 0 |
| … | … | … | … | … | … |
| stream2000 | 0 | 0 | 0 | 0 | 0 |
After the transaction feature of same characteristic features value is merged, the bitmap of all streams is as shown in table 9.Through above-mentioned it is found that according to original
Beginning method matches a transaction and needs to inquire comparison 10000 times.And in the way of bitmap, it is only necessary to compare 5 times.
Table 9
In this case the performance of deals match will be substantially improved.
This method is by bitmap labelling technique, using every filtering rule of transaction flow as one of bitmap.It is being flowed
When rule match, a transaction is often matched.It is 1 that the rule, which is corresponded in the position of bitmap,.If found when matching next time
The corresponding position of stream rule is 1, indicates successful match, then no longer carries out fields match.Same rule has been saved repeatedly to inquire
With the number for comparing All Activity, this method is to there is the case where identical filtering rule in a large amount of transaction flows, performance boost is especially
Obviously.
Certainly for the entirely different special circumstances of each stream rule, this method can not effective improving performance, but compare
It will not additionally increase processing load in original method.
True transaction monitoring scene both will not idealization, will not extremely make an exception.In conjunction with the demand really monitored,
It is verified by theory analysis and practical application, the calculation processing resource of 70%-80% can be saved.This means that if originally may be used
With the filtering rule of 2000 stream of processing, nearly 10000 stream rules can be then handled now.If originally already close to processing
Performance bottleneck can ensure system normal operation using the match party rule based on bitmap, while also allow to continue to extend completely
Configuration, successfully manages the reality that type of transaction is increasing, and monitoring demand increasingly refines.Compared to past original method,
Promotion at double is reached to performance, the deals match of system entirety has obtained no small improvement and optimization.
In addition, transaction data is filtered by the above method, it is not heavy in cocurrent flow rule in reply large-scale data amount
When the more situation of multiple rule, reduces processing time, and reduce resource consumption side, ensure that transaction flow in practical applications
Autgmentability, to provide reliable solution as the transaction flow rule match on transaction monitoring basis.
Corresponding to above method embodiment, a kind of structural schematic diagram of the filter device of transaction data shown in Figure 3,
The device includes：Filtering rule stream acquisition module 300, for obtaining a plurality of filtering rule stream；Every filtering rule stream includes one
A or multiple transaction features and the corresponding characteristic value of transaction feature；Merge transaction feature module 302, is used for a plurality of filtering rule
In stream, the identical transaction feature of characteristic value merges；Transaction feature after merging shares same matching identification；Transaction data obtains
Modulus block 304, for obtaining transaction data to be matched；Matching module, for by transaction data one by one with a plurality of filtering rule
Stream is matched；Transaction data with first filtering rule stream when being matched, to each of first filtering rule stream
Transaction feature is matched, and corresponding matching identification is added；Transaction data with the filtering in addition to first filtering rule stream
When regular flow is matched, the transaction feature to matching identification is not present in filtering rule stream matches, and adds corresponding
With mark；Mark carries module 306, if for above-mentioned transaction data and each transaction feature in current filter regular flow
Equal successful match carries the identity of current filter rule in above-mentioned transaction data.
Further, above-mentioned matching module is additionally operable to：Processing is decoded to transaction data, obtains the friendship in transaction data
Easy feature and the corresponding characteristic value of transaction feature；Transaction feature is including at least the IP address of transaction data generation, port address, friendship
It is a variety of in easy type, transaction return code；By transaction feature and the corresponding characteristic value of transaction feature one by one with a plurality of filtering rule
Stream is matched.
Present embodiments provide for a kind of a kind of filtering realizations of transaction data corresponding with above method embodiment
Device.Fig. 4 is a kind of structural schematic diagram of the filtering realization device of transaction data, which can be arranged in distributed storage system
On monitoring node in system；As shown in figure 4, the device includes memory 100 and processor 101；Wherein, memory 100 is used for
One or more computer instruction is stored, one or more computer instruction is executed by processor, to realize above-mentioned transaction data
Filter method, this method may include one or more in above method.
Further, the filtering realization device of transaction data shown in Fig. 4 further includes bus 102 and communication interface 103, processing
Device 101, communication interface 103 and memory 100 are connected by bus 102.
Wherein, memory 100 may include high-speed random access memory (RAM, Random Access Memory),
May further include non-labile memory (non-volatile memory), for example, at least a magnetic disk storage.By extremely
A few communication interface 103 (can be wired or wireless) is realized logical between the system network element and at least one other network element
Letter connection can use internet, wide area network, local network, Metropolitan Area Network (MAN) etc..Bus 102 can be isa bus, pci bus or
Eisa bus etc..The bus can be divided into address bus, data/address bus, controlling bus etc..For ease of indicating, only used in Fig. 4
One four-headed arrow indicates, it is not intended that an only bus or a type of bus.
Processor 101 may be a kind of IC chip, the processing capacity with signal.It is above-mentioned during realization
Each step of method can be completed by the integrated logic circuit of the hardware in processor 101 or the instruction of software form.On
The processor 101 stated can be general processor, including central processing unit (Central Processing Unit, abbreviation
CPU), network processing unit (Network Processor, abbreviation NP) etc.；It can also be digital signal processor (Digital
Signal Processing, abbreviation DSP), application-specific integrated circuit (Application Specific Integrated
Circuit, abbreviation ASIC), ready-made programmable gate array (Field-Programmable Gate Array, abbreviation FPGA) or
Person other programmable logic device, discrete gate or transistor logic, discrete hardware components.It may be implemented or execute sheet
Disclosed each method, step and logic diagram in disclosed embodiment.General processor can be microprocessor or this at
It can also be any conventional processor etc. to manage device.The step of method in conjunction with disclosed in disclosure embodiment, can direct body
Be now that hardware decoding processor executes completion, or in decoding processor hardware and software module combine and execute completion.It is soft
Part module can be located at random access memory, and flash memory, read-only memory, programmable read only memory or electrically erasable programmable are deposited
In the storage medium of this fields such as reservoir, register maturation.The storage medium is located at memory 100, and processor 101 reads storage
Information in device 100, in conjunction with its hardware complete aforementioned embodiments method the step of.
Embodiment of the present invention additionally provides a kind of machine readable storage medium, and machine readable storage medium storage is organic
Device executable instruction, for the machine-executable instruction when being called and being executed by processor, machine-executable instruction promotes processor
Realize that the filter method of above-mentioned transaction data, specific implementation can be found in method embodiment, details are not described herein.
The computer program production of the filter method, device and realization device of the transaction data that the embodiment of the present invention is provided
Product, including the computer readable storage medium of program code is stored, before the instruction that said program code includes can be used for execution
Method described in the embodiment of the method for face, specific implementation can be found in embodiment of the method, and details are not described herein.
In addition, in the description of the embodiment of the present invention unless specifically defined or limited otherwise, term " installation ", " phase
Even ", " connection " shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or be integrally connected；It can
Can also be electrical connection to be mechanical connection；It can be directly connected, can also indirectly connected through an intermediary, Ke Yishi
Connection inside two elements.For the ordinary skill in the art, above-mentioned term can be understood at this with concrete condition
Concrete meaning in invention.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer read/write memory medium.Based on this understanding, technical scheme of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be expressed in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic disc or CD.
In the description of the present invention, it should be noted that term "center", "upper", "lower", "left", "right", "vertical",
The orientation or positional relationship of the instructions such as "horizontal", "inner", "outside" be based on the orientation or positional relationship shown in the drawings, merely to
Convenient for the description present invention and simplify description, do not indicate or imply the indicated device or element must have a particular orientation,
With specific azimuth configuration and operation, therefore it is not considered as limiting the invention.In addition, term " first ", " second ",
" third " is used for description purposes only, and is not understood to indicate or imply relative importance.
Finally it should be noted that：Embodiment described above, only specific implementation mode of the invention, to illustrate the present invention
Technical solution, rather than its limitations, scope of protection of the present invention is not limited thereto, although with reference to the foregoing embodiments to this hair
It is bright to be described in detail, it will be understood by those of ordinary skill in the art that：Any one skilled in the art
In the technical scope disclosed by the present invention, it can still modify to the technical solution recorded in previous embodiment or can be light
It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make
The essence of corresponding technical solution is detached from the spirit and scope of technical solution of the embodiment of the present invention, should all cover the protection in the present invention
Within the scope of.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. a kind of filter method of transaction data, which is characterized in that including：
Obtain a plurality of filtering rule stream；Every filtering rule stream includes one or more transaction features and the transaction feature
Corresponding characteristic value；
By in a plurality of filtering rule stream, the identical transaction feature of characteristic value merges；The transaction feature after merging
Share same matching identification；
Obtain transaction data to be matched；
The transaction data is matched with a plurality of filtering rule stream one by one；
The transaction data with first filtering rule stream when being matched, to each of described first filtering rule stream
Transaction feature is matched, and corresponding matching identification is added；The transaction data in addition to first filtering rule stream
When filtering rule stream is matched, the transaction feature to matching identification is not present in the filtering rule stream matches, and adds
Corresponding matching identification；
If the transaction data and the equal successful match of each transaction feature in current filter regular flow, current filter is advised
Identity then carries in the transaction data.
2. according to the method described in claim 1, it is characterized in that, it is described by the transaction data one by one with a plurality of filtering
The step of regular flow is matched, including：
Processing is decoded to the transaction data, the transaction feature and the transaction feature obtained in the transaction data corresponds to
Characteristic value；The transaction feature is including at least the IP address of transaction data generation, port address, type of transaction, transaction
It is a variety of in return code；
The transaction feature and the corresponding characteristic value of the transaction feature are matched with a plurality of filtering rule stream one by one.
3. according to the method described in claim 2, it is characterized in that, described correspond to the transaction feature and the transaction feature
Characteristic value the step of being matched one by one with a plurality of filtering rule stream, including：
When the transaction feature and the corresponding characteristic value of the transaction feature are matched with current filter regular flow, institute is obtained
State the transaction feature and characteristic value in current filter regular flow；
According to the transaction feature and characteristic value in the current filter regular flow, compare transaction feature in the transaction data and
Characteristic value；
If same transaction feature, the corresponding characteristic value of transaction data feature corresponding with the current filter regular flow
It is worth identical, the then transaction feature successful match.
4. according to the method described in claim 3, it is characterized in that, when the transaction feature successful match, the matching is marked
Know is " 1 "；When it fails to match for the transaction feature, the matching identification is " 0 ".
5. according to the method described in claim 1, it is characterized in that, the method further includes：For a plurality of filtering rule stream
In each transaction feature allocate bit in advance；
In the matching process, the matching identification of addition is inserted into the corresponding bit.
6. according to the method described in claim 5, it is characterized in that, described by a plurality of filtering rule stream, characteristic value phase
The step of same transaction feature merges, including：
By in a plurality of filtering rule stream, the bit of the identical transaction feature distribution of characteristic value merges.
7. if according to the method described in claim 1, it is characterized in that, the transaction data and current filter regular flow
In the equal successful match of each transaction feature, the identity of current filter rule is carried into the step in the transaction data
Suddenly, including：
Field is added in the transaction data；
The identity of current filter regular flow is inserted in the field.
8. a kind of filter device of transaction data, which is characterized in that including：
Filtering rule stream acquisition module, for obtaining a plurality of filtering rule stream；Every filtering rule stream includes one or more
A transaction feature and the corresponding characteristic value of the transaction feature；
Merge transaction feature module, for by a plurality of filtering rule stream, the identical transaction feature of characteristic value to merge；
The transaction feature after merging shares same matching identification；
Transaction data acquisition module, for obtaining transaction data to be matched；
Matching module, for matching the transaction data with a plurality of filtering rule stream one by one；The transaction data
When being matched with first filtering rule stream, to each transaction feature progress in the first filtering rule stream
Match, adds corresponding matching identification；The transaction data is carried out with the filtering rule stream in addition to first filtering rule stream
When matching, the transaction feature to matching identification is not present in the filtering rule stream matches, and adds corresponding matching identification；
Mark carries module, if matched with each transaction feature in current filter regular flow for the transaction data
Success carries the identity of current filter rule in the transaction data.
9. device according to claim 8, which is characterized in that the matching module is additionally operable to：
Processing is decoded to the transaction data, the transaction feature and the transaction feature obtained in the transaction data corresponds to
Characteristic value；The transaction feature is including at least the IP address of transaction data generation, port address, type of transaction, transaction
It is a variety of in return code；
The transaction feature and the corresponding characteristic value of the transaction feature are matched with a plurality of filtering rule stream one by one.
10. a kind of filtering realization device of transaction data, which is characterized in that including memory and processor, wherein the storage
Device is for storing one or more computer instruction, and one or more computer instruction is executed by the processor, with reality
Existing claim 1 to 7 any one of them method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810216684.7A CN108449231B (en) | 2018-03-15 | 2018-03-15 | Transaction data filtering method and device and implementation device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810216684.7A CN108449231B (en) | 2018-03-15 | 2018-03-15 | Transaction data filtering method and device and implementation device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108449231A true CN108449231A (en) | 2018-08-24 |
| CN108449231B CN108449231B (en) | 2020-07-07 |
Family
ID=63194733
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810216684.7A Active CN108449231B (en) | 2018-03-15 | 2018-03-15 | Transaction data filtering method and device and implementation device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108449231B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851687A (en) * | 2019-11-11 | 2020-02-28 | 厦门市美亚柏科信息股份有限公司 | Data identification method, terminal equipment and storage medium |
| CN116993505A (en) * | 2023-09-25 | 2023-11-03 | 腾讯科技（深圳）有限公司 | Transaction processing method, device, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1669038A (en) * | 2002-05-17 | 2005-09-14 | 科罗尼尔第一州立投资有限公司 | Transaction management system |
| CN101655857A (en) * | 2009-09-18 | 2010-02-24 | 西安建筑科技大学 | Method for mining data in construction regulation field based on associative regulation mining technology |
| CN102012918A (en) * | 2010-11-26 | 2011-04-13 | 中金金融认证中心有限公司 | System and method for excavating and executing rule |
| CN102043789A (en) * | 2009-10-21 | 2011-05-04 | 阿里巴巴集团控股有限公司 | Method and device for updating data table |
| US8880539B2 (en) * | 2005-10-26 | 2014-11-04 | Cortica, Ltd. | System and method for generation of signatures for multimedia data elements |
| CN105426365A (en) * | 2014-08-01 | 2016-03-23 | 阿里巴巴集团控股有限公司 | Method and apparatus for distinguishing interactive behavior |
| CN106202389A (en) * | 2016-07-08 | 2016-12-07 | 中国银联股份有限公司 | A kind of method for monitoring abnormality based on transaction data and device |
| CN106991145A (en) * | 2017-03-23 | 2017-07-28 | 中国银联股份有限公司 | A kind of method and device of Monitoring Data |
| CN107256479A (en) * | 2017-05-19 | 2017-10-17 | 深圳市威富通科技有限公司 | The classification of trade mode performs method and device |
-
2018
- 2018-03-15 CN CN201810216684.7A patent/CN108449231B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1669038A (en) * | 2002-05-17 | 2005-09-14 | 科罗尼尔第一州立投资有限公司 | Transaction management system |
| US8880539B2 (en) * | 2005-10-26 | 2014-11-04 | Cortica, Ltd. | System and method for generation of signatures for multimedia data elements |
| CN101655857A (en) * | 2009-09-18 | 2010-02-24 | 西安建筑科技大学 | Method for mining data in construction regulation field based on associative regulation mining technology |
| CN102043789A (en) * | 2009-10-21 | 2011-05-04 | 阿里巴巴集团控股有限公司 | Method and device for updating data table |
| CN102012918A (en) * | 2010-11-26 | 2011-04-13 | 中金金融认证中心有限公司 | System and method for excavating and executing rule |
| CN105426365A (en) * | 2014-08-01 | 2016-03-23 | 阿里巴巴集团控股有限公司 | Method and apparatus for distinguishing interactive behavior |
| CN106202389A (en) * | 2016-07-08 | 2016-12-07 | 中国银联股份有限公司 | A kind of method for monitoring abnormality based on transaction data and device |
| CN106991145A (en) * | 2017-03-23 | 2017-07-28 | 中国银联股份有限公司 | A kind of method and device of Monitoring Data |
| CN107256479A (en) * | 2017-05-19 | 2017-10-17 | 深圳市威富通科技有限公司 | The classification of trade mode performs method and device |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110851687A (en) * | 2019-11-11 | 2020-02-28 | 厦门市美亚柏科信息股份有限公司 | Data identification method, terminal equipment and storage medium |
| CN116993505A (en) * | 2023-09-25 | 2023-11-03 | 腾讯科技（深圳）有限公司 | Transaction processing method, device, electronic equipment and storage medium |
| CN116993505B (en) * | 2023-09-25 | 2024-01-16 | 腾讯科技（深圳）有限公司 | Transaction processing method, device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108449231B (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3579494A1 (en) | Blockchain based roaming | |
| CN108365967A (en) | Method, system, terminal and the computer readable storage medium of dynamic configuration communications parameter | |
| CN108449231A (en) | A kind of filter method of transaction data, device and realization device | |
| CN109885546A (en) | User behaviors log storage method, device and electronic equipment | |
| CN109960587A (en) | The storage resource distribution method and device of super fusion cloud computing system | |
| CN102073954B (en) | A kind of system and method for realizing large-scale business financial clearing clearing | |
| CN110347708A (en) | A kind of data processing method and relevant device | |
| CN106330398A (en) | Method and apparatus for determining data transmission mode of service of Internet of Things | |
| CN108173727A (en) | A kind of intelligent appliance method of network entry and equipment | |
| CN104320354B (en) | A kind of processing method and processing device of data | |
| CN108540387A (en) | Method for network access control and device | |
| CN103685057B (en) | Flow statistical method and device | |
| CN103780435B (en) | The method and system classified using port numbers mask to data stream | |
| CN108984588A (en) | A kind of data processing method and device | |
| CN108712269A (en) | The method for retrieving and device of off-network user | |
| CN109039826B (en) | Collecting method, device and electronic equipment | |
| CN104135445A (en) | Stream-based link aggregation load balancing method and device | |
| CN108234687A (en) | A kind of smart IP address configuration method, device and terminal device | |
| CN110460540A (en) | A kind of low latency high frequency transaction system and method | |
| CN105959306A (en) | IP address identification method and device | |
| CN108834130A (en) | Mark distributing method, device, terminal and computer readable storage medium | |
| CN108337243A (en) | Message forwarding method, device and forwarding unit | |
| CN105659642A (en) | Charging processing method, centralized network control node, function node and system | |
| CN110266834A (en) | The regional lookup method and device of internet protocol-based address | |
| CN110209656A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Room 802, 8th Floor, Shining Building, 35 College Road, Haidian District, Beijing Applicant after: FUSIONSKYE (BEIJING) SOFTWARE Co.,Ltd. Address before: 100020 room 1005, Jin Ji Ye building, No. 2, Sheng Gu Road, anzhen bridge, Chaoyang District, Beijing Applicant before: FUSIONSKYE (BEIJING) TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |