CN105703972B - Data grab method and device applied to audit - Google Patents

Data grab method and device applied to audit Download PDF

Info

Publication number
CN105703972B
CN105703972B CN201610127635.7A CN201610127635A CN105703972B CN 105703972 B CN105703972 B CN 105703972B CN 201610127635 A CN201610127635 A CN 201610127635A CN 105703972 B CN105703972 B CN 105703972B
Authority
CN
China
Prior art keywords
tcp connection
data
network interface
module
tcp
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610127635.7A
Other languages
Chinese (zh)
Other versions
CN105703972A (en
Inventor
向非能
冯庆磊
贾永香
孔德勇
夏运
殷跃
吕大鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Priority to CN201610127635.7A priority Critical patent/CN105703972B/en
Publication of CN105703972A publication Critical patent/CN105703972A/en
Application granted granted Critical
Publication of CN105703972B publication Critical patent/CN105703972B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/02—Capturing of monitoring data
    • H04L43/028—Capturing of monitoring data by filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/04—Processing captured monitoring data, e.g. for logfile generation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/18—Protocol analysers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures

Abstract

The invention discloses a kind of data grab methods applied to audit, it include: that the first TCP connection set is filtered out from the TCP connection in effective status of acquisition, it include the TCP connection of mysql and mysql-proxy in the first TCP connection set, TCP connection in first TCP connection set is associated with the local network interface got, obtain the second TCP connection set, it include the incidence relation of TCP connection and network interface in the second TCP connection set, and filter is set for each of the second TCP connection set TCP connection, and it creates thread and monitors and the associated network interface of the TCP connection, and data are grabbed from the network interface of monitoring by filter, the data of crawl are saved into Circular buffer area.The present invention also provides a kind of data grabber devices applied to audit, make it possible to grab data, improve the performance that auditing service is carried out to SQL database.

Description

Data grab method and device applied to audit
Technical field
Audit technique field of the present invention, more particularly to data grab method and device applied to audit.
Background technique
Currently, auditing to the data in database, primarily to risk of both preventing, one is that inside has The personnel of permission execute illegal operation, the other is external bug excavation and attack.
However, how to realize number when auditing to database due to storing very big data volume in database According to screening and acquisition be a problem to be solved.
Above content is only used to facilitate the understanding of the technical scheme, and is not represented and is recognized that above content is existing skill Art.
Summary of the invention
The main purpose of the present invention is to provide a kind of data grab methods and device applied to audit, it is intended to solve existing Have the data volume in technology in database it is very big when, data screening and acquisition problem when being audited.
To achieve the above object, a kind of data grab method applied to audit provided by the invention, comprising:
The first TCP connection set is filtered out from the transmission control protocol TCP connection in effective status of acquisition, it is described The TCP connection of TCP connection and mysql-proxy in first TCP connection set comprising mysqld；
TCP connection in the first TCP connection set is associated with the local network interface got, obtains Two TCP connection set include the incidence relation of TCP connection and network interface in the second TCP connection set；
Filter is set for each of the second TCP connection set TCP connection, and creates thread monitoring and TCP Connect associated network interface；
Data are grabbed from the network interface of monitoring by the filter, the data of crawl are saved to Circular buffer area In.
Preferably, the TCP connection by the first TCP connection set is carried out with the local network interface got Association, before obtaining the second TCP connection set further include:
Delete the TCP connection that the first TCP connection set middle-end slogan is management end slogan.
Preferably, the TCP connection by the first TCP connection set is carried out with the local network interface got Association, after obtaining the second TCP connection set further include:
Determine that network interface is the network interface for being uniformly accessed into gateway TGW and using in the second TCP connection set；
It is set as the corresponding TCP connection of network interface that the TGW is used not grab the state of data.
Preferably, it is described by the filter from the network interface of monitoring grab data, by the data of crawl save to In Circular buffer area, comprising:
When the thread listens to the data of network interface, filter data is crossed by the filter, and grab and filter out Data；
The data of crawl are stored in the Circular buffer area.
Preferably, the described data of crawl are stored in the Circular buffer area includes:
The data of crawl are decomposed into data head and data content；
It parses the data head and obtains new data head, and by the data content fragment, obtain fragment data；
The new data head and the fragment data are saved into the Circular buffer area.
To achieve the above object, the present invention also proposes a kind of data grabber device applied to audit, comprising:
Filtering module, for filtering out the first TCP from the transmission control protocol TCP connection in effective status of acquisition Articulation set, the TCP connection of TCP connection and mysql-proxy in the first TCP connection set comprising mysqld；
Relating module, for by TCP connection and the local network interface that gets in the first TCP connection set into Row association, obtains the second TCP connection set, being associated with comprising TCP connection and network interface in the second TCP connection set System；
Setting and creation module, for filter to be arranged for each of the second TCP connection set TCP connection, And it creates thread and monitors and the associated network interface of TCP connection；
Preserving module is grabbed, for grabbing data from the network interface of monitoring by the filter, by the data of crawl It saves into Circular buffer area.
Preferably, described device further include:
Removing module, for deleting first TCP connection after filtering module obtains the first TCP connection set Gather the TCP connection that middle-end slogan is management end slogan.
Preferably, described device further include:
Determining module, for determining the 2nd TCP after the relating module obtains the second TCP connection set Network interface is the network interface for being uniformly accessed into gateway TGW and using in articulation set；
State setting module, the corresponding TCP connection of network interface for using the TGW are set as not grabbing data State.
Preferably, the crawl preserving module includes:
Handling module when for listening to the data of network interface in the thread, crosses filter data by the filter, And grab the data filtered out；
Preserving module, for the data of crawl to be stored in the Circular buffer area.
Preferably, the preserving module includes:
Decomposing module, for the data of crawl to be decomposed into data head and data content；
Fragment module is parsed, obtains new data head for parsing the data head, and by the data content fragment, obtain Fragment data；
Cache module, for saving the new data head and the fragment data into the Circular buffer area.
The present invention provides a kind of data grab method applied to audit, this method comprises: being in effective shape from acquisition The first TCP connection set is filtered out in the TCP connection of state, in the first TCP connection set comprising mysql TCP connection and The TCP connection of mysql-proxy carries out the TCP connection in the first TCP connection set with the local network interface got Association, obtains the second TCP connection set, includes the incidence relation of TCP connection and network interface in the second TCP connection set, And filter is set for each of the second TCP connection set TCP connection, and create thread monitoring and be associated with the TCP connection Network interface, and by filter from the network interface of monitoring grab data, the data of crawl are saved to Circular buffer area In, make it possible to grab data by way of filtering out the TCP connection of mysql and mysql-proxy, improves to SQL data The performance of library progress auditing service.
Detailed description of the invention
Fig. 1 is the flow diagram for being applied to the data grab method of audit in first embodiment of the invention；
Fig. 2 is the flow diagram of the addition step of first embodiment shown in Fig. 1 of the present invention；
Fig. 3 is the flow diagram of the refinement step of step 104 in first embodiment shown in Fig. 1 of the present invention；
Fig. 4 is the flow diagram of the refinement step of step 302 in embodiment illustrated in fig. 3 of the present invention；
Fig. 5 is the flow diagram for being applied to the data grab method of audit in second embodiment of the invention；
Fig. 6 is the schematic diagram of the addition functional module of second embodiment shown in Fig. 2 of the present invention；
Fig. 7 is the schematic diagram that the refinement functional module of preserving module 504 is grabbed in second embodiment shown in Fig. 5 of the present invention；
Fig. 8 is the schematic diagram of the refinement functional module of preserving module 702 in embodiment illustrated in fig. 7 of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
Since in the prior art, the data manipulation in SQL database is very more, the auditing service of SQL database will lead to The poor problem of performance.
For this purpose, the present invention proposes a kind of data grab method applied to audit, data can be grabbed, to improve to SQL The performance of database progress auditing service.
Referring to Fig. 1, to be applied to the flow diagram of the data grab method of audit in first embodiment of the invention, it should Grasping means includes:
Step 101 filters out the first TCP connection set, the first TCP from the TCP connection in effective status of acquisition The TCP connection of TCP connection and mysql-proxy in articulation set comprising mysqld；
In embodiments of the present invention, applied to the data grab method of audit by the data grabber device applied to audit (hereinafter referred to as: data grabber device) is realized, data grabber device will find out all TCP connections, and from the TCP connection found out It is middle to obtain the TCP connection for being in effective status, and the first TCP collection will be also filtered out from the TCP connection of the effective status of acquisition It closes.
Wherein, the TCP of the TCP connection comprising all mysql and all mysql-proxy connect in the first TCP set It connects.
Wherein, the TCP connection of effective status refers to the TCP connection of Listen state.
TCP connection in first TCP connection set is associated by step 102 with the local network interface got, is obtained It include the incidence relation of TCP connection and network interface to the second TCP connection set, in the second TCP connection set；
In embodiments of the present invention, data grabber device will acquire all local network interfaces, and the first TCP is connected The TCP connection connect in set is associated with the local network interface got, obtains the second TCP connection set, and at this Include the incidence relation between TCP connection and network interface in second TCP connection set.
Wherein, TCP connection is associated with network interface and is specifically included: checking the interface identifier of TCP connection, if the TCP connects The interface identifier connect is 0, then by the TCP connection and all-network interface conjunctionn, makes it possible to monitor all interfaces, if should The interface identifier of TCP connection is not 0, then is associated with the TCP connection with specified network interface, and monitor the finger of the TCP connection Fixed associated network interface.
Filter is arranged for each of the second TCP connection set TCP connection in step 103, and create thread monitor with The associated network interface of TCP connection；
In embodiments of the present invention, data grabber device is after obtaining the second TCP connection set, for the 2nd TCP company Each of set TCP connection setting filter is connect, and creates thread and monitors and the associated network interface of the TCP connection.
Step 104 grabs data from the network interface of monitoring by filter, and the data of crawl are saved to Circular buffer Qu Zhong.
In embodiments of the present invention, data grabber device will grab data from the network interface of monitoring by filter, and The data of crawl are saved into Circular buffer area.
In embodiments of the present invention, data grabber device filters out from the TCP connection in effective status of acquisition One TCP connection set, the TCP connection of TCP connection and mysql-proxy in the first TCP connection set comprising mysql, will TCP connection in the first TCP connection set is associated with the local network interface got, obtains the second TCP connection collection It closes, includes the incidence relation of TCP connection and network interface in the second TCP connection set, and in the second TCP connection set Filter is arranged in each TCP connection, and creates thread monitoring and the associated network interface of the TCP connection, and pass through filter Data are grabbed from the network interface of monitoring, the data of crawl are saved into Circular buffer area, are made it possible to by filtering out The mode of the TCP connection of mysql and mysql-proxy grabs data, improves the performance that auditing service is carried out to SQL database.
Further, before the step 102 of the first embodiment shown in Fig. 1 of the present invention, following steps can also be performed:
Delete the TCP connection that the first TCP connection set middle-end slogan is management end slogan.
In embodiments of the present invention, data grabber device will delete the first TCP after obtaining the first TCP connection set Port numbers in articulation set are the TCP connection of management end slogan.For example, if port numbers 4300 to 4500 are management port Number, then delete the TCP connection of the first TCP connection set middle-end slogan 4300 to 4500.
In embodiments of the present invention, due in management end slogan and do not include audit required for data, by deleting Except the TCP connection that port numbers are management end slogan, it can be avoided some unwanted data of crawl, so that the data grabbed are equal For required data of auditing, help to improve the performance for carrying out SQL database auditing service.
Further, referring to Fig. 2, the flow diagram of the addition step for first embodiment shown in Fig. 1 of the present invention, is somebody's turn to do Additional step includes:
Step 201 determines the network interface that network interface uses in the second TCP connection set for TGW；
Step 202 is set as the corresponding TCP connection of network interface that TGW is used not grab the state of data.
In embodiments of the present invention, data grabber device execute the step 102 in first embodiment shown in above-mentioned Fig. 1 it Afterwards, the second TCP connection set is obtained, and the data grabber device will determine that network interface is system in the second TCP connection set The network interface that one access gateway (TencentGateway, TGW) uses, and the network interface that TWG is used is set as not grabbing The state for evidence of fetching does not grab data from the network interface of TWG function.Such as: if the title of network interface is started with tunl It but is not tunl0:0, it is determined that the network interface is the network interface that TWG function uses, and it is not grab that the network interface, which is arranged, Data mode.
In embodiments of the present invention, it is set as not grabbing the state of data by the network interface for using TGW, so that will The network interface crawl data not used from TGW function, can effectively realize database service of the audit with TGW function.
Referring to Fig. 3, for the flow diagram of the refinement step of step 104 in first embodiment shown in Fig. 1 of the present invention, it should The refinement step of step 104 includes:
Step 301, when thread listens to the data of network interface, filter data is crossed by filter, and grab and filter out Data；
The data of crawl are stored in Circular buffer area by step 302.
In embodiments of the present invention, data grabber device is arranged for each of the second TCP connection set TCP connection Filter, and create thread and monitor with after the associated network interface of each TCP connection, network will be listened in thread connect When the data of mouth, filter data is crossed by filter, and grab the data filtered out, and the data of crawl are stored in Circular buffer Qu Zhong effectively realizes the crawl of data.
Referring to Fig. 4, for the flow diagram of the refinement step of step 302 in embodiment illustrated in fig. 3 of the present invention, the step 302 refinement step includes:
The data of crawl are decomposed into data head and data content by step 401；
Step 402, parsing data head obtain new data head, and by data content fragment, obtain fragment data；
Step 403 saves new data head and fragment data into Circular buffer area.
In embodiments of the present invention, the data grabbed are decomposed into number after grabbing data by data grabber device It according to head and data content, and parses the data head and obtains new data head, obtained data content will be decomposed and carry out fragment, obtained Fragment data, and new data head and fragment data are saved into Circular buffer area.
Wherein, new data head includes source IP, source port number, source MAC, destination IP, destination slogan and purpose MAC Address.
Wherein, the data in data head decomposed are mixed and disorderly, and format does not meet the requirement of Data Audit, passes through The data header parsing is obtained into new data head, so that the data in new data head are to arrange according to presetting rule, and meet number According to the call format of audit.
In embodiments of the present invention, by the way that the data of crawl are decomposed into data head and data content, and data head is parsed New data head is obtained, and data content fragment is obtained into fragment data, obtained new data head and fragment data are saved Into Circular buffer area, makes it possible to cache the data for meeting SQL database, meet the memory requirement of SQL database.
Referring to Fig. 5, to be applied to the signal of the function mould of the data grabber device of audit in second embodiment of the invention Figure, comprising: filtering module 501, relating module 502, setting and creation module 503 and crawl preserving module 504.
Wherein, filtering module 501, for being filtered from the transmission control protocol TCP connection in effective status of acquisition First TCP connection set out, the TCP connection of TCP connection and mysql-proxy in the first TCP connection set comprising mysqld；
In embodiments of the present invention, data grabber device will find out all TCP connections, and from the TCP connection found out The TCP connection for being in effective status is obtained, and will also filter out the first TCP set from the TCP connection of the effective status of acquisition.
Wherein, the TCP of the TCP connection comprising all mysql and all mysql-proxy connect in the first TCP set It connects.
Wherein, the TCP connection of effective status refers to the TCP connection of Listen state.
Relating module 502, for by TCP connection and the local network interface that gets in the first TCP connection set into Row association, obtains the second TCP connection set, includes the incidence relation of TCP connection and network interface in the second TCP connection set；
In embodiments of the present invention, data grabber device will acquire all local network interfaces, and by relating module 502 are associated the TCP connection in the first TCP connection set with the local network interface got, obtain the 2nd TCP company Set is connect, and includes the incidence relation between TCP connection and network interface in the second TCP connection set.
Wherein, TCP connection is associated with network interface and is specifically included: checking the interface identifier of TCP connection, if the TCP connects The interface identifier connect is 0, then by the TCP connection and all-network interface conjunctionn, makes it possible to monitor all interfaces, if should The interface identifier of TCP connection is not 0, then is associated with the TCP connection with specified network interface, and monitor the finger of the TCP connection Fixed associated network interface.
Setting and creation module 503, for filter to be arranged for each of the second TCP connection set TCP connection, and Thread is created to monitor and the associated network interface of TCP connection；
In embodiments of the present invention, after obtaining the second TCP connection set, setting and creation module 503 be this second Filter is arranged in each of TCP connection set TCP connection, and creates thread monitoring and connect with the associated network of the TCP connection Mouthful.
Preserving module 504 is grabbed, for grabbing data from the network interface of monitoring by filter, the data of crawl are protected It deposits into Circular buffer area.
In embodiments of the present invention, crawl preserving module 504 will grab data from the network interface of monitoring by filter, And the data of crawl are saved into Circular buffer area.
In embodiments of the present invention, filtering module 501 filters out first from the TCP connection in effective status of acquisition TCP connection set, the TCP connection of TCP connection and mysql-proxy in the first TCP connection set comprising mysql, association TCP connection in the first TCP connection set is associated by module 502 with the local network interface got, obtains second TCP connection set includes the incidence relation of TCP connection and network interface in the second TCP connection set, and by being arranged and creating Module 503 is that filter is arranged in each of the second TCP connection set TCP connection, and creates thread and monitor and the TCP connection Associated network interface, and data are grabbed from the network interface of monitoring by filter by crawl preserving module 504, by crawl Data are saved into Circular buffer area, make it possible to grab by way of filtering out the TCP connection of mysql and mysql-proxy Access evidence, improves the performance that auditing service is carried out to SQL database.
Further, in embodiments of the present invention, the data grabber device in second embodiment shown in Fig. 5 further include: delete Except module (not shown).
Wherein, removing module, for deleting the first TCP connection set after filtering module obtains the first TCP connection set Middle-end slogan is the TCP connection of management end slogan.
In embodiments of the present invention, after obtaining the first TCP connection set, removing module will delete first TCP connection Port numbers in set are the TCP connection of management end slogan.For example, if port numbers 4300 to 4500 are management end slogan, Delete the TCP connection of the first TCP connection set middle-end slogan 4300 to 4500.
In embodiments of the present invention, due in management end slogan and do not include audit required for data, by deleting Except the TCP connection that port numbers are management end slogan, it can be avoided some unwanted data of crawl, so that the data grabbed are equal For required data of auditing, help to improve the performance for carrying out SQL database auditing service.
Further, referring to Fig. 6, the schematic diagram of the addition functional module for second embodiment shown in Fig. 5 of the present invention, is somebody's turn to do Additional functional module includes: determining module 601 and state setting module 602.
Determining module 601, for determining the second TCP connection collection after relating module 502 obtains the second TCP connection set Network interface is the network interface for being uniformly accessed into gateway TGW and using in conjunction；
State setting module 602, the corresponding TCP connection of network interface for using TGW are set as not grabbing data State.
In embodiments of the present invention, the second TCP connection set is obtained in relating module 502, determining module 601 will determine should The network interface that network interface uses in second TCP connection set for TGW, and the net for being used TWG by state setting module 602 Network interface is set as not grabbing the state of data, i.e., does not grab data from the network interface of TWG function.
In embodiments of the present invention, it is set as not grabbing the state of data by the network interface for using TGW, so that will The network interface crawl data not used from TGW function, can effectively realize database service of the audit with TGW function.
Referring to Fig. 7, to grab the refinement functional module of preserving module 504 in second embodiment shown in Fig. 5 of the present invention Schematic diagram, the crawl preserving module 504 include: handling module 701 and preserving module 702.
Handling module 701 when for listening to the data of network interface in thread, crosses filter data by filter, and grab Take the data filtered out；
Preserving module 702, for the data of crawl to be stored in Circular buffer area.
In embodiments of the present invention, setting and creation module 503 connect for each of the second TCP connection set TCP Setting filter is connect, and creates thread and monitors with after the associated network interface of each TCP connection, handling module 701 will When thread listens to the data of network interface, filter data is crossed by filter, and grab the data filtered out, and by preservation mould The data of crawl are stored in Circular buffer area by block 702, effectively realize the crawl of data.
Referring to Fig. 8, for the schematic diagram of the refinement functional module of preserving module 702 in embodiment illustrated in fig. 7, the preservation mould Block 702 includes: decomposing module 801, parsing fragment module 802 and cache module 803.
Decomposing module 801, for the data of crawl to be decomposed into data head and data content；
Fragment module 802 is parsed, obtains new data head for parsing data head, and by data content fragment, obtain fragment Data；
Cache module 803, for saving new data head and fragment data into Circular buffer area.
In embodiments of the present invention, handling module 701 is after grabbing data, the number that decomposing module 801 will grab According to being decomposed into data head and data content, and the data head is parsed by parsing fragment module 802 and obtains new data head, will decomposed Obtained data content carries out fragment, obtains fragment data, and saved new data head and fragment data by cache module 803 Into Circular buffer area.
Wherein, new data head includes source IP, source port number, source MAC, destination IP, destination slogan and purpose MAC Address.
Wherein, the data in data head decomposed are mixed and disorderly, and format does not meet the requirement of Data Audit, passes through The data header parsing is obtained into new data head, so that the data in new data head are to arrange according to presetting rule, and meet number According to the call format of audit.
In embodiments of the present invention, by the way that the data of crawl are decomposed into data head and data content, and data head is parsed New data head is obtained, and data content fragment is obtained into fragment data, obtained new data head and fragment data are saved Into Circular buffer area, makes it possible to cache the data for meeting SQL database, meet the memory requirement of SQL database.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Based on this understanding, technical solution of the present invention substantially in other words does the prior art The part contributed out can be embodied in the form of software products, which is stored in a storage medium In (such as ROM/RAM, magnetic disk, CD), including some instructions are used so that a terminal device (can be mobile phone, computer, clothes Be engaged in device, air conditioner or the network equipment etc.) method that executes each embodiment of the present invention.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills Art field, is included within the scope of the present invention.

Claims (10)

1. a kind of data grab method applied to audit characterized by comprising
Filter out the first TCP connection set from transmission control protocol TCP connection of the acquisition in Listen state, described the One TCP connection collection is combined into the set of the TCP connection comprising mysqld and the TCP connection of mysql-proxy；
TCP connection in the first TCP connection set is associated with the local network interface got, obtains second TCP connection set, the incidence relation of TCP connection and network interface in the second TCP connection set comprising the mysqld, And the mysql-proxy TCP connection and network interface incidence relation；
Filter is set for each of the second TCP connection set TCP connection, and creates thread and monitors and described second The associated network interface of TCP connection in TCP connection set；
Data are grabbed from the network interface in the second TCP connection set of monitoring by the filter, by crawl Data are saved into Circular buffer area；
Wherein, the TCP connection by the first TCP connection set and the local network interface got are associated Step includes: the interface identifier for checking TCP connection, if the interface identifier of the TCP connection is 0, by the TCP connection and all nets Network interface conjunctionn makes it possible to monitor all interfaces, if the interface identifier of the TCP connection is not 0, by the TCP connection with Specified network interface association, and monitor the network interface of the specified associations of the TCP connection.
2. the method according to claim 1, wherein the TCP connection by the first TCP connection set It is associated with the local network interface got, before obtaining the second TCP connection set further include:
Delete the TCP connection that the first TCP connection set middle-end slogan is management end slogan.
3. the method according to claim 1, wherein the TCP connection by the first TCP connection set It is associated with the local network interface got, after obtaining the second TCP connection set further include:
Determine that network interface is the network interface for being uniformly accessed into gateway TGW and using in the second TCP connection set；
It is set as the corresponding TCP connection of network interface that the TGW is used not grab the state of data.
4. according to claim 1 to method described in 3 any one, which is characterized in that it is described by the filter from monitoring Network interface grab data, the data of crawl are saved into Circular buffer area, comprising:
When the thread listens to the data of network interface, filter data is crossed by the filter, and grab the number filtered out According to；
The data of crawl are stored in the Circular buffer area.
5. according to the method described in claim 4, it is characterized in that, described be stored in the Circular buffer area for the data of crawl In include:
The data of crawl are decomposed into data head and data content；
It parses the data head and obtains new data head, and by the data content fragment, obtain fragment data；
The new data head and the fragment data are saved into the Circular buffer area.
6. a kind of data grabber device applied to audit characterized by comprising
Filtering module connects for filtering out the first TCP from the transmission control protocol TCP connection in Listen state of acquisition Set is connect, the first TCP connection collection is combined into the set of the TCP connection comprising mysqld and the TCP connection of mysql-proxy；
Relating module, for checking the interface identifier of TCP connection, if the interface identifier of the TCP connection is 0, by the TCP connection With all-network interface conjunctionn, make it possible to monitor all interfaces, it, should if the interface identifier of the TCP connection is not 0 TCP connection is associated with specified network interface, and monitors the network interface of the specified associations of the TCP connection, obtains the 2nd TCP company Set is connect, the TCP of the TCP connection comprising the mysqld and the mysql-proxy connect in the second TCP connection set Connect the incidence relation with network interface；
Setting and creation module for filter to be arranged for each of the second TCP connection set TCP connection, and are created It builds thread monitoring and connect associated network interface with the TCP in the second TCP connection set；
Grab preserving module, for by the filter from the network interface in the second TCP connection set of monitoring Data are grabbed, the data of crawl are saved into Circular buffer area.
7. device according to claim 6, which is characterized in that described device further include:
Removing module, for deleting the first TCP connection set after filtering module obtains the first TCP connection set Middle-end slogan is the TCP connection of management end slogan.
8. device according to claim 6, which is characterized in that described device further include:
Determining module, for determining second TCP connection after the relating module obtains the second TCP connection set Network interface is the network interface for being uniformly accessed into gateway TGW and using in set；
State setting module, the corresponding TCP connection of network interface for using the TGW are set as not grabbing the shape of data State.
9. according to device described in claim 6 to 8 any one, which is characterized in that the crawl preserving module includes:
Handling module when for listening to the data of network interface in the thread, crosses filter data by the filter, and grab Take the data filtered out；
Preserving module, for the data of crawl to be stored in the Circular buffer area.
10. device according to claim 9, which is characterized in that the preserving module includes:
Decomposing module, for the data of crawl to be decomposed into data head and data content；
Fragment module is parsed, obtains new data head for parsing the data head, and by the data content fragment, obtain fragment Data；
Cache module, for saving the new data head and the fragment data into the Circular buffer area.
CN201610127635.7A 2016-03-07 2016-03-07 Data grab method and device applied to audit Active CN105703972B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610127635.7A CN105703972B (en) 2016-03-07 2016-03-07 Data grab method and device applied to audit

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610127635.7A CN105703972B (en) 2016-03-07 2016-03-07 Data grab method and device applied to audit

Publications (2)

Publication Number Publication Date
CN105703972A CN105703972A (en) 2016-06-22
CN105703972B true CN105703972B (en) 2019-09-03

Family

ID=56220078

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610127635.7A Active CN105703972B (en) 2016-03-07 2016-03-07 Data grab method and device applied to audit

Country Status (1)

Country Link
CN (1) CN105703972B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110569178B (en) * 2019-09-12 2022-12-27 成都中科大旗软件股份有限公司 Interface early warning method and system based on big data platform

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102243652A (en) * 2011-06-16 2011-11-16 苏州阔地网络科技有限公司 Database connection management method and device
CN202652270U (en) * 2012-06-15 2013-01-02 上海理工大学 Database audit system
CN104063473A (en) * 2014-06-30 2014-09-24 江苏华大天益电力科技有限公司 Database auditing monitoring system and database auditing monitoring method
CN105337976A (en) * 2015-11-06 2016-02-17 西安交大捷普网络科技有限公司 Real-time high-efficiency database audit realization method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6904137B2 (en) * 2001-07-31 2005-06-07 Sbc Technology Resources, Inc. System and method for creating and accessing outgoing telephone call log

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102243652A (en) * 2011-06-16 2011-11-16 苏州阔地网络科技有限公司 Database connection management method and device
CN202652270U (en) * 2012-06-15 2013-01-02 上海理工大学 Database audit system
CN104063473A (en) * 2014-06-30 2014-09-24 江苏华大天益电力科技有限公司 Database auditing monitoring system and database auditing monitoring method
CN105337976A (en) * 2015-11-06 2016-02-17 西安交大捷普网络科技有限公司 Real-time high-efficiency database audit realization method

Also Published As

Publication number Publication date
CN105703972A (en) 2016-06-22

Similar Documents

Publication Publication Date Title
CN104063473B (en) A kind of database audit monitoring system and its method
CN103646209B (en) The method and apparatus intercepting bundled software based on cloud security
CN101599963B (en) Suspected network threat information screener and screening and processing method
US7961633B2 (en) Method and system for real time detection of threats in high volume data streams
CN109271793B (en) Internet of things cloud platform equipment category identification method and system
CN106953837A (en) With the visual integrating security system of threat
CN112291232A (en) Safety capability and safety service chain management platform based on tenants
KR20000057209A (en) Method and apparatus for automated network-wide surveillance and security breach intervention
CN104333533B (en) A kind of packet zero-copy acquisition methods for industrial control system network
CN105703972B (en) Data grab method and device applied to audit
CN113098851B (en) Method, device, system, equipment and medium for implementing virtual firewall
CN103618720A (en) Method and system for Trojan network communication detecting and evidence obtaining
CN103365963B (en) Database audit system compliance method for quickly detecting
CN110324334A (en) Secure group policy management method, device, equipment and computer readable storage medium
CN109800571A (en) Event-handling method and device and storage medium and electronic device
CN105809031A (en) Database auditing method, apparatus and system
CN106997313A (en) A kind of signal processing method of application program, system and terminal device
CN107426017A (en) A kind of method for carrying out data analysis by gathering switch network flow
CN202652270U (en) Database audit system
CN113904829A (en) Application firewall system based on machine learning
DE112021000455T5 (en) DEEP PACKET ANALYSIS
CN106100895A (en) Application performance achievement data acquisition method and system
CN106302306A (en) A kind of flow statistical method based on access control list ACL and device
CN102053970B (en) Database auditing method and system
CN113792076A (en) Data auditing system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant