CN105337976A - Real-time high-efficiency database audit realization method - Google Patents
Real-time high-efficiency database audit realization method Download PDFInfo
- Publication number
- CN105337976A CN105337976A CN201510747643.7A CN201510747643A CN105337976A CN 105337976 A CN105337976 A CN 105337976A CN 201510747643 A CN201510747643 A CN 201510747643A CN 105337976 A CN105337976 A CN 105337976A
- Authority
- CN
- China
- Prior art keywords
- data packets
- effective data
- packet
- database audit
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a real-time high-efficiency database audit realization method. A database audit device stores data packets received from a network interface card in a large memory area, through filtering and shunting the data packets, valid data packets are obtained for going into annular queues, layered processing is performed on the valid data packets obtained from the annular queues, and finally, recording and responding are performed on the valid data packets after the layered processing. According to the invention, the data packets in a network can be grabbed in real time, the data packets are stored in the large memory area distributed beforehand, pointer addresses of the data packets are added to the corresponding annular queues, the data packets are processed through a "zero copy" technology, and the processing speed of the data packets is greatly improved. At the same time, a packet capture engine performs filtering processing on vast data packets so that resource occupation of an audit system is substantially reduced; and since the multiple annular queues are corresponding to multiple analysis engines, such a concurrent processing mechanism improves the processing capability of the data packets.
Description
Technical field
The invention belongs to database audit technical field, be specifically related to a kind of database audit method of real-time high-efficiency.
Background technology
Along with computer and network technologies development, the application of information system is more and more extensive.Database is as the core of information technology and basis, and carry increasing key service system, gradually become in business and public safety and have strategic assets most, can the safe and stable operation of database also directly decide operation system and normally use.
The development rapidly of the Internet makes the value of business data library information and accessibility be improved, and meanwhile, also causes database information assets to be faced with formidable challenges.The security threat of database is mainly from two aspects, and on the one hand from the illegal invasion of enterprise external, hacker, for operation system or database leak, takes various attack means, distorts or steals data.This part threat effectively can be prevented by disposing the product such as fire compartment wall, IPS at business network entrance.And threat is on the other hand from enterprises, the malicious sabotage of interior employee, violation operation and unauthorized access, often bring data leak in a large number and badly damaged, even cause Database Systems to be collapsed.And these operations often do not possess attack signature, be difficult to be identified by common Information Security Defending System, just more hard to guard against, protect in the urgent need to the effective means of one.
Summary of the invention
In view of this, main purpose of the present invention is the database audit implementation method providing a kind of real-time high-efficiency.
For achieving the above object, technical scheme of the present invention is achieved in that
The embodiment of the present invention provides a kind of database audit implementation method of real-time high-efficiency, the method is: the packet received from network interface card is stored in large memory field by database audit equipment, after described packet is filtered and is shunted, acquisition effective data packets enters annular and lines up, layered shaping is carried out to the effective data packets of lining up middle acquisition from described annular, finally the effective data packets after layered shaping is recorded and respond.
In such scheme, described by after filtering described packet and shunt, acquisition effective data packets enters annular and lines up, be specially: the paging having stored packet in the large memory field of described database audit equipment searching loop, read the packet of fixed number successively, and judge whether the source port of packet and destination interface are registered, if registered, determine that described packet is effective data packets and sends into corresponding flow queue, otherwise described packet is discharged from paging; For the effective data packets read from flow queue, five-tuple information according to described effective data packets determines hash value according to CRC20 algorithm, the queue that described effective data packets should flow into is determined, then by the circle queue of the pointer of described effective data packets stored in correspondence according to the described hash value determined.
In such scheme, the described effective data packets to lining up middle acquisition from described annular carries out layered shaping, finally the effective data packets after layered shaping recorded and respond, be specially: described database audit equipment obtains effective data packets from circle queue, too layer head is gone to described effective data packets, other layer protocols, IPV4/6 head, TCP head/UPD head operation, extract the application layer data in described effective data packets, then carry out corresponding plug-in unit to affiliated application layer data to resolve, parse the field of needs and mate with the field that audit strategy configures, if the match is successful, then the data field of parsing is built into daily record to carry out recording and responding.
In such scheme, described five-tuple information comprises source IP, object IP, source port, destination interface, agreement ID.
In such scheme, his layer protocol described comprises VLAN, L2TP, PPPOE, MPLS, self-defined tunneling.
In such scheme, described database audit equipment read simultaneously several annulars line up in valid data parallel carry out layered shaping, finally the effective data packets after layered shaping recorded and respond.
Compared with prior art, beneficial effect of the present invention:
The present invention can capture the packet in network in real time, by packet stored in the large memory field distributed in advance, and the pointer address of this packet is added in corresponding circle queue, the process of packet is realized by " zero-copy " technology, substantially increases the processing speed of packet.The engine of packet capturing simultaneously, by carrying out filtration treatment to mass data bag, significantly reduces the resource occupation of auditing system; Due to the corresponding multiple analysis engine of multiple circle queue, this concurrent processing mechanism, improves the disposal ability of packet.
Accompanying drawing explanation
Fig. 1 provides a kind of flow chart of database audit implementation method of real-time high-efficiency for the embodiment of the present invention.
Embodiment
Below in conjunction with the drawings and specific embodiments, the present invention is described in detail.
The embodiment of the present invention provides a kind of database audit implementation method of real-time high-efficiency, and the method realizes especially by following steps:
Step 101: the packet received from network interface card is stored in large memory field by database audit equipment.
Concrete, the memory field opened up time described large internal memory position system starts.
Step 102: after filtering described packet and shunt, acquisition effective data packets enters annular and lines up.
Concrete, filter process: because each agreement of audit that needs can register its corresponding port information, so stored the paging of packet in the large memory field of described database audit equipment searching loop, read the packet of fixed number successively, and judge whether the source port of packet and destination interface are registered, if registered, determine that described packet is effective data packets and sends into corresponding flow queue, otherwise described packet is discharged from paging;
Branching process: for effective data packets, five-tuple information according to described effective data packets determines hash value according to CRC20 algorithm, the queue that described effective data packets should flow into is determined, then by the circle queue of the pointer of described effective data packets stored in correspondence according to the described hash value determined.
Described five-tuple information comprises source IP, object IP, source port, destination interface, agreement ID.
Step 103: layered shaping is carried out to the effective data packets of lining up middle acquisition from described annular, finally the effective data packets after layered shaping is recorded and respond.
Concrete, described circle queue is that prototype is transformed with round-robin queue, and circle queue joins end to end, and the bag be stored in circle queue take timestamp as order arrangement, stores the pointer of packet, is also the address of packet in internal memory.Circle queue comprises each one of pointer end to end, and head pointer is responsible for data fetch packet, and tail pointer is responsible for deposit data, when head and the tail point to same data area, represent that circle queue stores full, then by data release in the region of head pointer indication, ensure that data are from tail pointer always stored in circle queue.
Described database audit equipment obtains effective data packets from circle queue, too layer head, other layer protocols, IPV4/6 head, TCP head/UPD head operation are gone to described effective data packets, extract the application layer data in described effective data packets, then carry out corresponding plug-in unit to affiliated application layer data to resolve, parse the field of needs and mate with the field that audit strategy configures, if the match is successful, then the data field of parsing is built into daily record and carries out recording and responding; Otherwise this packet will discharge.
His layer protocol described comprises VLAN, L2TP, PPPOE, MPLS, self-defined tunneling.
Described database audit equipment read simultaneously several annulars line up in valid data parallel carry out layered shaping, finally the effective data packets after layered shaping recorded and respond.
The mysql protocol data bag of the ipv4 type encapsulating vlan head is such as got from circle queue, first the ether layer head of this packet and vlan head are removed, then ip head is removed, wrap if this packet is out of order bag or restructuring, this packet can store and carry out sorting and reorganization operation with other packets of this stream by analysis engine, tcp head is removed again after having operated, remaining application layer data is given mysql plug-in unit to go to resolve according to the protocol format of mysql, compare with the field in the strategy of configuration after mysql plug-in unit parses the field information of needs, if comparison success, is built into daily record and carries out recording and responding by the data field of parsing.
The process of described effective data packets completes, need to analyze data Packet analyzing result, carry out alarm for database violation operation, notification data library manager process in time, the analytic functions such as the inquiry of daily record, statistics are also provided, and can define according to user the form generating various form.
The above, be only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.
Claims (6)
1. the database audit implementation method of a real-time high-efficiency, it is characterized in that, the method is: the packet received from network interface card is stored in large memory field by database audit equipment, after described packet is filtered and is shunted, acquisition effective data packets enters annular and lines up, layered shaping is carried out to the effective data packets of lining up middle acquisition from described annular, finally the effective data packets after layered shaping is recorded and respond.
2. the database audit implementation method of real-time high-efficiency according to claim 1, it is characterized in that, described by after filtering described packet and shunt, acquisition effective data packets enters annular and lines up, be specially: the paging having stored packet in the large memory field of described database audit equipment searching loop, read the packet of fixed number successively, and judge whether the source port of packet and destination interface are registered, if registered, determine that described packet is effective data packets and sends into corresponding flow queue, otherwise described packet is discharged from paging; For the effective data packets read from flow queue, five-tuple information according to described effective data packets determines hash value according to CRC20 algorithm, the queue that described effective data packets should flow into is determined, then by the circle queue of the pointer of described effective data packets stored in correspondence according to the described hash value determined.
3. the database audit implementation method of real-time high-efficiency according to claim 1, it is characterized in that, the described effective data packets to lining up middle acquisition from described annular carries out layered shaping, finally the effective data packets after layered shaping recorded and respond, be specially: described database audit equipment obtains effective data packets from circle queue, too layer head is gone to described effective data packets, other layer protocols, IPV4/6 head, TCP head/UPD head operation, extract the application layer data in described effective data packets, then carry out corresponding plug-in unit to affiliated application layer data to resolve, parse the field of needs and mate with the field that audit strategy configures, if the match is successful, then the data field of parsing is built into daily record to carry out recording and responding.
4. the database audit implementation method of real-time high-efficiency according to claim 2, is characterized in that: described five-tuple information comprises source IP, object IP, source port, destination interface, agreement ID.
5. the database audit implementation method of real-time high-efficiency according to claim 3, is characterized in that: his layer protocol described comprises VLAN, L2TP, PPPOE, MPLS, self-defined tunneling.
6. the database audit implementation method of real-time high-efficiency according to claim 3, it is characterized in that: described database audit equipment read simultaneously several annulars line up in valid data parallel carry out layered shaping, finally the effective data packets after layered shaping recorded and respond.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510747643.7A CN105337976A (en) | 2015-11-06 | 2015-11-06 | Real-time high-efficiency database audit realization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510747643.7A CN105337976A (en) | 2015-11-06 | 2015-11-06 | Real-time high-efficiency database audit realization method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105337976A true CN105337976A (en) | 2016-02-17 |
Family
ID=55288259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510747643.7A Pending CN105337976A (en) | 2015-11-06 | 2015-11-06 | Real-time high-efficiency database audit realization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105337976A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105703972A (en) * | 2016-03-07 | 2016-06-22 | 深圳前海微众银行股份有限公司 | Data capturing method and device applied to auditing process |
| CN106060149A (en) * | 2016-06-24 | 2016-10-26 | 北京交通大学 | Mobile internet mass data analysis and audit technical architecture |
| CN106789728A (en) * | 2017-01-25 | 2017-05-31 | 甘肃农业大学 | A kind of voip traffic real-time identification method based on NetFPGA |
| CN107465567A (en) * | 2017-06-29 | 2017-12-12 | 西安交大捷普网络科技有限公司 | A kind of data forwarding method of database fire wall |
| CN108667921A (en) * | 2018-04-27 | 2018-10-16 | 中国农业银行股份有限公司 | A kind of banking recommendation information generation method and system based on network bypass |
| CN109600304A (en) * | 2018-12-21 | 2019-04-09 | 成都九洲电子信息系统股份有限公司 | Based on time wheel mail data reduction, threat detection and trend behavior analysis method |
| CN110502391A (en) * | 2019-07-12 | 2019-11-26 | 苏宁云计算有限公司 | The grasping means and system of SQL information in a kind of MySQL database |
| CN111240599A (en) * | 2020-01-17 | 2020-06-05 | 北京马赫谷科技有限公司 | Data stream storage method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007038856A1 (en) * | 2005-10-05 | 2007-04-12 | Nortel Networks Limited | Provider link state bridging |
| CN101150485A (en) * | 2007-11-15 | 2008-03-26 | 曙光信息产业(北京)有限公司 | A management method for network data transmission of zero copy buffer queue |
| CN101764760A (en) * | 2010-03-24 | 2010-06-30 | 深圳市中科新业信息科技发展有限公司 | Multilink message capturing method, and method and system for processing multilink message |
| CN102739473A (en) * | 2012-07-09 | 2012-10-17 | 南京中兴特种软件有限责任公司 | Network detecting method using intelligent network card |
| CN104994032A (en) * | 2015-05-15 | 2015-10-21 | 京信通信技术(广州)有限公司 | Information processing method and apparatus |
-
2015
- 2015-11-06 CN CN201510747643.7A patent/CN105337976A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007038856A1 (en) * | 2005-10-05 | 2007-04-12 | Nortel Networks Limited | Provider link state bridging |
| CN101150485A (en) * | 2007-11-15 | 2008-03-26 | 曙光信息产业(北京)有限公司 | A management method for network data transmission of zero copy buffer queue |
| CN101764760A (en) * | 2010-03-24 | 2010-06-30 | 深圳市中科新业信息科技发展有限公司 | Multilink message capturing method, and method and system for processing multilink message |
| CN102739473A (en) * | 2012-07-09 | 2012-10-17 | 南京中兴特种软件有限责任公司 | Network detecting method using intelligent network card |
| CN104994032A (en) * | 2015-05-15 | 2015-10-21 | 京信通信技术(广州)有限公司 | Information processing method and apparatus |
Non-Patent Citations (1)
| Title |
|---|
| 董志良: "《电子商务概论》", 30 June 2014 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105703972A (en) * | 2016-03-07 | 2016-06-22 | 深圳前海微众银行股份有限公司 | Data capturing method and device applied to auditing process |
| CN105703972B (en) * | 2016-03-07 | 2019-09-03 | 深圳前海微众银行股份有限公司 | Data grab method and device applied to audit |
| CN106060149A (en) * | 2016-06-24 | 2016-10-26 | 北京交通大学 | Mobile internet mass data analysis and audit technical architecture |
| CN106789728A (en) * | 2017-01-25 | 2017-05-31 | 甘肃农业大学 | A kind of voip traffic real-time identification method based on NetFPGA |
| CN107465567A (en) * | 2017-06-29 | 2017-12-12 | 西安交大捷普网络科技有限公司 | A kind of data forwarding method of database fire wall |
| CN107465567B (en) * | 2017-06-29 | 2021-05-07 | 西安交大捷普网络科技有限公司 | Data forwarding method of database firewall |
| CN108667921A (en) * | 2018-04-27 | 2018-10-16 | 中国农业银行股份有限公司 | A kind of banking recommendation information generation method and system based on network bypass |
| CN108667921B (en) * | 2018-04-27 | 2021-12-14 | 中国农业银行股份有限公司 | Bank business recommendation information generation method and system based on network bypass |
| CN109600304A (en) * | 2018-12-21 | 2019-04-09 | 成都九洲电子信息系统股份有限公司 | Based on time wheel mail data reduction, threat detection and trend behavior analysis method |
| CN110502391A (en) * | 2019-07-12 | 2019-11-26 | 苏宁云计算有限公司 | The grasping means and system of SQL information in a kind of MySQL database |
| CN111240599A (en) * | 2020-01-17 | 2020-06-05 | 北京马赫谷科技有限公司 | Data stream storage method and device |
| CN111240599B (en) * | 2020-01-17 | 2021-06-25 | 北京马赫谷科技有限公司 | Data stream storage method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105337976A (en) | Real-time high-efficiency database audit realization method | |
| CN1574839B (en) | Multi-layered firewall architecture | |
| CN110753064A (en) | Machine learning and rule matching fused security detection system | |
| CN110401624A (en) | The detection method and system of source net G system mutual message exception | |
| CN110572412A (en) | Firewall based on intrusion detection system feedback in cloud environment and implementation method thereof | |
| US20170034195A1 (en) | Apparatus and method for detecting abnormal connection behavior based on analysis of network data | |
| CN105282169A (en) | DDoS attack warning method and system based on SDN controller threshold | |
| CN101321171A (en) | Method and apparatus for detecting distributed refusal service attack | |
| CN107302534A (en) | A kind of DDoS network attack detecting methods and device based on big data platform | |
| WO2009018737A1 (en) | Method and network device for preventing dos attacks | |
| CN112688932A (en) | Honeypot generation method, honeypot generation device, honeypot generation equipment and computer readable storage medium | |
| CN107690051A (en) | One kind alarm video recording method and device | |
| Ma et al. | A design of firewall based on feedback of intrusion detection system in cloud environment | |
| US20150331808A1 (en) | Packet capture deep packet inspection sensor | |
| CN105260378A (en) | Database audit method and device | |
| Chi | Intrusion detection system based on snort | |
| JP2002124996A (en) | Fast packet acquiring engine/security | |
| CN101582880B (en) | Method and system for filtering messages based on audited object | |
| CN1350231A (en) | By-pass investigation and remisson method for rejecting service attack | |
| CN102053970B (en) | Database auditing method and system | |
| JP2007249348A (en) | Data collection device and method in application trace-back and its program | |
| Hsiao et al. | High-throughput intrusion detection system with parallel pattern matching | |
| KR101384618B1 (en) | A system for analyzing dangerous situation using node analysis | |
| KR101615587B1 (en) | System for implementing Deep Packet Inspection Simulation for detecting and analyzing cyber attack in electronic warfare and Method thereof | |
| CN106657087B (en) | Method for realizing industrial firewall dynamically tracked by Ethernet/Ip protocol |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160217 |