CN106878029A - A kind of network data auditing system and method - Google Patents
A kind of network data auditing system and method Download PDFInfo
- Publication number
- CN106878029A CN106878029A CN201510924912.2A CN201510924912A CN106878029A CN 106878029 A CN106878029 A CN 106878029A CN 201510924912 A CN201510924912 A CN 201510924912A CN 106878029 A CN106878029 A CN 106878029A
- Authority
- CN
- China
- Prior art keywords
- audit
- data
- management module
- log
- report
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/14—Charging, metering or billing arrangements for data wireline or wireless communications
- H04L12/1403—Architecture for metering, charging or billing
- H04L12/1407—Policy-and-charging control [PCC] architecture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/14—Charging, metering or billing arrangements for data wireline or wireless communications
- H04L12/141—Indication of costs
- H04L12/1421—Indication of expected costs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0205—Traffic management, e.g. flow control or congestion control at the air interface
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/24—Accounting or billing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W84/00—Network topologies
- H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
- H04W84/10—Small scale networks; Flat hierarchical networks
- H04W84/12—WLAN [Wireless Local Area Networks]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Debugging And Monitoring (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a kind of network data auditing system, the network data that multiple AP equipment are analysed to respectively carries out shallow parsing and forms shallow parsing data；With deep analysis data be assigned in audit cluster unit shallow parsing data by two kinds of different transmission channels by MASTER administrative units；Audit cluster unit carries out drastic reduction analysis to the shallow parsing data and deep parsing data, and generates audit log and log information queue；Storage cluster unit is stored the audit log and log information queue；Report cluster unit to monitor the log information queue of the storage cluster unit, and after the description information for getting the audit log to be reported, the audit log is reported into the data center according to the form that reports of data center.Present invention also offers corresponding method.Network data auditing system of the invention and method realized by three levels, realize being separated on logically and physically between level and level, the characteristics of possess high-performance and enhanced scalability.
Description
Technical field
The present invention relates to technical field of data processing, more specifically to a kind of network data auditing system
And method.
Background technology
WiFi wireless networks have become many public place standard auxiliary facilities, and for public place, client provides
Free service on net.Tradition carries out data image mode audit based on single-point place, it is impossible to solve to examine very well
Meter daily record and the incidence relation of access terminals, high cost, deployment are complicated；In addition, original based on audit place
Packet is directly forwarded to audit server, although the initial data of the front end that can directly audit, but original
Packet increased legacy network bandwidth, have impact on normal online.
The content of the invention
The technical problem to be solved in the present invention is, for open place WLAN wireless networks of the prior art
The drawbacks described above of network Data Audit, there is provided a kind of network data auditing system and method.
Technical proposal that the invention solves the above-mentioned problems there is provided a kind of network data auditing system, the system
System includes：
Multiple AP equipment, the network data for being analysed to respectively carries out shallow parsing and forms shallow parsing data；
MASTER administrative units, are connected with the multiple AP equipment, for by it is described it is shallow parsing data and
Deep analysis data are assigned in audit cluster unit by two kinds of different transmission channels；
The audit cluster unit, is connected with the MASTER administrative units, for respectively to the shallow solution
Analysis data and the deep parsing data carry out drastic reduction analysis, and generate audit log and log information team
Row；
Storage cluster unit, is connected, with the audit cluster unit for the audit log and daily record to be disappeared
Breath queue is stored；
Cluster unit is reported, is connected with the storage cluster unit, for monitoring the storage cluster unit
Log information queue, and after the description information for getting the audit log to be reported, according in data
The audit log is reported to the data center by the form that reports of the heart.
In above-mentioned network data auditing system, the MASTER administrative units include load equalizer, described
Load equalizer is used to set up shallow parsing data transmission channel and deep parsing number respectively according to load-balancing algorithm
According to transmission channel.
In above-mentioned network data auditing system, the audit cluster unit includes multiple audit main frames, each institute
The nodal function for stating audit main frame is identical, and each described audit main frame includes shallow parsing data processing mould
Block and deep parsing data processing module, wherein the shallow parsing data processing module is used for the shallow parsing number
According to carrying out regression analysis, and generate audit log and log information queue；The deep parsing data processing module
For carrying out secondary parsing to the deep parsing data, and generate audit log and log information queue.
In above-mentioned network data auditing system, the storage cluster unit includes multiple memory databases, each
The memory database is identical, and each described memory database is used for the audit log and daily record
Message queue is stored；
It is described to report cluster unit to report main frame including multiple, it is the multiple report main frame respectively with the storage
The multiple memory database of cluster unit is corresponded.
In above-mentioned network data auditing system, MASTER administrative units also include audit cluster management module,
Storage cluster management module and cluster management module is reported, the audit main frame and the audit cluster management
Module is connected, and the memory database is connected with the storage cluster management module, described to report main frame and institute
State and report cluster management module to connect, wherein：The audit cluster management module is used to register audit main frame section
Point；The storage cluster management module is used to register memory database node；It is described to report cluster management module
The corresponding relation between node and the AP equipment and the data center is reported for registration data；
It is described audit cluster management module be additionally operable to perform expanded configuration operate cause it is described audit main frame according to
Business development situation carries out smooth extension；The storage cluster management module is additionally operable to perform expanded configuration operation
So that the memory database supports smooth extension.
Present invention also offers a kind of network data auditing method, network data auditing system is applied to, it is described
System include multiple AP equipment, MASTER administrative units, audit cluster unit, storage cluster unit with
And report cluster unit；Methods described includes：
The network data that S1, multiple AP equipment are analysed to respectively carries out shallow parsing and forms shallow parsing data；
The shallow parsing data and the deep analysis data are passed through two kinds by S2, MASTER administrative unit
Different transmission channels is assigned in audit cluster unit；
S3, audit cluster unit carry out drastic reduction to the shallow parsing data and the deep parsing data respectively
Analysis, and generate audit log and log information queue.
S4, storage cluster unit are stored the audit log and log information queue；
S5, report cluster unit to monitor the log information queue of the storage cluster unit, and treated getting
After the description information of the audit log for reporting, form is reported by the audit log according to data center
Be reported to the data center.
In above-mentioned network data auditing method, the step S2 also includes：Load equalizer is according to load balancing
Algorithm sets up shallow parsing data transmission channel and deep parsing data transmission channel respectively.
In above-mentioned network data auditing method, in the step S3, the audit cluster unit includes multiple
Audit main frame, the nodal function of each audit main frame is identical, and each described audit main frame includes
Shallow parsing data processing module and deep parsing data processing module, wherein the shallow parsing data processing module is used
In regression analysis is carried out to the shallow parsing data, and audit log is generated；The deep parsing data processing mould
Block is used to carry out secondary parsing to the deep parsing data, and generates audit log.
In above-mentioned network data auditing method, in the step S4, the storage cluster unit includes multiple
Memory database, each described memory database is identical, and each described memory database is for described
Audit log and log information queue are stored；
It is described to report cluster unit to report main frame including multiple in the step S5, it is the multiple to report master
Machine is corresponded with the multiple memory database of the storage cluster unit respectively.
In above-mentioned network data auditing method, the MASTER administrative units also include audit cluster management mould
Block, storage cluster management module and cluster management module is reported, the audit main frame and the audit cluster
Management module is connected, and the memory database is connected with the storage cluster management module, described to report main frame
Cluster management module is reported to be connected with described, wherein：The audit cluster management module is used to register audit master
Machine node；The storage cluster management module is used to register memory database node；It is described to report cluster management
Module reports the corresponding pass between node and the AP equipment and the data center for registration data
System；Methods described also includes：
The audit cluster management module performs expanded configuration and operates so that audit main frame carries out feelings according to business
Condition carries out smooth extension；
The storage cluster management module performs expanded configuration and operates so that memory database supports smooth extension.
The beneficial effect of network data auditing system of the invention and method has：
First, the transmission of basic user behaviors log data is carried out by shallow parsing data transmission channel, and by depth
Degree parsing data transmission channel realizes the transmission of the data for needing secondary parsing, and transmission means substantially optimizes, bandwidth
Take small, it is ensured that effectiveness of audit, also reach the purpose for saving bandwidth；
Second, multiple memory databases of storage cluster administrative unit are physically separate, are being patrolled
A memory database storage cluster is constituted on volume, unified memory database production and consumption is externally provided
Interface, so after Audit data by that after the treatment of main frame of auditing this unified memory database can be called to produce
With consumption interface, now, total data is that read-write operation is completed in internal memory, and read-write efficiency is high；
3rd, audit main frame can dynamically realize smooth extension according to business development situation, in MASTER
Corresponding expanded configuration operation is completed in administrative unit, meanwhile, support the smooth extension of memory database, phase
Ying Di, completes corresponding expanded configuration operation in MASTER administrative units；
4th, report main frame to monitor the log information queue in each self-corresponding memory database, by daily record
Message queue obtains the Audit data for needing to report, and it is each other independent, Bu Huixiang that each reports main frame
Mutually influence, it is ensured that stability that data are reported and efficiently.
Brief description of the drawings
Fig. 1 is the structural representation of network data auditing system embodiment of the invention.
Fig. 2 is the flow chart of network data auditing method embodiment of the invention.
Specific embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, below in conjunction with accompanying drawing and reality
Example is applied, the present invention will be described in further detail.It should be appreciated that specific embodiment described herein is only
It is used to explain the present invention, is not intended to limit the present invention.
As shown in figure 1, being the structural representation of network data auditing system embodiment of the invention.With reference to Fig. 1,
The system includes multiple AP equipment, MASTER administrative units, audit cluster unit, storage cluster unit
And cluster unit is reported, wherein, multiple AP equipment are connected with MASTER administrative units respectively, audit
Cluster unit is connected with MASTER administrative units, and storage cluster unit is connected with audit cluster unit.
The network data that multiple AP equipment are analysed to respectively carries out shallow parsing and forms shallow parsing data, wherein,
Shallow parsing data are the basic user behaviors log of AP equipment, and deep analysis data are secondary parsing text to be resolved
Part.The shallow parsing data and deep analysis data are transferred to MASTER administrative units by AP equipment afterwards.
MASTER administrative units are used to for shallow parsing data to pass through two kinds of different biographies with deep analysis data
Defeated channel allocation to audit cluster unit on.Wherein, MASTER administrative units include load equalizer, should
Load equalizer is used to set up shallow parsing data transmission channel and deep parsing number respectively according to load-balancing algorithm
According to transmission channel.
Audit cluster unit is used to carry out drastic reduction analysis to shallow parsing data and deep parsing data, and generates
Audit log.Specifically, the audit cluster unit includes multiple audit main frames, the node of each audit main frame
Function is identical, and each audit main frame is used to carry out shallow parsing data regression analysis, and to deep parsing number
According to carrying out secondary parsing, and generate audit log.In the present embodiment, MASTER administrative units are by shallow solution
Data and deep analysis data distribution are analysed to audit main frame.Further, each audit main frame includes shallow solution
Analysis data processing module and deep parsing data processing module, shallow parsing data processing module are used for shallow parsing number
According to carrying out regression analysis, and audit log and log information queue are generated, deep parsing data processing module difference
For carrying out secondary parsing to deep parsing data, and generate audit log and log information queue.For example,
Shallow parsing data processing module is first decompressed to shallow parsing data；Then, being extracted according to predetermined format should
The corresponding audit field of predetermined format；Finally, the data center that the Audit data according to AP equipment need to be reported
List, audit field is written in audit log successively, and then generate audit log.
Storage cluster unit is used to be stored audit log and log information queue, specifically, the storage
Cluster unit includes multiple memory databases, and each memory database is identical, the plurality of memory database
Unified data production and data consumption interface is externally provided, each memory database be used for audit log and
Log information queue is stored, wherein, the storage location of audit log uses hash algorithm, therefore, should
Multiple memory databases are physically separate, logically constitute an internal storage data library storage
Cluster.The expression formula of the hash algorithm is：(AP [MAC]+data report centre code+daily record to Hash
Type)；Comprising List, (AP [MAC]+data report centre code+daily record class to the element of log information queue
Type | daily record ID).The storage cluster unit also includes virtual memory database access interface, the virtual memory number
Be used to providing a listening port according to storehouse access interface, and safeguard with the actual connection of multiple memory databases and
Heartbeat detection.
Cluster unit is reported for monitoring storage cluster unit, and is getting retouching for audit log to be reported
After stating information, audit log is reported into data center according to the form that reports of data center.Specifically, on
Report cluster unit reports main frame including multiple, multiple report main frame respectively with multiple internal memories of storage cluster unit
Database is corresponded.In the present embodiment, by independently separating to obtain description information and memory database
In audit log, it is to avoid audit log is directly transmitted in description information, the transmission of description information is improve
Efficiency, meanwhile, main frame is reported by multiple, individual data center can realize the multi-process of audit log, many
The parallel of thread reports, and, it is independent parallel that each reports main frame, is not influenceed each other, is protected
The stability that reports of card data and efficiently.
In an embodiment of the present invention, audit and reports cluster unit also at cluster unit, storage cluster unit
It is connected with MASTER administrative units respectively.MASTER administrative units be additionally operable to registration audit host node,
Memory database node, data report the corresponding relation between node and AP equipment and data center.Enter
One step ground, audit main frame can be according to the smooth extension of business development situation dynamic implement, accordingly, MASTER
Administrative unit is additionally operable to perform expanded configuration operation so that audit main frame is smoothed according to business development situation
Extension.MASTER administrative units are additionally operable to perform expanded configuration operation so that memory database supports smooth expansion
Exhibition.Specifically, MASTER administrative units also include audit cluster management module, storage cluster management module
And report cluster management module, audit main frame is connected with cluster management module of auditing, memory database with deposit
Accumulation management module is connected, and reports main frame to be connected with cluster management module is reported, wherein, cluster pipe of auditing
Reason module is used to register audit host node, and storage cluster management module is used to register memory database node,
Cluster management module is reported to be reported for registration data corresponding between node and AP equipment and data center
Relation.Audit cluster management module is additionally operable to perform expanded configuration operation so that audit main frame is carried out according to business
Situation carries out smooth extension；Storage cluster management module is additionally operable to perform expanded configuration operation so that internal storage data
Support smooth extension in storehouse.
As shown in Fig. 2 being the flow chart of network data auditing method embodiment of the invention, the network data
Auditing method is applied in above-mentioned network data auditing system.With reference to Fig. 2, the method is comprised the following steps：
The network data that S10, multiple AP equipment are analysed to respectively carries out shallow parsing and forms shallow parsing data；
The shallow parsing data and the deep analysis data are passed through two kinds by S20, MASTER administrative unit
Different transmission channels is assigned in audit cluster unit；
S30, audit cluster unit carry out depth also to the shallow parsing data and the deep parsing data respectively
Original analysis, and generate audit log and log information queue.
S40, storage cluster unit are stored the audit day and log information queue will；
S50, report cluster unit to monitor the log information queue of the storage cluster unit, and getting
After the description information of the audit log to be reported, form is reported by the audit day according to data center
Will is reported to the data center.
In above-mentioned steps S20, MASTER administrative units include load equalizer, and the load equalizer is used
In shallow parsing data transmission channel is set up respectively according to load-balancing algorithm data transmission channel is parsed with deep.
In above-mentioned steps S30, the audit cluster unit includes multiple audit main frames, each audit main frame
Nodal function is identical, and each audit main frame is used to carry out regression analysis to shallow parsing data, and to depth
Parsing data carry out secondary parsing, and generate audit log and log information queue respectively.In the present embodiment,
MASTER administrative units are by shallow data and the deep analysis data distribution of parsing to audit main frame.Further
Ground, each audit main frame includes shallow parsing data processing module and deep parsing data processing module, shallow parsing number
It is used to carry out regression analysis to shallow parsing data according to processing module, and generates audit log and log information team
Row, deep parsing data processing module is respectively used to carry out secondary parsing to deep parsing data, and generates audit day
Will and log information queue.For example, shallow parsing data processing module is first decompressed to shallow parsing data
Contracting；Then, the corresponding audit field of the predetermined format is extracted according to predetermined format；Finally, set according to AP
Data center's list that standby Audit data need to be reported, audit field is written in audit log successively, is entered
And generate audit log.
In above-mentioned steps S40, the storage cluster unit includes multiple memory databases, each internal storage data
Storehouse is identical, and the plurality of memory database externally provides unified data production and data consumption interface, often
Individual memory database is used to be stored audit log and log information queue, wherein, audit log is deposited
Storage space is put and uses hash algorithm, therefore, the plurality of memory database is physically separate, is being patrolled
A memory database storage cluster is constituted on volume.The expression formula of the hash algorithm is：Hash(AP[MAC]
+ data report centre code+Log Types)；The element of log information queue comprising List (AP [MAC]+
Data report centre code+Log Types | daily record ID).The storage cluster unit also includes virtual memory data
Storehouse access interface, the virtual memory database access interface be used for provide a listening port, and safeguard with it is many
The actual connection of individual memory database and heartbeat detection.
In above-mentioned steps S50, cluster unit is reported to report main frame including multiple, multiple reports main frame to distinguish
Multiple memory databases with storage cluster unit are corresponded.In the present embodiment, by independently separating
Obtain the audit log in description information and memory database, it is to avoid directly transmission is audited day in description information
Will, improves the transmission efficiency of description information, meanwhile, main frame is reported by multiple, individual data center can
Realize that the multi-process of audit log, the parallel of multithreading are reported, and, each report main frame be it is independent simultaneously
OK, do not influence each other, it is ensured that stability that data are reported and efficiently.
In an embodiment of the present invention, MASTER administrative units are additionally operable to registration audit host node, interior
Deposit memory node, data and report corresponding relation between node and AP equipment and data center.Further
Ground, audit main frame can be according to the smooth extension of business development situation dynamic implement, accordingly, MASTER
Administrative unit is additionally operable to perform expanded configuration operation so that audit main frame is smoothed according to business development situation
Extension.MASTER administrative units are additionally operable to perform expanded configuration operation so that memory database supports smooth expansion
Exhibition.Specifically, MASTER administrative units also include audit cluster management module, storage cluster management module
And report cluster management module, audit main frame is connected with cluster management module of auditing, memory database with deposit
Accumulation management module is connected, and reports main frame to be connected with cluster management module is reported, wherein, cluster pipe of auditing
Reason module is used to register audit host node, and storage cluster management module is used to register memory database node,
Cluster management module is reported to be reported for registration data corresponding between node and AP equipment and data center
Relation.Further, the method also includes：
Audit cluster management module performs expanded configuration and operates so that audit main frame enters according to business development situation
The smooth extension of row；
Storage cluster management module performs expanded configuration and operates so that memory database supports smooth extension.
Summary, network data auditing system of the invention and method are realized by three levels, respectively examined
Count, store and report, realize being separated on logically and physically between level and level, possess high-performance and Gao Ke
The characteristics of autgmentability, specific beneficial effect has：
First, the transmission of basic user behaviors log data is carried out by shallow parsing data transmission channel, and by depth
Degree parsing data transmission channel realizes the transmission of the data for needing secondary parsing, and transmission means substantially optimizes, bandwidth
Take small, it is ensured that effectiveness of audit, also reach the purpose for saving bandwidth；
Second, multiple memory databases of storage cluster administrative unit are physically separate, are being patrolled
A memory database storage cluster is constituted on volume, unified memory database production and consumption is externally provided
Interface, so after Audit data by that after the treatment of main frame of auditing this unified memory database can be called to produce
With consumption interface, now, total data is that read-write operation is completed in internal memory, and read-write efficiency is high；
3rd, audit main frame can dynamically realize smooth extension according to business development situation, in MASTER
Corresponding expanded configuration operation is completed in administrative unit, meanwhile, support the smooth extension of memory database, phase
Ying Di, completes corresponding expanded configuration operation in MASTER administrative units；
4th, report main frame to monitor the log information queue in each self-corresponding memory database, by daily record
Message queue obtains the Audit data for needing to report, and it is each other independent, Bu Huixiang that each reports main frame
Mutually influence, it is ensured that stability that data are reported and efficiently.
The above, the only present invention preferably specific embodiment, but protection scope of the present invention not office
Be limited to this, any one skilled in the art the invention discloses technical scope in, can be easily
The change or replacement expected, should all be included within the scope of the present invention.Therefore, protection of the invention
Scope should be defined by scope of the claims.
Claims (10)
1. a kind of network data auditing system, it is characterised in that the system includes：
Multiple AP equipment, the network data for being analysed to respectively carries out shallow parsing and forms shallow parsing data；
MASTER administrative units, are connected with the multiple AP equipment, for by it is described it is shallow parsing data and
Deep analysis data are assigned in audit cluster unit by two kinds of different transmission channels；
The audit cluster unit, is connected with the MASTER administrative units, for respectively to the shallow solution
Analysis data and the deep parsing data carry out drastic reduction analysis, and generate audit log and log information team
Row；
Storage cluster unit, is connected, with the audit cluster unit for the audit log and daily record to be disappeared
Breath queue is stored；
Cluster unit is reported, is connected with the storage cluster unit, for monitoring the storage cluster unit
Log information queue, and after the description information for getting the audit log to be reported, according in data
The audit log is reported to the data center by the form that reports of the heart.
2. the network data auditing system according to claim 1, it is characterised in that the MASTER
Administrative unit includes load equalizer, and the load equalizer is used to set up shallow respectively according to load-balancing algorithm
Parsing data transmission channel and deep parsing data transmission channel.
3. the network data auditing system according to claim 1, it is characterised in that the audit collection
Group unit includes multiple audit main frames, and the nodal function of each audit main frame is identical, and each institute
Stating audit main frame includes shallow parsing data processing module and deep parsing data processing module, wherein the shallow parsing
Data processing module is used to carry out regression analysis to the shallow parsing data, and generates audit log and daily record disappears
Breath queue；The deep parsing data processing module is used to carry out secondary parsing to the deep parsing data, and raw
Into audit log and log information queue.
4. the network data auditing system according to claim 1, it is characterised in that the storage collection
Group unit includes multiple memory databases, and each described memory database is identical, and each described internal memory
Database is used to be stored the audit log and log information queue；
It is described to report cluster unit to report main frame including multiple, it is the multiple report main frame respectively with the storage
The multiple memory database of cluster unit is corresponded.
5. the network data auditing system according to any one of claim 2-4, it is characterised in that
MASTER administrative units also include audit cluster management module, storage cluster management module and report cluster
Management module, the audit main frame is connected with the audit cluster management module, the memory database and institute
The connection of storage cluster management module is stated, it is described to report main frame to report cluster management module to be connected with described, wherein：
The audit cluster management module is used to register audit host node；The storage cluster management module is used to step on
Note memory database node；It is described to report cluster management module to report node and described for registration data
Corresponding relation between AP equipment and the data center；
It is described audit cluster management module be additionally operable to perform expanded configuration operate cause it is described audit main frame according to
Business development situation carries out smooth extension；The storage cluster management module is additionally operable to perform expanded configuration operation
So that the memory database supports smooth extension.
6. a kind of network data auditing method, is applied to network data auditing system, it is characterised in that institute
Stating system includes multiple AP equipment, MASTER administrative units, audit cluster unit, storage cluster unit
And report cluster unit；Methods described includes：
The network data that S1, multiple AP equipment are analysed to respectively carries out shallow parsing and forms shallow parsing data；
The shallow parsing data and the deep analysis data are passed through two kinds by S2, MASTER administrative unit
Different transmission channels is assigned in audit cluster unit；
S3, audit cluster unit carry out drastic reduction to the shallow parsing data and the deep parsing data respectively
Analysis, and generate audit log and log information queue.
S4, storage cluster unit are stored the audit log and log information queue；
S5, report cluster unit to monitor the log information queue of the storage cluster unit, and treated getting
After the description information of the audit log for reporting, form is reported by the audit log according to data center
Be reported to the data center.
7. the network data auditing method according to any one of claim 6, it is characterised in that described
Step S2 also includes：Load equalizer set up respectively according to load-balancing algorithm it is shallow parsing data transmission channel and
Deep parsing data transmission channel.
8. the network data auditing method according to claim 1, it is characterised in that in the step
In S3, the audit cluster unit includes multiple audit main frames, and the nodal function of each audit main frame is complete
It is exactly the same, and each described audit main frame includes shallow parsing data processing module and deep parsing data processing mould
Block, wherein the shallow parsing data processing module is used to carry out regression analysis to the shallow parsing data, and it is raw
Into audit log；The deep parsing data processing module is used to carry out secondary parsing to the deep parsing data,
And generate audit log.
9. the network data auditing method according to claim 6, it is characterised in that in the step
In S4, the storage cluster unit includes multiple memory databases, and each described memory database is identical,
And each described memory database is used for the audit log and log information queue is stored；
It is described to report cluster unit to report main frame including multiple in the step S5, it is the multiple to report master
Machine is corresponded with the multiple memory database of the storage cluster unit respectively.
10. the network data auditing method according to any one of claim 6-9, it is characterised in that
The MASTER administrative units also include audit cluster management module, storage cluster management module and report
Cluster management module, the audit main frame is connected with the audit cluster management module, the memory database
It is connected with the storage cluster management module, it is described to report main frame to report cluster management module to be connected with described,
Wherein：The audit cluster management module is used to register audit host node；The storage cluster management module
For registering memory database node；It is described report cluster management module for registration data report node and
Corresponding relation between the AP equipment and the data center；Methods described also includes：
The audit cluster management module performs expanded configuration and operates so that audit main frame carries out feelings according to business
Condition carries out smooth extension；
The storage cluster management module performs expanded configuration and operates so that memory database supports smooth extension.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510924912.2A CN106878029B (en) | 2015-12-14 | 2015-12-14 | A kind of network data auditing system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510924912.2A CN106878029B (en) | 2015-12-14 | 2015-12-14 | A kind of network data auditing system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106878029A true CN106878029A (en) | 2017-06-20 |
| CN106878029B CN106878029B (en) | 2019-11-22 |
Family
ID=59178335
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510924912.2A Active CN106878029B (en) | 2015-12-14 | 2015-12-14 | A kind of network data auditing system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106878029B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609114A (en) * | 2017-09-13 | 2018-01-19 | 广东欧珀移动通信有限公司 | Log information report method, device and storage medium, ADSP and terminal |
| CN110719282A (en) * | 2019-10-10 | 2020-01-21 | 国网山东省电力公司信息通信公司 | Authentication dual-active system based on unified authority |
| CN113342849A (en) * | 2021-05-28 | 2021-09-03 | 百果园技术(新加坡)有限公司 | Data auditing method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1996876A (en) * | 2006-06-30 | 2007-07-11 | 深圳市中科新业信息科技发展有限公司 | Distributed audit system |
| CN103152352A (en) * | 2013-03-15 | 2013-06-12 | 北京邮电大学 | Perfect information security and forensics monitoring method and system based on cloud computing environment |
| JP2013238957A (en) * | 2012-05-14 | 2013-11-28 | Hitachi Solutions Ltd | Auditing result analysis system and auditing result analysis method |
| CN103618652A (en) * | 2013-12-17 | 2014-03-05 | 沈阳觉醒软件有限公司 | Audit and depth analysis system and audit and depth analysis method of business data |
-
2015
- 2015-12-14 CN CN201510924912.2A patent/CN106878029B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1996876A (en) * | 2006-06-30 | 2007-07-11 | 深圳市中科新业信息科技发展有限公司 | Distributed audit system |
| JP2013238957A (en) * | 2012-05-14 | 2013-11-28 | Hitachi Solutions Ltd | Auditing result analysis system and auditing result analysis method |
| CN103152352A (en) * | 2013-03-15 | 2013-06-12 | 北京邮电大学 | Perfect information security and forensics monitoring method and system based on cloud computing environment |
| CN103618652A (en) * | 2013-12-17 | 2014-03-05 | 沈阳觉醒软件有限公司 | Audit and depth analysis system and audit and depth analysis method of business data |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609114A (en) * | 2017-09-13 | 2018-01-19 | 广东欧珀移动通信有限公司 | Log information report method, device and storage medium, ADSP and terminal |
| CN110719282A (en) * | 2019-10-10 | 2020-01-21 | 国网山东省电力公司信息通信公司 | Authentication dual-active system based on unified authority |
| CN110719282B (en) * | 2019-10-10 | 2021-10-29 | 国网山东省电力公司信息通信公司 | Authentication dual-active system based on unified authority |
| CN113342849A (en) * | 2021-05-28 | 2021-09-03 | 百果园技术(新加坡)有限公司 | Data auditing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106878029B (en) | 2019-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Meng et al. | Improving the scalability of data center networks with traffic-aware virtual machine placement | |
| CN104462121B (en) | Data processing method, apparatus and system | |
| KR102082355B1 (en) | Processing Techniques for Large Network Data | |
| CN103560967B (en) | The virtual data center mapping method of a kind of business demand perception | |
| CN104216893B (en) | Partition management method, server and the system of multi-tenant shared data table | |
| CN104620539B (en) | System and method for supporting SNMP requests by cluster | |
| CN104618521B (en) | For carrying out the method, apparatus and computer-readable medium of node duplicate removal in a network | |
| CN106559488A (en) | A kind of method of the electrical network geographical information space service for setting up tenant's driving | |
| CN105897827A (en) | Server node, local area network server cluster and realizing method thereof | |
| CN103118130A (en) | Cluster management method and cluster management system for distributed service | |
| US20220200902A1 (en) | Method, apparatus and storage medium for application identification | |
| US20230283556A1 (en) | Collecting, processing, and distributing telemetry data | |
| CN106878029A (en) | A kind of network data auditing system and method | |
| WO2018169876A1 (en) | Systems and methods for compute node management protocols | |
| CN104486116A (en) | Multidimensional query method and multidimensional query system of flow data | |
| US11133990B2 (en) | System and method for providing a dynamic comparative network health analysis of a network environment | |
| WO2014110239A1 (en) | Methods, systems, and computer program products for distributed packet traffic performance analysis in a communication network | |
| CN105871957A (en) | Monitoring framework design method, monitoring server, proxy unit and center control server | |
| CN105681199B (en) | The processing method and processing device of message data in a kind of vehicle bus | |
| CN105718299A (en) | Virtual machine configuration method, device and system | |
| CN107766207A (en) | Distributed automatic monitoring method, system, computer-readable recording medium and terminal device | |
| DE102017213160A1 (en) | Compilation for node device GPU-based parallel processing | |
| Pearce et al. | One quadrillion triangles queried on one million processors | |
| CN101304342B (en) | Enhancement type Ethernet interface apparatus | |
| WO2020124488A1 (en) | Application process mapping method, electronic device, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |