CN109165363A - A kind of configuration method of network data snapshot - Google Patents
A kind of configuration method of network data snapshot Download PDFInfo
- Publication number
- CN109165363A CN109165363A CN201810982000.4A CN201810982000A CN109165363A CN 109165363 A CN109165363 A CN 109165363A CN 201810982000 A CN201810982000 A CN 201810982000A CN 109165363 A CN109165363 A CN 109165363A
- Authority
- CN
- China
- Prior art keywords
- snapshot
- data
- configuration
- rule
- sub
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a kind of configuration methods of network data snapshot, including specified by user or system acquisition obtains external network data, and create default snapshot node, choose snapshot node, and generate be not configured accordingly increase sub- snapshot newly, the sub- snapshot that increases newly being not configured is configured according to configuration rule, obtains configured sub- snapshot；Circulation obtains different sub- snapshots；Configured sub- snapshot is run, into wait state, to be serviced section automatically turns on operating status when having vacant resource, carries out data extraction task, and form snapshot tree.The present invention passes through specific configuration condition, accurately data are extracted in specified snapshot generates sub- snapshot, and each snapshot is shown by hierarchical relationship when adding, original big data can be made to be split into the data snapshot of multiple specific types and condition, disperse data storage pressure, while also reducing the final result quantities presented in front of the user.
Description
Technical field
The present invention relates to data processing method more particularly to a kind of configuration methods of network data snapshot.
Background technique
The development of internet for network data analysis industry this be opportunity be also challenge, in face of at geometry multiplication by stages
Data volume, the transmission speed being getting faster, the features such as data age is more and more stronger, existing processing and analysis mode are very
Hardly possible fast and accurately obtains usable results, so urgently scheme that is a kind of completely new and can solve the above pain spot is born in the industry.
Summary of the invention
The object of the invention is that providing a kind of configuration method of network data snapshot to solve the above-mentioned problems.
The present invention through the following technical solutions to achieve the above objectives:
A kind of configuration method of network data snapshot, comprising the following steps:
S1, specified by user or system acquisition obtains external network data, and creates default snapshot node, snapshot node
It include: session snapshot, DNS snapshot, HTTP snapshot, mailbox log snapshot, FTP log snapshot and SSL certificate snapshot；
S2, choose snapshot node, and generate be not configured accordingly increase sub- snapshot newly, according to configuration rule to being not configured
It increases sub- snapshot newly to be configured, obtains configured sub- snapshot；
S3, circulation step S2 obtain different sub- snapshots according to different snapshot nodes and different parameter configurations；
The configured sub- snapshot of S4, operation, into wait state, to be serviced section automatically turns on operation shape when having vacant resource
State carries out data extraction task；
If emerged in operation mistake, the sub- snapshot is just placed in error condition, operation finishes if normal, just by the sub- snapshot
It is placed in completion status, the snapshot of completion status can be checked；
S5, circulation step S2, S3, S4 form snapshot tree.
Specifically, in above-mentioned steps S1:
The session snapshot is to extract all data packets；
The DNS snapshot is to extract all data packets relevant to DNS request；
The HTTP snapshot is to extract all data packets relevant to HTTP request；
The mailbox log snapshot is to extract mailbox daily record data packet；
The FTP log snapshot is to extract FTP daily record data packet；
The SSL certificate snapshot is to extract SSL certificate data packet；
Specifically, the configuration rule in above-mentioned steps S2 includes:
The configuration rule of the session snapshot includes: base rule, high level rules, Exception Model；
The configuration rule of the DNS snapshot includes: base rule, high level rules, Exception Model；
The configuration rule of the HTTP snapshot includes: base rule, high level rules, Exception Model；
The configuration rule of the mailbox log snapshot includes: base rule, high level rules；
The configuration rule of the FTP log snapshot includes: base rule, high level rules；
The configuration rule of the SSL certificate snapshot includes: base rule, high level rules；
The base rule includes: countries and regions, data session size, agreement；
The high level rules extract expression formula by the data that user writes and complete configuration；
The Exception Model includes: DDOS attack, and worm attack, Trojan characteristics, heartbeat is abnormal, and connection is abnormal, and behavior is different
Often, Traffic Anomaly, multiplexed port.
Further, the snapshot of completion, the number extracted after the operation that can check snapshot are run in above-mentioned steps S4
According to.
Further, above-mentioned all snapshot nodes, which will record, checks the operation footprint of data after user opens snapshot and divides
Analyse footprint.
It further, further include curing data and reduction data in above-mentioned steps S4；
When the curing data is included in snapshot operation, it can be taken from father's snapshot according to the rule that recent snapshot configures from the background
Specific data out, and the data of taking-up are recorded in hard disk, realize data write；
When the reduction data are included in snapshot operation, it can be taken from father's snapshot according to the rule that recent snapshot configures from the background
Specific data and remaining data are abandoned out, realizes data reduction.
Preferably, the snapshot tree in above-mentioned steps S5 is saved as into snapshot template, when obtaining new external network data,
New snapshot tree can be directly produced according to snapshot template.
The beneficial effects of the present invention are:
A kind of configuration method of network data snapshot of the present invention provides configuration and the hierarchical of a kind of network data snapshot
The method of display extracts accurately data in specified snapshot and generates sub- snapshot, and is each by specific configuration condition
Hierarchical relationship when snapshot is by addition is shown, original big data can be made to be split into the number of multiple specific types and condition
According to snapshot, multiple data snapshot synchronization process are improved out the speed of result by background service, have dispersed data storage pressure,
The final result quantities presented in front of the user are also reduced simultaneously, so that user can faster find the number for wanting to see with more acurrate
According to.
Detailed description of the invention
Fig. 1 is a kind of flow chart of the configuration method of network data snapshot of the present invention；
Fig. 2 is the schematic diagram of snapshot tree of the present invention.
Specific embodiment
The present invention will be further explained below with reference to the attached drawings:
As shown in Figure 1, a kind of configuration method of network data snapshot of the present invention, comprising the following steps:
For the operation convenient for user, this method is configured with operation interface.
S1, specified by user or system acquisition obtains external network data, and creates default snapshot node, snapshot node
It include: session snapshot, DNS snapshot, HTTP snapshot, mailbox log snapshot, FTP log snapshot and SSL certificate snapshot；Any snapshot
Node has 5 status functions, increases newly, waiting, operation, mistake, completes 5 states, wherein newly-increased be divided into two sub- states again,
It is newly-increased (being not configured), newly-increased (configured) respectively.
Session snapshot is to extract all data packets；
DNS snapshot is to extract all data packets relevant to DNS request；
HTTP snapshot is to extract all data packets relevant to HTTP request；
Mailbox log snapshot is to extract mailbox daily record data packet；
FTP log snapshot is to extract FTP daily record data packet；
SSL certificate snapshot is to extract SSL certificate data packet；
S2, snapshot node is chosen, a sub- snapshot will be generated simultaneously below snapshot choosing by clicking addition snapshot button
What generation was not configured accordingly increases sub- snapshot newly, configures according to configuration rule to the sub- snapshot that increases newly being not configured, and obtains
The sub- snapshot of configuration；When the user clicks when newly-increased button in any snapshot, the sub- snapshot of generation is in newly-increased (being not configured) shape
State, click, which increases the configuration button of snapshot newly and completes to match to postpone to be in, increases (configured) newly；
The configuration rule of session snapshot, DNS snapshot and HTTP snapshot includes: base rule, high level rules, abnormal mould
Type；
The configuration rule of mailbox log snapshot, FTP log snapshot and SSL certificate snapshot includes: base rule, advanced rule
Then；
Configuration snapshot button to be clicked in operation interface, pops up the configuration snapshot page, this page is divided into three subpage frames again,
It is respectively: base rule, high level rules, Exception Model.The configuration of these three pages is mutual exclusion, and end user clicks confirmation
As last configuration subject to the subpage frame of Shi Xuanzhong.A base rule option is presented in the base rule page, and user can be with
Optional configuration is therein one or more.The high level rules page is an input frame, user can using Freely input expression formula as
Configuration condition.The Exception Model page is an available model list, user can optionally one as condition.
Base rule include: countries and regions (China, Japan, the U.S., Canada etc.), data session size (be less than 1K,
1K-1M, 1M-20M etc.), agreement (ARP, ICMP, SMB, DHCP, DNS, HTTP, FTP etc.)；
High level rules extract expression formula by the data that user writes and complete configuration, and expression formula can have IP, IP with project
Section, MAC, port, time etc.；
Exception Model includes: DDOS attack, and worm attack, Trojan characteristics, heartbeat is abnormal, and connection is abnormal, abnormal behavior, stream
Amount is abnormal, multiplexed port.
Base rule, high level rules, Exception Model three can only select one to configure, and click the configuration button of new snapshot,
It will pop up the configuration page of snapshot, select arbitrary disposition project and confirm the parameter configuration for completing snapshot.Finally to click really
Subject to the configuration item page stopped when recognizing.
S3, circulation step S2 obtain different sub- snapshots according to different snapshot nodes and different parameter configurations,
Arbitrary snapshot node is chosen, the new snapshot button on node is clicked, system can will show one choosing below snapshot
A new snapshot, the new snapshot are connected with the line for having arrow by one between snapshot is chosen, and arrow is directed toward new snapshot,
And different sub- snapshots is obtained by different parameter configurations.
S4, it is waited for after clicking the operation button of snapshot, runs configured sub- snapshot, into wait state,
To be serviced section automatically turns on operating status when having vacant resource, carry out data extraction task；If emerged in operation mistake, just should
Sub- snapshot is placed in error condition, and operation finishes if normal, the sub- snapshot is just placed in completion status, the snapshot of completion status can be looked into
It sees；
User can double-click the snapshot node in operation completion status, and at this time program can be switched to the exhibition of snapshot data
Show the page, show the data of snapshot, mainly contains data session detail, data packet detail.
S5, circulation step S2, S3, S4, form snapshot tree as shown in Figure 2.
Any snapshot node all can record and go back the operation footprint function of original subscriber, and user checks and analysis snapshot contents
When, it will record the analysis footprint (search condition) of each step of user, facilitate user's rolling back action, when user closes the snapshot opened
When, it will record the state (search condition) of present analysis, page layout (size and location of each piece of content display region), choosing
In session row, analysis scene when closing can be completely reverted to when opening the snapshot again next time further,
In snapshot operation, specific data can be taken out according to the rule of recent snapshot configuration from father's snapshot from the background, and
The data of taking-up are recorded in hard disk, realize data write, and improve the speed of user search snapshot contents.
In snapshot operation, specific data can be taken out according to the rule of recent snapshot configuration from father's snapshot from the background and thrown
Remaining data are abandoned, realize data reduction, and improve the speed of user search snapshot contents.
Any non-default snapshot node is chosen, the deletion snapshot button on node is clicked, popping up selection after prompting is, i.e.,
It will be deleted selected snapshot.
All snapshots all can be with isolated operation or configuration batch operation after the completion, when the state of snapshot (is defaulted fast to be newly-built
According to), configured completion (other snapshots in addition to default snapshot) when, the operation button on snapshot node can be clicked, start to hold
The data acquisition of row snapshot, can start all states with key batch is to create (default snapshot), configured completion (except default snapshot
Other outer snapshots) snapshot.
Snapshot template can be saved as by configuring the snapshot tree finished, after all snapshots all configure, can click snapshot
Tree saves as button, and recent snapshot tree is saved as template, when getting new external data source, can click selection template
Button pops up the stencil-chosen page, arbitrarily selects template item therein that can directly generate snapshot tree, saves deduplication configuration
Time and manpower consumption.
The limitation that technical solution of the present invention is not limited to the above specific embodiments, it is all to do according to the technique and scheme of the present invention
Technology deformation out, falls within the scope of protection of the present invention.
Claims (7)
1. a kind of configuration method of network data snapshot, it is characterised in that: the following steps are included:
S1, specified by user or system acquisition obtains external network data, and creates default snapshot node, snapshot node packet
It includes: session snapshot, DNS snapshot, HTTP snapshot, mailbox log snapshot, FTP log snapshot and SSL certificate snapshot；
S2, choose snapshot node, and generate be not configured accordingly increase sub- snapshot newly, it is newly-increased to what is be not configured according to configuration rule
Sub- snapshot is configured, and configured sub- snapshot is obtained；
S3, circulation step S2 obtain different sub- snapshots according to different snapshot nodes and different parameter configurations；
The configured sub- snapshot of S4, operation, into wait state, to be serviced section automatically turns on operating status when having vacant resource,
Carry out data extraction task；
If emerged in operation mistake, the sub- snapshot is just placed in error condition, operation finishes if normal, is just placed in the sub- snapshot
The snapshot of completion status, completion status can be checked；
S5, circulation step S2, S3, S4 form snapshot tree.
2. a kind of configuration method of network data snapshot according to claim 1, it is characterised in that: in above-mentioned steps S1:
The session snapshot is to extract all data packets；
The DNS snapshot is to extract all data packets relevant to DNS request；
The HTTP snapshot is to extract all data packets relevant to HTTP request；
The mailbox log snapshot is to extract mailbox daily record data packet；
The FTP log snapshot is to extract FTP daily record data packet；
The SSL certificate snapshot is to extract SSL certificate data packet.
3. a kind of configuration method of network data snapshot according to claim 1, it is characterised in that: in above-mentioned steps S2
Configuration rule includes:
The configuration rule of the session snapshot includes: base rule, high level rules, Exception Model；
The configuration rule of the DNS snapshot includes: base rule, high level rules, Exception Model；
The configuration rule of the HTTP snapshot includes: base rule, high level rules, Exception Model；
The configuration rule of the mailbox log snapshot includes: base rule, high level rules；
The configuration rule of the FTP log snapshot includes: base rule, high level rules；
The configuration rule of the SSL certificate snapshot includes: base rule, high level rules；
The base rule includes: countries and regions, data session size, agreement；
The high level rules extract expression formula by the data that user writes and complete configuration；
The Exception Model includes: DDOS attack, and worm attack, Trojan characteristics, heartbeat is abnormal, and connection is abnormal, abnormal behavior, stream
Amount is abnormal, multiplexed port.
4. a kind of configuration method of network data snapshot according to claim 1, it is characterised in that: transported in above-mentioned steps S4
The snapshot that row is completed, the data extracted after the operation that can check snapshot.
5. a kind of configuration method of network data snapshot according to claim 1, it is characterised in that: all snapshot nodes are equal
It will record and check the operation footprint of data after user opens snapshot and analyze footprint.
6. a kind of configuration method of network data snapshot according to claim 1, it is characterised in that: in above-mentioned steps S4 also
Including curing data and reduction data；
When the curing data is included in snapshot operation, it can be taken out from father's snapshot according to the rule of recent snapshot configuration from the background special
Fixed data, and the data of taking-up are recorded in hard disk, realize data write；
When the reduction data are included in snapshot operation, it can be taken out from father's snapshot according to the rule of recent snapshot configuration from the background special
Fixed data simultaneously abandon remaining data, realize data reduction.
7. a kind of configuration method of network data snapshot according to claim 1, it is characterised in that: will be in above-mentioned steps S5
Snapshot tree save as snapshot template, can be directly new according to the production of snapshot template when obtaining new external network data
Snapshot tree.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810982000.4A CN109165363A (en) | 2018-08-27 | 2018-08-27 | A kind of configuration method of network data snapshot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810982000.4A CN109165363A (en) | 2018-08-27 | 2018-08-27 | A kind of configuration method of network data snapshot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109165363A true CN109165363A (en) | 2019-01-08 |
Family
ID=64896791
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810982000.4A Pending CN109165363A (en) | 2018-08-27 | 2018-08-27 | A kind of configuration method of network data snapshot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109165363A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110278465A (en) * | 2019-06-21 | 2019-09-24 | 湖南快乐阳光互动娱乐传媒有限公司 | A kind of full link data of video CD N distribution HTTP is collected and analysis method and system |
| CN111061559A (en) * | 2019-11-13 | 2020-04-24 | 成都安思科技有限公司 | Distributed data mining and statistical method based on data deduplication |
| CN111124756A (en) * | 2019-12-18 | 2020-05-08 | 浪潮(北京)电子信息产业有限公司 | Snapshot relationship display method, device, equipment and medium |
| CN112615857A (en) * | 2020-12-17 | 2021-04-06 | 杭州迪普科技股份有限公司 | Network data processing method, device and system |
| CN112702207A (en) * | 2020-12-25 | 2021-04-23 | 深圳市高德信通信股份有限公司 | Network data configuration method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106775957A (en) * | 2017-01-20 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of virtual machine snapshot implementation method |
| CN107026835A (en) * | 2015-11-03 | 2017-08-08 | 丛林网络公司 | Integrating security system with rule optimization |
| US20180101448A1 (en) * | 2016-10-10 | 2018-04-12 | AlphaPoint | Distributed ledger comprising snapshots |
-
2018
- 2018-08-27 CN CN201810982000.4A patent/CN109165363A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107026835A (en) * | 2015-11-03 | 2017-08-08 | 丛林网络公司 | Integrating security system with rule optimization |
| US20180101448A1 (en) * | 2016-10-10 | 2018-04-12 | AlphaPoint | Distributed ledger comprising snapshots |
| CN106775957A (en) * | 2017-01-20 | 2017-05-31 | 郑州云海信息技术有限公司 | A kind of virtual machine snapshot implementation method |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110278465A (en) * | 2019-06-21 | 2019-09-24 | 湖南快乐阳光互动娱乐传媒有限公司 | A kind of full link data of video CD N distribution HTTP is collected and analysis method and system |
| CN111061559A (en) * | 2019-11-13 | 2020-04-24 | 成都安思科技有限公司 | Distributed data mining and statistical method based on data deduplication |
| CN111124756A (en) * | 2019-12-18 | 2020-05-08 | 浪潮(北京)电子信息产业有限公司 | Snapshot relationship display method, device, equipment and medium |
| CN112615857A (en) * | 2020-12-17 | 2021-04-06 | 杭州迪普科技股份有限公司 | Network data processing method, device and system |
| CN112615857B (en) * | 2020-12-17 | 2023-02-17 | 杭州迪普科技股份有限公司 | Network data processing method, device and system |
| CN112702207A (en) * | 2020-12-25 | 2021-04-23 | 深圳市高德信通信股份有限公司 | Network data configuration method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109165363A (en) | A kind of configuration method of network data snapshot | |
| US11425229B2 (en) | Generating event streams from encrypted network traffic monitored by remote capture agents | |
| US10812514B2 (en) | Configuring the generation of additional time-series event data by remote capture agents | |
| CN107210928B (en) | Distributed and adaptive computer network analysis | |
| US11159386B2 (en) | Enriched flow data for network analytics | |
| US9755967B2 (en) | Method, apparatus, and system for configuring flow table in OpenFlow network | |
| US20160127180A1 (en) | Streamlining configuration of protocol-based network data capture by remote capture agents | |
| CN108234164B (en) | Cluster deployment method and device | |
| WO2020228527A1 (en) | Data stream classification method and message forwarding device | |
| CN103780610A (en) | Network data recovery method based on protocol characteristics | |
| CN104639391A (en) | Method for generating network flow record and corresponding flow detection equipment | |
| CN106997394B (en) | A kind of data random ordering arrival processing method and system | |
| WO2016074434A1 (en) | Data extraction method and device | |
| CN106650425A (en) | Method and device for controlling security sandbox | |
| CN104767636A (en) | Router setting method and device | |
| CN104050003A (en) | Method for starting Nutch collecting system with shell script | |
| CN105719072B (en) | System and method for associating multi-segment component transactions | |
| van De Wiel et al. | Enabling non-expert analysis of large volumes of intercepted network traffic | |
| WO2018214702A1 (en) | Method for acquiring standard configuration template of network device, and computation device | |
| CN114244555B (en) | Security policy adjusting method | |
| EP3800833B1 (en) | Deep packet inspection application classification systems and methods | |
| Yang et al. | Network traffic analysis based on Hadoop | |
| CN115589362B (en) | Method for generating and identifying device type fingerprint, device and medium | |
| WO2016192208A1 (en) | Virtual local area network (vlan) filtration processing method and apparatus | |
| CN116405294A (en) | Behavior subject association analysis method, system, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190108 |