CN108052358A - A kind of system and method for distributed deployment - Google Patents
A kind of system and method for distributed deployment Download PDFInfo
- Publication number
- CN108052358A CN108052358A CN201711322472.9A CN201711322472A CN108052358A CN 108052358 A CN108052358 A CN 108052358A CN 201711322472 A CN201711322472 A CN 201711322472A CN 108052358 A CN108052358 A CN 108052358A
- Authority
- CN
- China
- Prior art keywords
- transaction
- data
- server
- record
- file storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
Abstract
The present invention provides a kind of system and method for distributed deployment, including a host server and at least a slave server, host server include with slave server：Data acquisition module monitors the packet information in the data link of detecting and continuous collecting current server in real time；The packet information collected is parsed into transaction original record by data processing module, calculate transaction association field, transaction record is distributed according to transaction association field, the transaction record distributed is associated to record for transaction association, synthesis transaction association records to obtain transaction data and by transaction data storage into the file storage module of current server, it polymerize transaction data, the transaction data after polymerization is stored to the database of current server；The data exhibiting module of host server reads required database, the data in file storage and presentation.Realize different physical locations, the flow collection of the trans-regional cross-network segment in strange land.
Description
Technical field
The present invention relates to internet data monitoring fields, and in particular, to a kind of system and method for distributed deployment.
Background technology
In present bank, stock trader, network operator, mostly using Network Data Control system as routine analysis
With the important means of risk control.And the critically important part of Network Data Control system is environment deployment and data acquisition.With
The growth of science and technology and portfolio, network interconnection data to be processed also explosive growth therewith monitors these data for acquisition,
It is even more that the data center of above-mentioned industry has evolved into three center of two places, has multiple machines again inside each data center
Room, also trans-regional cross-network segment, traditional centralized deployment have been difficult to meet demand, have been faced with these internal servers simultaneously
Such as collection point deployment is difficult, and single-point server stress is excessive or failure causes the not available problem of cluster.
Meanwhile cumbersome multicenter framework is built and often wastes many manpower and materials, being susceptible to faulty operation causes portion
Administration's failure, and even if deployment is completed, additions and deletions server or change system architecture equally face many challenges.
It is found by retrieval：
The patent document of Publication No. CN105404528A discloses a kind of distributed automatization deployment system, used
Scheme needs additional automatic deployment server, and to pass through the multiple friendship between management end, deployment services device, client three
Interoperability adds deployment time with the probability of problem occurs；
The patent document of Publication No. CN106375419A discloses the dispositions method and device of a kind of distributed type assemblies,
Whether Dynamical Deployment using accounting can only be more than to set threshold value come additions and deletions node by decision node number, it is impossible to meet complicated
Network scenarios.
The content of the invention
For in the prior art the defects of, the object of the present invention is to provide a kind of system and method for distributed deployment, with
Solve the problems, such as the flow collection of the trans-regional cross-network segment of different physical locations, strange land.
A kind of method of the distributed deployment provided according to the present invention, including：
Data collection steps：The data packet letter in detecting and the data link of continuous collecting current server is monitored in real time
Breath；
Data processing step：The packet information collected is parsed into transaction original record, calculates transaction association word
Section, transaction record is distributed according to transaction association field, is associated the transaction record distributed and is recorded for transaction association, synthesizes transaction association
Record obtains transaction data and by transaction data storage into the file storage module of current server, polymerize transaction data, will
Transaction data after polymerization is stored to the database of current server；
Data exhibiting step：Host server reads required database, the data in file storage module and presentation.
Preferably, the transaction data stored in file storage module includes data packet, transaction original record and friendship
Easy associated record.
Preferably, the transaction data stored in file storage module is according to source and time point multistage catalogue.
Preferably, transaction data after the polymerization stored in database according to path profile and month point storehouse, according to capture point
Divide table and divide table according to extent of polymerization, extent of polymerization includes capture point, server ip, website and dimension.
Preferably, data exhibiting step includes：
Transaction record rendering step：By the transaction association field and the data in file storage are combined, transaction is presented
Record information；
Indication information rendering step：Database is inquired about by customized time, server, website and dimension, is presented
Indication information under each dimension combination.
A kind of system of the distributed deployment provided according to the present invention, including a host server and at least one from
Machine server, the host server include with the slave server：
Data acquisition module：The data packet letter in detecting and the data link of continuous collecting current server is monitored in real time
Breath；
Data processing module：The packet information collected is parsed into transaction original record, calculates transaction association word
Section, transaction record is distributed according to transaction association field, is associated the transaction record distributed and is recorded for transaction association, synthesizes transaction association
Record obtains transaction data and by transaction data storage into the file storage module of current server, polymerize transaction data, will
Transaction data after polymerization is stored to the database of current server；
The host server further includes data exhibiting module；
Data exhibiting module：Database, the data in file storage module and presentation needed for reading.
Preferably, the transaction data stored in file storage module includes data packet, transaction original record and friendship
Easy associated record.
Preferably, the transaction data stored in file storage module is according to source and time point multistage catalogue.
Preferably, transaction data after the polymerization stored in database according to path profile and month point storehouse, according to capture point
Divide table and divide table according to extent of polymerization, extent of polymerization includes capture point, server ip, website and dimension.
Preferably, data exhibiting module includes：
Submodule is presented in transaction record：By the transaction association field and the data in file storage are combined, presents and hands over
Easily record information；
Submodule is presented in indication information：Database is inquired about by customized time, server, website and dimension, is in
Indication information under existing each dimension combination.
Compared with prior art, the present invention has following advantageous effect：
1) strange land multicenter is disposed, and can gather the network data for monitoring different physical locations；
2) load balancing of multiple servers realization flow, the pressure of sharing data flow, risk are controllable；
3) installation of distributed structure/architecture deployment and modification process are simple to operation, and configuration is simple, flexible and changeable；
4) high availability mechanism evades Single Point of Faliure.
Description of the drawings
Upon reading the detailed description of non-limiting embodiments with reference to the following drawings, other feature of the invention,
Objects and advantages will become more apparent upon：
Fig. 1 is the system module figure of the present invention；
Fig. 2 is the operation step map of data exhibiting module of the present invention；
Fig. 3 is the distributed environment installation steps figure of the present invention；
Fig. 4 is the distributed environment amendment step figure of the present invention.
Specific embodiment
With reference to specific embodiment, the present invention is described in detail.Following embodiment will be helpful to the technology of this field
Personnel further understand the present invention, but the invention is not limited in any way.It should be pointed out that the ordinary skill to this field
For personnel, without departing from the inventive concept of the premise, several changes and improvements can also be made.These belong to the present invention
Protection domain.
As shown in Figure 1, the system of distributed deployment provided in this embodiment has A, B Liang Ge data centers, wherein A is host
Server, B are slave server.Two servers include with lower module：
Data acquisition module：The data packet letter in detecting and the data link of continuous collecting current server is monitored in real time
Breath；
Data processing module：It obtains the packet information collected and is parsed into transaction original record, calculate transaction and close
Join field, transaction record is distributed according to transaction association field, associate the transaction record distributed and recorded for transaction association, synthesis transaction
Associated record obtains transaction data and by transaction data storage into the file storage module of current server, polymerize number of deals
According to the transaction data after polymerization is stored to the database of current server, is sent after completing the data processing step of one minute
End signal to database, the page according to corresponding field in database can determine whether one minute (time shaft of page presentation be by
Minute driving) data have and do not handle completion, the just time shaft and path profile on the page of driving data display module after finishing
Data are moved along one minute；
In addition, host server further includes data exhibiting module；
Data exhibiting module：Database, the data in file storage module and presentation needed for reading.
In the present invention, the transaction data stored in file storage module include data packet, transaction original record with
And transaction association record, and according to source and time point multistage catalogue.
Transaction data after the polymerization stored in database divide according to path profile and month point storehouse, according to capture point table and
Divide table according to extent of polymerization, extent of polymerization includes capture point, server ip, website and dimension.
Data exhibiting module includes：
Submodule is presented in transaction record：By the transaction association field and the data in file storage are combined, presents and hands over
Easily record information；
Submodule is presented in indication information：Database is inquired about by customized time, server, website and dimension, is in
Indication information under existing each dimension combination.
As shown in Fig. 2, the system based on above-mentioned distributed deployment, the concrete operations of data exhibiting module have：
Step 1, page host process, exist only on host server, and when being operated on the page, which receives foreground
Foregrounding task is issued associated process and performed by page request, while the host process also carries whole system record information
Task, such as system login, page access, abnormal operation etc.；
Step 2, page process, slave server have respective page process, for page master in receiving step 1 into
The task for the current server that journey is distributed simultaneously is issued to corresponding middleware；
Step 3, turobject, middleware, slave server page process has relevant middleware, for across clothes
The decomposition for device statistics of being engaged in, distributes with merging, and the decomposition of across path profile capture point statistics is with merging；By keyword query of merchandising,
With reference to the content in file storage module, specific transaction record information is presented on the page；And by self defined time,
The indication information under each dimension combination is presented in server, website, dimension inquiry database on the page.
According to Fig. 1 and Fig. 2, slave server has respective flow chart of data processing and database file storage,
When pertaining only to the inquiry of slave basic data, can digital independent exhibition directly be completed by the process and middleware of slave server
It is existing, so as to fulfill high availability mechanism, even if there is Single Point of Faliure, the operation of whole system will not be influenced.
Distributed system design thought is the id identifiers of every server existence anduniquess, there is storage machine information from the background
Server configuration files include ID the and IP ends of all machines in host ip port information and entire distributed system in file
Message ceases, and associated process meeting real-time loading corresponds to configuration and is shown in the capture point option of path profile edit page.When establishing
After Trading Model, each module process of system can judge that processing is deposited by the resolve packet of the machine network interface card into transaction data according to ID
Home server is put into, finally by the data exhibiting module of host by data display to front page layout.
As shown in figure 3, specific distributed environment installation and deployment flow is as follows：
1) installation kit is run on host server, type of server selects master in installation process；
2) installation kit is run on slave server, type of server selects slave in installation process, while fills in host clothes
Be engaged in device IP；
3) after installation is complete, in page creation service path figure, the capture point of slave network interface card is selected, establishes transaction mould
Type.
As shown in figure 4, specific distributed environment modification deployment flow is as follows：
1) host server backstage modification server configuration files are newest server info；
2) amended server configuration files are copied to all slave servers；
3) network interface card in service path figure is reselected, creates new Trading Model.
According to the method that the operating procedure of the system environmental structure and comprising modules is provided, a specific implementation is provided such as
Under：
1st, installation kit is run on a data center server A1 on A ground, in installation process, selection is arranged to host；
2nd, run installation kit on another data center server A2 on A ground, in installation process, selection be arranged to from
Machine, while host is inputted as A1 server ips；
3rd, installation kit is run on a data center server B1 on B ground, in installation process, selection is arranged to slave,
Host is inputted simultaneously as A1 server ips；
4th, after the completion of the installation kit of three machines performs, then the environment deployment of three central distribution formula of two places is successfully completed；
5th, in browser access systematic connection, access path figure edit page selects each server according to actual conditions
Network interface card build Trading Model as capture point；
6th, user can on the page the entire Trading Model of real time inspection data cases；
7th, user can be in the page by the data target situation of the capture point inquiry arbitrary dimension of random time.
One skilled in the art will appreciate that except realizing system provided by the invention in a manner of pure computer readable program code
It, completely can be by the way that method and step progress programming in logic be provided come the present invention and its beyond each device, module, unit
System and its each device, module, unit with logic gate, switch, application-specific integrated circuit, programmable logic controller (PLC) and embedding
Enter the form of the controller that declines etc. to realize identical function.So system provided by the invention and its every device, module, list
Member is considered a kind of hardware component, and the device for being used to implement various functions, module, unit to including in it also may be used
To be considered as the structure in hardware component；The device for being used to implement various functions, module, unit can also be considered as either real
The software module of existing method can be the structure in hardware component again.
Specific embodiments of the present invention are described above.It is to be appreciated that the invention is not limited in above-mentioned
Particular implementation, those skilled in the art can make a variety of changes or change within the scope of the claims, this not shadow
Ring the substantive content of the present invention.In the case where there is no conflict, the feature in embodiments herein and embodiment can arbitrary phase
Mutually combination.
Claims (10)
- A kind of 1. method of distributed deployment, which is characterized in that including：Data collection steps：The packet information in detecting and the data link of continuous collecting current server is monitored in real time；Data processing step：The packet information collected is parsed into transaction original record, calculates transaction association field, root Transaction record is distributed according to transaction association field, the transaction record distributed is associated and is recorded for transaction association, synthesis transaction association record It obtains transaction data and by transaction data storage into the file storage module of current server, polymerize transaction data, will polymerize Transaction data afterwards is stored to the database of current server；Data exhibiting step：Host server reads required database, the data in file storage module and presentation.
- 2. the method for distributed deployment according to claim 1, which is characterized in that is stored in file storage module is described Transaction data includes data packet, transaction original record and transaction association record.
- 3. the method for distributed deployment according to claim 1, which is characterized in that is stored in file storage module is described Transaction data is according to source and time point multistage catalogue.
- 4. the method for distributed deployment according to claim 1, which is characterized in that the friendship after the polymerization stored in database Easy data divide table according to path profile and month point storehouse, according to capture point and divide table according to extent of polymerization, and extent of polymerization includes catching Obtain point, server ip, website and dimension.
- 5. the method for distributed deployment according to claim 1, which is characterized in that data exhibiting step includes：Transaction record rendering step：By the transaction association field and the data in file storage are combined, transaction record is presented Information；Indication information rendering step：Database is inquired about by customized time, server, website and dimension, is presented each Indication information under dimension combination.
- 6. a kind of system of distributed deployment, which is characterized in that including a host server and an at least slave service Device, the host server include with the slave server：Data acquisition module：The packet information in detecting and the data link of continuous collecting current server is monitored in real time；Data processing module：The packet information collected is parsed into transaction original record, calculates transaction association field, root Transaction record is distributed according to transaction association field, the transaction record distributed is associated and is recorded for transaction association, synthesis transaction association record It obtains transaction data and by transaction data storage into the file storage module of current server, polymerize transaction data, will polymerize Transaction data afterwards is stored to the database of current server；The host server further includes data exhibiting module；Data exhibiting module：Database, the data in file storage module and presentation needed for reading.
- 7. the system of distributed deployment according to claim 6, which is characterized in that is stored in file storage module is described Transaction data includes data packet, transaction original record and transaction association record.
- 8. the system of distributed deployment according to claim 6, which is characterized in that is stored in file storage module is described Transaction data is according to source and time point multistage catalogue.
- 9. the system of distributed deployment according to claim 6, which is characterized in that the friendship after the polymerization stored in database Easy data divide table according to path profile and month point storehouse, according to capture point and divide table according to extent of polymerization, and extent of polymerization includes catching Obtain point, server ip, website and dimension.
- 10. the system of distributed deployment according to claim 6, which is characterized in that data exhibiting module includes：Submodule is presented in transaction record：By the transaction association field and the data in file storage are combined, transaction note is presented Record information；Submodule is presented in indication information：Database is inquired about by customized time, server, website and dimension, is presented each Indication information under a dimension combination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711322472.9A CN108052358B (en) | 2017-12-12 | 2017-12-12 | Distributed deployment system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711322472.9A CN108052358B (en) | 2017-12-12 | 2017-12-12 | Distributed deployment system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108052358A true CN108052358A (en) | 2018-05-18 |
| CN108052358B CN108052358B (en) | 2021-07-13 |
Family
ID=62131966
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711322472.9A Active CN108052358B (en) | 2017-12-12 | 2017-12-12 | Distributed deployment system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108052358B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110572436A (en) * | 2019-08-12 | 2019-12-13 | 浙江讯盟科技有限公司 | Multi-place cross-cluster server deployment method and system |
| CN110768866A (en) * | 2019-10-23 | 2020-02-07 | 通号城市轨道交通技术有限公司 | Method and device for building distributed bottom layer framework |
| CN112069260A (en) * | 2020-09-09 | 2020-12-11 | 中国人民银行清算总中心 | Data storage and account checking method and system |
| CN112685402A (en) * | 2019-10-17 | 2021-04-20 | 拉扎斯网络科技（上海）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN113438619A (en) * | 2021-06-29 | 2021-09-24 | 中车青岛四方车辆研究所有限公司 | Multi-data-source information acquisition system and method based on Internet of things |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2555129A1 (en) * | 2011-08-03 | 2013-02-06 | Amadeus s.a.s. | Method and system to maintain strong consistency of distributed replicated contents in a client/server system |
| CN103368790A (en) * | 2013-06-21 | 2013-10-23 | 冯轶 | Performance delay monitoring method aiming at electronic trading system and system thereof |
| CN104778188A (en) * | 2014-02-24 | 2015-07-15 | 贵州电网公司信息通信分公司 | Distributed device log collection method |
| CN105005518A (en) * | 2015-08-24 | 2015-10-28 | 上海天旦网络科技发展有限公司 | System and method for automatically aggregating transaction data of redundant system as well as processor |
| CN106960037A (en) * | 2017-03-22 | 2017-07-18 | 河海大学 | A kind of distributed index the resources integration and share method across intranet and extranet |
-
2017
- 2017-12-12 CN CN201711322472.9A patent/CN108052358B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2555129A1 (en) * | 2011-08-03 | 2013-02-06 | Amadeus s.a.s. | Method and system to maintain strong consistency of distributed replicated contents in a client/server system |
| CN103368790A (en) * | 2013-06-21 | 2013-10-23 | 冯轶 | Performance delay monitoring method aiming at electronic trading system and system thereof |
| CN104778188A (en) * | 2014-02-24 | 2015-07-15 | 贵州电网公司信息通信分公司 | Distributed device log collection method |
| CN105005518A (en) * | 2015-08-24 | 2015-10-28 | 上海天旦网络科技发展有限公司 | System and method for automatically aggregating transaction data of redundant system as well as processor |
| CN106960037A (en) * | 2017-03-22 | 2017-07-18 | 河海大学 | A kind of distributed index the resources integration and share method across intranet and extranet |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110572436A (en) * | 2019-08-12 | 2019-12-13 | 浙江讯盟科技有限公司 | Multi-place cross-cluster server deployment method and system |
| CN110572436B (en) * | 2019-08-12 | 2020-09-22 | 浙江讯盟科技有限公司 | Multi-place cross-cluster server deployment method and system |
| CN112685402A (en) * | 2019-10-17 | 2021-04-20 | 拉扎斯网络科技（上海）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN112685402B (en) * | 2019-10-17 | 2023-03-24 | 拉扎斯网络科技（上海）有限公司 | Data storage and query method and device, electronic equipment and storage medium |
| CN110768866A (en) * | 2019-10-23 | 2020-02-07 | 通号城市轨道交通技术有限公司 | Method and device for building distributed bottom layer framework |
| CN112069260A (en) * | 2020-09-09 | 2020-12-11 | 中国人民银行清算总中心 | Data storage and account checking method and system |
| CN113438619A (en) * | 2021-06-29 | 2021-09-24 | 中车青岛四方车辆研究所有限公司 | Multi-data-source information acquisition system and method based on Internet of things |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108052358B (en) | 2021-07-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108052358A (en) | A kind of system and method for distributed deployment | |
| CN110245035A (en) | A kind of link trace method and device | |
| CN109328335A (en) | Intelligence configuration discovery technique | |
| CN103631699A (en) | Log management system and method for log monitoring, acquiring and querying | |
| CN104216948B (en) | A kind of difference control method of data model | |
| CN108737555A (en) | Industrial equipment management method based on Internet of Things and system | |
| DE102010007967A1 (en) | Method, computer program product and computer-readable storage medium for the generic creation of a structure tree for describing an IT process | |
| CN107018042A (en) | Method for tracing and tracing system for online service system | |
| CN104331457A (en) | Database node-based data access method and system | |
| CN102073579B (en) | Method for merging and optimizing audit events of Linux file system | |
| CN102932195A (en) | Networking protocol analysis-based business analysis monitoring method and system | |
| CN104122871B (en) | Real-time monitoring method of semiconductor test data | |
| CN105208090A (en) | Zookeeper-based Leader selection method | |
| CN105354697A (en) | Financial account rule base based automatic online auditing method and system | |
| US8166004B2 (en) | Verifying data consistency among structured files | |
| CN104766240A (en) | Electronic banking data processing system and method | |
| CN112308727A (en) | Insurance claim settlement service processing method and device | |
| CN113672692B (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN110347664A (en) | A kind of data visualization processing method and processing device | |
| CN113721950A (en) | Resource loss prevention and control processing method and device | |
| CN110825609B (en) | Service testing method, device and system | |
| CN110457351A (en) | A kind of management system and management method based on government financing platform data information | |
| CN105871659A (en) | Monitoring method and device of newly added server | |
| CN115480748A (en) | Service arrangement method, device and storage medium | |
| CN115392501A (en) | Data acquisition method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |