CN112818045A - Data access unified management platform for big data - Google Patents
Data access unified management platform for big data Download PDFInfo
- Publication number
- CN112818045A CN112818045A CN202110085764.5A CN202110085764A CN112818045A CN 112818045 A CN112818045 A CN 112818045A CN 202110085764 A CN202110085764 A CN 202110085764A CN 112818045 A CN112818045 A CN 112818045A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- platform
- cluster
- data acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 22
- 238000012423 maintenance Methods 0.000 claims abstract description 11
- 230000005540 biological transmission Effects 0.000 claims description 9
- 230000003993 interaction Effects 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 claims description 3
- 238000011161 development Methods 0.000 abstract description 5
- 230000006870 function Effects 0.000 abstract description 5
- 238000013515 script Methods 0.000 abstract description 5
- 238000007726 management method Methods 0.000 description 34
- 230000002159 abnormal effect Effects 0.000 description 9
- 238000013480 data collection Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000018109 developmental process Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000000034 method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000000725 suspension Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000033772 system development Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Fuzzy Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a unified management platform of data access of big data, its characterized in that: the system comprises a high-availability load balancing module, a distributed cooperation module, a data acquisition cluster module, a data calculation cluster module and a WEB unified management scheduling platform. By arranging the unified access management platform, the unified management can be performed on processing functions with different input and output modes and corresponding data, independent programs or scripts do not need to be written in each data acquisition link, the development and maintenance difficulty is effectively reduced, and the performance and the stability of the platform are also obviously improved.
Description
Technical Field
The application relates to the technical field of big data, in particular to a data access unified management platform for big data.
Background
With the increasing popularity of big data technologies and applications, more and more companies choose to hug big data in the face of increasing traffic and data growth. However, with the development of companies, the business is continuously expanded, data shows an explosive growth situation, and data collection work becomes more important as the basis of big data work, and meanwhile, more problems and challenges are faced.
Although a distributed cluster scheme is adopted for file storage and calculation, the data acquisition link is mostly writing an independent program or script, even an acquisition tool of a BS framework. Therefore, a series of problems are generated, such as single point of failure, general performance and stability, unreasonable allocation and utilization of computing resources, difficulty in unified maintenance and management, and the like. For example:
1) data input and output are mostly local disks or RDBMSs, and the mode is relatively single. As services become more and more complex, processing functions adapted to different input and output modes need to be developed continuously, and development and maintenance difficulties are increased continuously.
2) Under the existing framework, all components are relatively independent and loose in structure. Each component needs to be maintained independently, the association and dependency relationship among the components are difficult to maintain, and the components are difficult to maintain and easy to operate mistakenly in a complex business scene.
3) In the data collection process, necessary audit statistics are lacked. The management of data assets is lacking, and data is not easy to find and troubleshoot when problems occur.
4) The fault tolerance is poor. When network fluctuation, interruption or other conditions cause data acquisition abnormity, data is easy to miss or dirty data is generated, and the data quality is reduced.
Disclosure of Invention
In order to solve the technical problems, the application provides a data access unified management platform for big data.
A big data access unified management platform comprises a high-availability + load balancing module, a distributed cooperation module, a data acquisition cluster module, a data calculation cluster module and a WEB unified management scheduling platform;
the high-availability + load balancing module is connected with the data acquisition cluster module, the data acquisition cluster module is connected with the data calculation cluster module, the output end of the distributed cooperation module is connected with the data acquisition cluster module and the data calculation cluster module, and the WEB unified management scheduling platform is connected with the high-availability + load balancing module, the data acquisition cluster module and the data calculation cluster module.
Optionally, the high-availability + load balancing module is a load balancing framework with a two-layer structure, that is, the LVS processing module of the first layer and the Nginx load balancing module of the second layer are communicated with the two-layer framework through a Redirect.
Optionally, the distributed coordination module is configured to coordinate the data collection cluster module and the data computation cluster module.
Optionally, the distributed collaboration module implements the collaboration based on a Zookeeper component.
Optionally, the data collection cluster module includes a plurality of streaming data collectors a and a plurality of batch data collectors B.
Optionally, the data collection cluster module is configured to start one or more collection services according to configuration and service requirements, and distribute data according to the configuration and registered computing services in the distributed coordination module; sending the log data to a message queue; and receiving the supplementary transmission message in the message queue, generating a supplementary transmission task serving as a batch processing task, and performing supplementary transmission on the data.
Optionally, the data computing cluster module is configured to start a corresponding computing service according to configuration and service requirements, register information in the distributed coordination module, receive data sent by the collection cluster, and send the data to a data warehouse or other components according to the configuration after computing; and sending the log data to a message queue.
Optionally, the platform further includes a fault-tolerant identification module, and the fault-tolerant identification module is configured to retrieve log data from the message queue for analysis and statistics.
Optionally, the platform further comprises an execution unit and a service interface; the actuator is used for controlling the starting and stopping of the service, the issuing of the configuration and the cluster monitoring; the service interface is used for providing an interface for the front-end equipment and executing user operation through interaction with the execution unit.
Optionally, the WEB unified management scheduling platform is configured to provide a unified management scheduling platform for a user, so as to implement maintenance of the entire service.
The invention has the beneficial effects that:
the big data access unified management platform provided by the application comprises a high-availability load balancing module, a distributed cooperation module, a data acquisition cluster module, a data calculation cluster module and a WEB unified management scheduling platform. By arranging the unified access management platform, the unified management can be performed on processing functions with different input and output modes and corresponding data, independent programs or scripts do not need to be written in each data acquisition link, the development and maintenance difficulty is effectively reduced, and the performance and the stability of the platform are also obviously improved.
In addition, the high availability and load balancing module is arranged as an inlet of the service cluster, so that the high availability and load balancing capability of the management platform to the cluster data can be obviously improved; the distributed cooperation module can optimize the cooperation work of the acquisition cluster and the calculation cluster according to the real-time requirement, and the data processing efficiency of the management platform is obviously improved.
In addition, the management platform in the application further comprises a fault-tolerant identification module which can perform sub-statistics on the log data transmitted to the message queue by the collection cluster and the calculation cluster, so that conflicts, errors and abnormal data can be found in time, the data quality is fully guaranteed, and the abnormal data and reasons can be easily checked and analyzed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a schematic structural diagram of a data access unified management platform for big data disclosed in an embodiment of the present application;
fig. 2 is a schematic structural diagram of an improved big data access unified management platform disclosed in the second embodiment of the present application;
fig. 3 is a schematic structural diagram of another improved big data access unified management platform disclosed in the third embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship which the present invention product is usually put into use, it is only for convenience of describing the present application and simplifying the description, but it is not intended to indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and thus, should not be construed as limiting the present application.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.
Although a distributed cluster scheme has been adopted in the prior art to realize storage and calculation of files, the data acquisition link is mostly writing an independent program or script, even an acquisition tool of a BS architecture. Therefore, a series of problems are generated, such as single point of failure, general performance and stability, unreasonable allocation and utilization of computing resources, difficulty in unified maintenance and management, and the like. Aiming at the series of problems, the application provides a whole set of solution, namely, a statistical access management platform is designed to realize big data access management of the distributed cluster equipment, so that the system development and maintenance difficulty is reduced, the data acquisition and processing efficiency can be obviously improved, the data quality can be ensured, and compared with the existing scheme, the technical effect is obvious and the market application value is high.
Example one
Referring to fig. 1, fig. 1 is a schematic structural diagram of a big data access unified management platform disclosed in an embodiment of the present application. As shown in fig. 1, a big data access unified management platform according to an embodiment of the present application includes a high availability + load balancing module, a distributed coordination module, a data collection cluster module, a data computation cluster module, and a WEB unified management scheduling platform;
the high-availability + load balancing module is connected with the data acquisition cluster module, the data acquisition cluster module is connected with the data calculation cluster module, the output end of the distributed cooperation module is connected with the data acquisition cluster module and the data calculation cluster module, and the WEB unified management scheduling platform is connected with the high-availability + load balancing module, the data acquisition cluster module and the data calculation cluster module.
According to the scheme, through the setting of the unified access management platform, unified management can be performed on processing functions with different input and output modes and corresponding data, independent programs or scripts are not required to be written in each data acquisition link, development and maintenance difficulty is effectively reduced, meanwhile, due to the fact that data acquisition and calculation processing are performed on the unified platform, a more reasonable processing scheme is convenient to set, and the performance and stability of the platform can be remarkably improved.
Optionally, the high-availability + load balancing module is a load balancing framework with a two-layer structure, that is, the LVS processing module of the first layer and the Nginx load balancing module of the second layer are communicated with the two-layer framework through a Redirect.
In order to realize high availability and load balance of a platform access layer, a two-layer load balance framework is designed in the application, and when the two-layer load balance framework is specifically realized: the request of the user can firstly reach the LVS processing module, and according to the strategy of DR mode synchronous request forwarding, after the LVS processing module receives the request, the request is transmitted to the Nginx load balancing module at the rear end through Redirect, and then the Nginx load balancing module performs secondary load balancing. Through the two-layer load balancing framework, the scheme of the application realizes high availability and load balancing in the access layer.
Optionally, the distributed coordination module is configured to coordinate the data collection cluster module and the data computation cluster module.
Optionally, the distributed collaboration module implements the collaboration based on a Zookeeper component.
Zookeeper is a distributed coordination service that provides functions such as configuration information maintenance, naming services, distributed synchronization, and service grouping. It has the following advantages: the Zookeeper exposed interface is wait-free and provides an event-driven mechanism; the Zookeeper implementation is efficient; zookeeper guarantees for each client that requests execute In FIFO (First In, First Out, First In First Out) order and has linearization capability for all requests that change Zookeeper status. The Zookeeper is applied to the cooperation of collecting and calculating two clusters by utilizing the characteristics of the Zookeeper, and can be specifically realized by adopting the following mode:
the Zookeeper assembly receives and stores service requirements sent by a platform, formulates configuration and service requirements based on the service requirements and sends the configuration and service requirements to the data acquisition cluster module so as to trigger the data acquisition cluster module to carry out data acquisition work, simultaneously monitors the working state and the processing load of the data calculation cluster module, and triggers the data calculation cluster module to carry out data processing based on the configuration and the service requirements when the working state of the data calculation cluster module is idle or the processing load is lower than a preset value.
Example two
Referring to fig. 2, fig. 2 is a schematic structural diagram of another big data access unified management platform disclosed in the embodiment of the present application. The second embodiment is a further improvement on the first embodiment, and the difference between the second embodiment and the first embodiment is that:
the data acquisition cluster module comprises a plurality of streaming data acquisition units and a plurality of batch processing data acquisition units.
Optionally, the data collection cluster module is configured to start one or more collection services according to configuration and service requirements, and distribute data according to the configuration and registered computing services in the distributed coordination module; sending the log data to a message queue; and receiving the supplementary transmission message in the message queue, generating a supplementary transmission task serving as a batch processing task, and performing supplementary transmission on the data.
Optionally, the data computing cluster module is configured to start a corresponding computing service according to configuration and service requirements, register information in the distributed coordination module, receive data sent by the collection cluster, and send the data to a data warehouse or other components according to the configuration after computing; and sending the log data to a message queue.
EXAMPLE III
Referring to fig. 3, fig. 3 is a schematic structural diagram of another big data access unified management platform disclosed in the embodiment of the present application. The third embodiment is a further improvement on the second embodiment, and the difference between the third embodiment and the second embodiment is that:
the platform also comprises a fault-tolerant identification module which is used for calling log data from the message queue to analyze and count.
Data collection abnormality is often caused by network fluctuation, interruption or other conditions, and is particularly characterized in that data is easy to miss or dirty data is generated, so that the reliability of the data is seriously influenced by the caused data quality reduction. Therefore, the fault-tolerant identification module is arranged for calling the log data from the message queue for analysis and statistics and discovering abnormal data in time. In addition, the processing of the abnormal data may include deletion, re-acquisition after suspension, abnormal marking, etc. of the abnormal data, and a deep learning algorithm may be used to analyze and identify the cause of the abnormal data and perform statistical analysis of the abnormal conditions, so as to facilitate the management platform to evaluate the system stability and perform the abnormal tracing.
Optionally, the platform further comprises an execution unit and a service interface; the actuator is used for controlling the starting and stopping of the service, the issuing of the configuration and the cluster monitoring; the service interface is used for providing an interface for the front-end equipment and executing user operation through interaction with the execution unit.
Optionally, the WEB unified management scheduling platform is configured to provide a unified management scheduling platform for a user, so as to implement maintenance of the entire service.
Those skilled in the art will appreciate that the functional implementation of all or part of the functional modules of the above embodiments can be implemented by a program to instruct related hardware, and the program can be stored in a computer-readable storage medium, which can include: ROM, RAM, magnetic or optical disks, and the like.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. The utility model provides a data access unified management platform of big data which characterized in that: the system comprises a high-availability load balancing module, a distributed cooperation module, a data acquisition cluster module, a data calculation cluster module and a WEB unified management scheduling platform;
the high-availability + load balancing module is connected with the data acquisition cluster module, the data acquisition cluster module is connected with the data calculation cluster module, the output end of the distributed cooperation module is connected with the data acquisition cluster module and the data calculation cluster module, and the WEB unified management scheduling platform is connected with the high-availability + load balancing module, the data acquisition cluster module and the data calculation cluster module.
2. The platform of claim 1, wherein: the high-availability + load balancing module is a load balancing framework with a two-layer structure, namely an LVS processing module of a first layer and an Nginx load balancing module of a second layer, and the two-layer framework is communicated through a Redirect.
3. The platform of claim 1, wherein: and the distributed cooperation module is used for cooperation of the data acquisition cluster module and the data calculation cluster module.
4. The platform of claim 1, wherein: the distributed collaboration module implements the collaboration based on a Zookeeper component.
5. The platform of claim 1, wherein: the data acquisition cluster module comprises a plurality of streaming data acquisition units A and a plurality of batch processing data acquisition units B.
6. The platform of claim 5, wherein: the data acquisition cluster module is used for starting one or more acquisition services according to configuration and service requirements and distributing data according to the configuration and registered computing services in the distributed cooperation module; sending the log data to a message queue; and receiving the supplementary transmission message in the message queue, generating a supplementary transmission task serving as a batch processing task, and performing supplementary transmission on the data.
7. The platform of claim 5, wherein: the data computing cluster module is used for starting corresponding computing service according to configuration and service requirements, registering information to the distributed coordination module, receiving data sent by the collection cluster, and sending the data to a data warehouse or other components according to configuration after computing; and sending the log data to a message queue.
8. The platform of claim 6 or 7, wherein: the platform also comprises a fault-tolerant identification module which is used for calling log data from the message queue to analyze and count.
9. The platform of claim 1, wherein: the platform further comprises an execution unit and a service interface; the actuator is used for controlling the starting and stopping of the service, the issuing of the configuration and the cluster monitoring; the service interface is used for providing an interface for the front-end equipment and executing user operation through interaction with the execution unit.
10. The platform of claim 1, wherein: the WEB unified management scheduling platform is used for providing a unified management scheduling platform for a user so as to realize maintenance of the whole service.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110085764.5A CN112818045A (en) | 2021-01-22 | 2021-01-22 | Data access unified management platform for big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110085764.5A CN112818045A (en) | 2021-01-22 | 2021-01-22 | Data access unified management platform for big data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112818045A true CN112818045A (en) | 2021-05-18 |
Family
ID=75858706
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110085764.5A Pending CN112818045A (en) | 2021-01-22 | 2021-01-22 | Data access unified management platform for big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112818045A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104036025A (en) * | 2014-06-27 | 2014-09-10 | 蓝盾信息安全技术有限公司 | Distribution-base mass log collection system |
| CN104519088A (en) * | 2013-09-27 | 2015-04-15 | 方正宽带网络服务股份有限公司 | Buffer memory system realization method and buffer memory system |
| CN106100894A (en) * | 2016-07-11 | 2016-11-09 | 华南理工大学 | A kind of highly reliable cluster operation management method |
| CN107733986A (en) * | 2017-09-15 | 2018-02-23 | 中国南方电网有限责任公司 | Support the protection of integrated deployment and monitoring operation big data support platform |
| CN109451065A (en) * | 2018-12-26 | 2019-03-08 | 中电福富信息科技有限公司 | A kind of soft load balancing shunts automated system and its operation method |
| CN110659180A (en) * | 2019-09-05 | 2020-01-07 | 国家计算机网络与信息安全管理中心 | Data center infrastructure management system based on cluster technology |
| CN111277672A (en) * | 2020-03-31 | 2020-06-12 | 上海积成能源科技有限公司 | Non-blocking input and output model-based energy Internet of things data acquisition method and software gateway |
| CN111787073A (en) * | 2020-06-18 | 2020-10-16 | 多加网络科技(北京)有限公司 | Current-limiting fusing platform and method for unified service |
-
2021
- 2021-01-22 CN CN202110085764.5A patent/CN112818045A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104519088A (en) * | 2013-09-27 | 2015-04-15 | 方正宽带网络服务股份有限公司 | Buffer memory system realization method and buffer memory system |
| CN104036025A (en) * | 2014-06-27 | 2014-09-10 | 蓝盾信息安全技术有限公司 | Distribution-base mass log collection system |
| CN106100894A (en) * | 2016-07-11 | 2016-11-09 | 华南理工大学 | A kind of highly reliable cluster operation management method |
| CN107733986A (en) * | 2017-09-15 | 2018-02-23 | 中国南方电网有限责任公司 | Support the protection of integrated deployment and monitoring operation big data support platform |
| CN109451065A (en) * | 2018-12-26 | 2019-03-08 | 中电福富信息科技有限公司 | A kind of soft load balancing shunts automated system and its operation method |
| CN110659180A (en) * | 2019-09-05 | 2020-01-07 | 国家计算机网络与信息安全管理中心 | Data center infrastructure management system based on cluster technology |
| CN111277672A (en) * | 2020-03-31 | 2020-06-12 | 上海积成能源科技有限公司 | Non-blocking input and output model-based energy Internet of things data acquisition method and software gateway |
| CN111787073A (en) * | 2020-06-18 | 2020-10-16 | 多加网络科技(北京)有限公司 | Current-limiting fusing platform and method for unified service |
Non-Patent Citations (2)
| Title |
|---|
| 刘海;张瞩熹;任雯;肖岩平;: "面向异构数据源的分布式集成工具研究与设计", 计算机应用研究, no. 1, pages 214 - 216 * |
| 陈荣赏等: "LVS/DR、NGINX混合负载均衡集群技术应用研究", 《湖北民族学院学报(自然科学版)》, vol. 35, no. 1, pages 23 - 26 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109347974B (en) | Hybrid scheduling system for improving online service quality and cluster resource utilization rate | |
| CN112860393B (en) | Distributed task scheduling method and system | |
| US20060074946A1 (en) | Point of view distributed agent methodology for network management | |
| CN109857558A (en) | A kind of data flow processing method and system | |
| Setzer et al. | Decision support for virtual machine reassignments in enterprise data centers | |
| CN103401704A (en) | Implementation scheme of distributed log collecting server | |
| CN113422842B (en) | Distributed power utilization information data acquisition system considering network load | |
| CN111464612B (en) | Method for providing stable computing service in severe environment | |
| CN102457578A (en) | Distributed network monitoring method based on event mechanism | |
| CN112631764A (en) | Task scheduling method and device, computer equipment and computer readable medium | |
| CN106131227A (en) | Balancing method of loads, meta data server system and load balance system | |
| CN114666335B (en) | Distributed system load balancing device based on data distribution service DDS | |
| Li et al. | Research and application of server cluster load balancing technology | |
| Jiang et al. | Research on intelligent monitoring scheme for microservice application systems | |
| CN103338160B (en) | Based on optimization system and the optimization method of the polling service of buffer queue | |
| CN113672452A (en) | Method and system for monitoring operation of data acquisition task | |
| CN112818045A (en) | Data access unified management platform for big data | |
| CN103326880A (en) | Genesys calling system high-availability cloud computing system and method | |
| Huang et al. | Research and application of microservice in power grid dispatching control system | |
| CN112579552A (en) | Log storage and calling method, device and system | |
| CN114567633A (en) | Cloud platform system supporting full life cycle of multi-stack database and management method | |
| CN113110935A (en) | Distributed batch job processing system | |
| CN112417015A (en) | Data distribution method and device, storage medium and electronic device | |
| CN112581080A (en) | Lightweight distributed workflow engine construction system | |
| CN113190341A (en) | Server resource scheduling method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |