CN108881477A - A method of it is acquired and is monitored based on distributed file - Google Patents
A method of it is acquired and is monitored based on distributed file Download PDFInfo
- Publication number
- CN108881477A CN108881477A CN201810778031.8A CN201810778031A CN108881477A CN 108881477 A CN108881477 A CN 108881477A CN 201810778031 A CN201810778031 A CN 201810778031A CN 108881477 A CN108881477 A CN 108881477A
- Authority
- CN
- China
- Prior art keywords
- acquisition
- file
- cluster
- webapp
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/12—Network monitoring probes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Abstract
The invention discloses a kind of methods based on the acquisition monitoring of distributed file, include the following steps：S1：Server-side is constructed using Mysql cluster and Redis cluster, and arranges WEBAPP in server-side；S2：Task definition is provided by WEBAPP；S3：Task is stored in server-side；S4：Client obtains mission bit stream by reading Redis cluster every time, carries out task processing；S5：Task starts, and monitoring programme checks the integrality of collecting flowchart, and compensation is taken to acquire；S6：Monitor logging ftp logs in situation and network condition；S7：Client acquisition data are stored to long-range storage cluster；S8：WEBAPP control downloading uploads or stops acquisition tasks.Method provided by the invention based on the acquisition monitoring of distributed file, decentralization are not present host node child node, can extend acquisition applications at any time；WEBAPP visualized operation is provided, can be used to intuitive and convenient；It using Docker mode, can be existed simultaneously with the acquisition of multiple versions without conflicting mutually, compatibility is strong.
Description
Technical field
The present invention relates to a kind of file collection method more particularly to a kind of sides based on the acquisition monitoring of distributed file
Method.
Background technique
In the period of big data is more and more important, data acquisition is big data exploitation/most indispensable part of analysis.Before
Phase needs hardware or the network equipment carries out master data acquisition, and the data file that each producer or equipment are collected needs a concentration
Data center transmitted or safeguarded.Due to the diversity and complexity of file, need special designing one it is stable, can supervise
The file of control acquires monitoring method, for liberating human cost, the process of quick positioning acquisition and wrong the reason of occurring.
Meanwhile when data volume is increasing, the stability and scalability of ensuring method are needed, the side of acquisition is needed
Method carries out extending transversely, it is also desirable to have self-test, the functions such as be restarted automatically.
Existing method has following three kinds：
1,Flume-NG：Flume NG is a distribution, reliable, available system, it can be by different data sources
Massive logs data are efficiently collected, are polymerize, are moved, and are finally stored into a centralization data-storage system.By original
Flume OG Flume NG till now, carried out framework reconstruct, and NG version incompatible original OG version completely now
This.After framework reconstructs, the small tool that Flume NG is more like a light weight is very simply easily adapted to various mode days
Will is collected, and supports failover and load balancing.
2,Kafka：Kafka is that a kind of distributed post of high-throughput subscribes to message system, it can handle consumer
Everything flow data in the website of scale.This movement (web page browsing, the action of search and other users) is in the modern times
One key factor of many social functions on network.These data are often as the requirement of handling capacity and pass through processing day
Will and log aggregation solve.For the daily record data and off-line analysis system as Hadoop, but require place in real time
The limitation of reason, this is a feasible solution.The purpose of Kafka is by the loaded in parallel mechanism of Hadoop come unified line
Upper and offline Message Processing, also for providing real-time consumption by cluster.
3,Facebook Scribe：Scribe is the result collection system of Facebook open source, inside Facebook
Application through obtaining.It can be from collector journal on various Log Sources, and storing to a central storage system (can be NFS, divide
Cloth file system etc.) on, in order to concentrate statistical analysis processing.It is " distributed collection is uniformly processed " of log
Provide expansible a, scheme highly fault tolerant.When the network of central storage system or machine break down, scribe
Log can be dumped to local or another position, after central storage system restores, scribe can be by the log weight of unloading
Newly it is transferred to central storage system.It is usually used in combination with Hadoop, and scribe is used for the push log into HDFS, and
Hadoop is periodically handled by MapReduce operation.
Therefore the shortcomings that prior art, is as follows：
Flume-NG：1, basic component and example are only provided, specific business and Row control are needed according to business demand
It is perfect；2, it is all to pass through command line mode that starting, which is restarted,；3, it is checked without visualization tool；
Kafka：1, it is only suitable for simple data acquisition or the transmission of message；2, lack the monitoring module of data；3, with magnetic
It takes inventory and is taken as cost；
Facebook Scribe：1, it is developed using C++, developer is relatively smaller；2. domestic document and use are few.
Therefore, the prior art could be improved.
Summary of the invention
The technical problem to be solved in the present invention is to provide a kind of methods based on the acquisition monitoring of distributed file, solve number
Problem is monitored according to the integrality and data flow of acquisition.
The present invention is adopted the technical solution adopted is that providing one kind based on distributed file to solve above-mentioned technical problem
Collect the method for monitoring, which is characterized in that include the following steps：S1：Server-side is constructed using Mysql cluster and Redis cluster, and
WEBAPP is arranged in server-side；S2：Task definition is provided by WEBAPP, every kind of business or each file path are one and appoint
Business；S3：Task is stored in server-side, information scanning one time of file is saved in server-side；S4：Client passes through each
It reads Redis cluster and obtains mission bit stream, and record file read-write information simultaneously, carry out task processing；S5：Task starts, prison
The integrality of program checkout collecting flowchart is controlled, discovery has service stopping or acquisition current task overlong time, then alerts, and
Restart collecting flowchart；When the file of the file of downloading and scanning is inconsistent, take compensation acquire, compensation acquisition three times not at
Function is then placed in warning information, also provides warning information when file is empty, warning information is shown in WEBAPP；S6：Monitoring
Program records ftp and logs in situation and network condition, and monitoring information is saved in server-side, browses monitoring letter by WEBAPP
Breath carries out location of mistake；S7：Client acquisition data are stored to long-range storage cluster, and long-range storage cluster breaks down, stops automatically
Only data upload to long-range storage cluster, and the locally downloading disk of data is saved, until long-range storage cluster restores；
S8：Control switch is separately provided to downloading and upload in WEBAPP, when long-range storage cluster is problematic, WEBAPP control downloading,
Upload or stop acquisition tasks.
Further, the Mysql cluster-based storage historical data, the data that the Redis cluster-based storage is handled in real time.
Further, the Redis cluster is using character string, chained list, set, ordered set and hash type data class
The data-storage system of type provides a variety of different sortords, and disk or handle periodically is written in the data of update
In the additional record file of modification operation write-in.
Further, the long-range storage cluster includes HDFS and Kafka cluster, the special industry of Kafka cluster-based storage
Business demand data, the HDFS completely save the file of all acquisitions.
Further, the client is installed by Docker mode, and the Docker is application container engine, described
Container in Docker uses sandbox mechanism.
Further, the monitoring information include the ftp information of destination path, filename, file size, compressed format,
Acquisition time, downloading deadline, processing time last time and last time file size.
The present invention, which compares the prior art, following beneficial effect：It is provided by the invention to be supervised based on the acquisition of distributed file
The method of control, decentralization are not present host node child node, can extend acquisition applications at any time；WEBAPP visualization behaviour is provided
Make, intuitive and convenient business personnel can be allowed also to can be used；Using DOCKER mode, can be existed simultaneously with the acquisition of multiple versions
Without conflicting mutually, compatibility is strong.
Detailed description of the invention
Fig. 1 is the method flow diagram based on the acquisition monitoring of distributed file in the embodiment of the present invention；
Fig. 2 is the method architecture diagram based on the acquisition monitoring of distributed file in the embodiment of the present invention.
Specific embodiment
The invention will be further described with reference to the accompanying drawings and examples.
Fig. 1 is the method flow diagram based on the acquisition monitoring of distributed file in the embodiment of the present invention.
Referring to Figure 1, the method provided by the invention based on the acquisition monitoring of distributed file, wherein including walking as follows
Suddenly：
S1：Server-side is constructed using Mysql cluster and Redis cluster, and arranges WEBAPP in server-side；
S2：Task definition is provided by WEBAPP, every kind of business or each file path are a task；
S3：Task is stored in server-side, information scanning one time of file is saved in server-side；
S4：Client obtains mission bit stream by reading Redis cluster every time, and records file read-write information simultaneously, into
The processing of row task；
S5：Task starts, and monitoring programme checks the integrality of collecting flowchart, and discovery has service stopping or acquisition as predecessor
Business overlong time, then alert, and restart collecting flowchart；When the file of the file of downloading and scanning is inconsistent, compensation is taken
Acquisition, compensation acquisition is unsuccessful three times, then is placed in warning information, also provides warning information, warning information when file is empty
It is shown in WEBAPP；
S6：Monitor logging ftp logs in situation and network condition, and monitoring information is saved in server-side, passes through
WEBAPP browses monitoring information, facilitates positioning mistake；
S7：Client acquisition data are stored to long-range storage cluster, and long-range storage cluster breaks down, is automatically stopped in data
Long-range storage cluster is passed to, and the locally downloading disk of data is saved, until long-range storage cluster restores；
S8：Control switch is separately provided to downloading and upload in WEBAPP, when long-range storage cluster is problematic, WEBAPP control
System downloading uploads or stops acquisition tasks.
Referring also to Fig. 2, the Mysql cluster-based storage historical data, the number that the Redis cluster-based storage is handled in real time
According to；The Redis cluster is the data storage system using character string, chained list, set, ordered set and hash type data type
System provides the sequence of multitude of different ways, and disk periodically is written in the data of update or modification operation write-in is chased after
In the record file added.WEBAPP provides information inquiry, task definition, the information such as alarm.Occur that ftp is obstructed or long-range storage collection
Group's failure, meeting photos and sending messages notify maintenance personnel.The foundation for simplifying task, is operated by WEB interface, can individually test a number
According to reading downloading upload process.For convenience of test, handles file and read and store the sequence of operations such as long-range storage cluster
It can be increased by the page.
Long-range storage cluster includes HDFS and Kafka cluster, the Kafka cluster-based storage particular service demand data, institute
State the file that HDFS completely saves all acquisitions.Since kafka has the characteristic of the high amount of gulping down, needs and data can long Kubo
It deposits, being directed to particular service demand data can store on kafak, and the personnel of different technologies type selecting is facilitated to carry out data acquisition.
The file of all acquisitions is completely saved on HDFS, and timing merges small documents, reduces the accumulation of metadata.HDFS is designed to
It is suitble to operate in the distributed file system on common hardware.It and existing distributed file system have many common ground.But
Meanwhile the difference of it and other distributed file systems is also apparent.HDFS is the system of an Error Tolerance, is fitted
Conjunction is deployed on cheap machine.HDFS can provide the data access of high-throughput, be very suitable to answering on large-scale dataset
With.HDFS relaxes a part of POSIX constraint, to realize that streaming reads the purpose of file system data
Client is installed by Docker mode, and the acquisition monitoring programme of Docker installation can guarantee all clients
Using consistency is all saved, reduces to greatest extent and bug occur and save the O&M time；The Docker is that application container is drawn
It holds up, the Docker container is not using having interface between sandbox mechanism container, since every machine has different disk names
Or other environmental variances etc. are inconsistent, and the loss of time can be caused to the deployment of application program, can be reduced greatly by Docker
The deployment time of amount, or even a large amount of quickly upgrading and rollback are realized by script operation.Therefore the extension of acquisition can nothing
Seam carries out, and does not need to stop original acquisition.
Monitoring information includes ftp information, filename, file size, the compressed format, acquisition time, downloading of destination path
Deadline, processing time last time and last time file size.
In conclusion the method provided by the invention for being acquired and being monitored based on distributed file, advantage are as follows：1, it acquires
Docker mode is taken in deployment, reduces the influence of environmental factor；2, a machine can dispose the acquisition journey of multiple and different versions
Sequence is not interfere with each other；3, historical information can be counted；4, non-stop layer mode is taken, client and server-side pass through cluster
Mode, single node, which breaks down, does not influence the progress of overall task；5, web interface inquiry and addition task are provided, and led to
The monitoring to all nodes of process is crossed, specific acquisition accident can be quickly positioned.
Although the present invention is disclosed as above with preferred embodiment, however, it is not to limit the invention, any this field skill
Art personnel, without departing from the spirit and scope of the present invention, when can make a little modification and perfect therefore of the invention protection model
It encloses to work as and subject to the definition of the claims.
Claims (6)
1. a kind of method based on the acquisition monitoring of distributed file, which is characterized in that include the following steps：
S1：Server-side is constructed using Mysql cluster and Redis cluster, and arranges WEBAPP in server-side；
S2：Task definition is provided by WEBAPP, every kind of business or each file path are a task；
S3：Task is stored in server-side, information scanning one time of file is saved in server-side；
S4：Client obtains mission bit stream by reading Redis cluster every time, and records file read-write information simultaneously, is appointed
Business processing；
S5：Task starts, and monitoring programme checks the integrality of collecting flowchart, when discovery has service stopping or acquisition current task
Between it is too long, then alert, and restart collecting flowchart；When the file of the file of downloading and scanning is inconsistent, compensation is taken to acquire,
Compensation acquisition is unsuccessful three times, then is placed in warning information, also provides warning information when file is empty, warning information is shown in
In WEBAPP；
S6：Monitor logging ftp logs in situation and network condition, and monitoring information is saved in server-side, passes through WEBAPP
Monitoring information is browsed, location of mistake is carried out；
S7：Client acquisition data are stored to long-range storage cluster, and long-range storage cluster breaks down, is automatically stopped data and uploads to
Long-range storage cluster, and the locally downloading disk of data is saved, until long-range storage cluster restores；
S8：Control switch is separately provided to downloading and upload in WEBAPP, when long-range storage cluster is problematic, is controlled by WEBAPP
Downloading uploads or stops acquisition tasks.
2. the method as described in claim 1 based on the acquisition monitoring of distributed file, which is characterized in that the Mysql collection
Group's store historical data, the data that the Redis cluster-based storage is handled in real time.
3. the method as claimed in claim 2 based on the acquisition monitoring of distributed file, which is characterized in that the Redis collection
Group be using character string, chained list, set, ordered set and hash type data type data-storage system, provide it is a variety of not
Same sortord, and disk periodically is written into the data of update or modification is operated the record file of write-in addition
In.
4. the method as described in claim 1 based on the acquisition monitoring of distributed file, which is characterized in that the long-range storage
Cluster includes HDFS and Kafka cluster, the Kafka cluster-based storage particular service demand data, and the HDFS is for completely protecting
Deposit the file of all acquisitions.
5. the method as described in claim 1 based on the acquisition monitoring of distributed file, which is characterized in that the client is logical
It crosses Docker mode to install, the Docker is application container engine, and the container in the Docker uses sandbox mechanism.
6. the method as described in claim 1 based on the acquisition monitoring of distributed file, which is characterized in that the monitoring information
Ftp information, filename, file size, compressed format, acquisition time, downloading deadline, last time processing including destination path
Time and last time file size.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810778031.8A CN108881477B (en) | 2018-07-16 | 2018-07-16 | Distributed file acquisition monitoring method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810778031.8A CN108881477B (en) | 2018-07-16 | 2018-07-16 | Distributed file acquisition monitoring method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108881477A true CN108881477A (en) | 2018-11-23 |
| CN108881477B CN108881477B (en) | 2020-09-29 |
Family
ID=64302415
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810778031.8A Active CN108881477B (en) | 2018-07-16 | 2018-07-16 | Distributed file acquisition monitoring method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108881477B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109862075A (en) * | 2018-12-29 | 2019-06-07 | 北京奥鹏远程教育中心有限公司 | A kind of method for routing of Redis Service Instance |
| CN110336889A (en) * | 2019-07-15 | 2019-10-15 | 山东省气象科学研究所 | A kind of numerical weather prediction model operation intelligent monitoring platform and monitoring method |
| CN110377488A (en) * | 2019-07-15 | 2019-10-25 | 福建威盾科技集团有限公司 | A kind of method and system for unifying O&M and dynamic expansion |
| CN110995832A (en) * | 2019-11-29 | 2020-04-10 | 安徽江淮汽车集团股份有限公司 | Vehicle data monitoring method and system |
| CN111416842A (en) * | 2020-03-06 | 2020-07-14 | 科大讯飞股份有限公司 | Automatic resource cluster distribution and hot update system and method |
| CN111596950A (en) * | 2020-05-15 | 2020-08-28 | 博易智软(北京)技术有限公司 | Distributed data development engine system |
| CN111769982A (en) * | 2020-06-22 | 2020-10-13 | 上海理想信息产业（集团）有限公司 | Large-scale network data acquisition method and device based on timeout factor |
| CN112235361A (en) * | 2020-09-28 | 2021-01-15 | 青海绿能数据有限公司 | Photovoltaic power plant data switching platform |
| CN114285588A (en) * | 2020-09-21 | 2022-04-05 | 奇安信科技集团股份有限公司 | Method, device, equipment and storage medium for acquiring attack object information |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8683016B1 (en) * | 2002-12-20 | 2014-03-25 | Versata Development Group, Inc. | Data recording components and processes for acquiring selected web site data |
| CN105119913A (en) * | 2015-08-13 | 2015-12-02 | 东南大学 | Web server architecture based on Docker and interactive method between modules |
| CN105449846A (en) * | 2014-08-29 | 2016-03-30 | 国家电网公司 | DC online monitoring and management system and management method |
| CN107945496A (en) * | 2017-11-28 | 2018-04-20 | 苏州合利美电子科技有限公司 | A kind of data acquisition monitoring system based on intelligent electric meter |
-
2018
- 2018-07-16 CN CN201810778031.8A patent/CN108881477B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8683016B1 (en) * | 2002-12-20 | 2014-03-25 | Versata Development Group, Inc. | Data recording components and processes for acquiring selected web site data |
| CN105449846A (en) * | 2014-08-29 | 2016-03-30 | 国家电网公司 | DC online monitoring and management system and management method |
| CN105119913A (en) * | 2015-08-13 | 2015-12-02 | 东南大学 | Web server architecture based on Docker and interactive method between modules |
| CN107945496A (en) * | 2017-11-28 | 2018-04-20 | 苏州合利美电子科技有限公司 | A kind of data acquisition monitoring system based on intelligent electric meter |
Non-Patent Citations (1)
| Title |
|---|
| 冷国宁: ""分布式业务系统监控系统设计与应用"", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109862075A (en) * | 2018-12-29 | 2019-06-07 | 北京奥鹏远程教育中心有限公司 | A kind of method for routing of Redis Service Instance |
| CN109862075B (en) * | 2018-12-29 | 2022-05-03 | 北京奥鹏远程教育中心有限公司 | Routing method of Redis service instance |
| CN110336889B (en) * | 2019-07-15 | 2021-05-25 | 山东省气象科学研究所 | Intelligent monitoring platform and monitoring method for operation in numerical weather forecast mode |
| CN110336889A (en) * | 2019-07-15 | 2019-10-15 | 山东省气象科学研究所 | A kind of numerical weather prediction model operation intelligent monitoring platform and monitoring method |
| CN110377488A (en) * | 2019-07-15 | 2019-10-25 | 福建威盾科技集团有限公司 | A kind of method and system for unifying O&M and dynamic expansion |
| CN110995832A (en) * | 2019-11-29 | 2020-04-10 | 安徽江淮汽车集团股份有限公司 | Vehicle data monitoring method and system |
| CN111416842A (en) * | 2020-03-06 | 2020-07-14 | 科大讯飞股份有限公司 | Automatic resource cluster distribution and hot update system and method |
| CN111596950A (en) * | 2020-05-15 | 2020-08-28 | 博易智软(北京)技术有限公司 | Distributed data development engine system |
| CN111769982A (en) * | 2020-06-22 | 2020-10-13 | 上海理想信息产业（集团）有限公司 | Large-scale network data acquisition method and device based on timeout factor |
| CN111769982B (en) * | 2020-06-22 | 2023-03-24 | 上海理想信息产业（集团）有限公司 | Large-scale network data acquisition method and device based on timeout factor |
| CN114285588A (en) * | 2020-09-21 | 2022-04-05 | 奇安信科技集团股份有限公司 | Method, device, equipment and storage medium for acquiring attack object information |
| CN112235361A (en) * | 2020-09-28 | 2021-01-15 | 青海绿能数据有限公司 | Photovoltaic power plant data switching platform |
| CN112235361B (en) * | 2020-09-28 | 2022-12-27 | 青海绿能数据有限公司 | Photovoltaic power plant data switching platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108881477B (en) | 2020-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108881477A (en) | A method of it is acquired and is monitored based on distributed file | |
| JP7460237B2 (en) | Distributed Industrial Performance Monitoring and Analysis | |
| US10353918B2 (en) | High availability and disaster recovery in large-scale data warehouse | |
| US11507594B2 (en) | Bulk data distribution system | |
| RU2688451C1 (en) | Industrial automation system for data storage in industrial production medium, method for storage data and intelligent programmable logic controller | |
| CA2835446C (en) | Data analysis system | |
| CN105653425B (en) | Monitoring system based on complex event processing engine | |
| US10503145B2 (en) | System and method for asset fleet monitoring and predictive diagnostics using analytics for large and varied data sources | |
| CN110651265A (en) | Data replication system | |
| US11960443B2 (en) | Block data storage system in an event historian | |
| CN109918349A (en) | Log processing method, device, storage medium and electronic device | |
| KR102508817B1 (en) | High availability distribution intelligence system using message transmission bus | |
| Fu et al. | Real-time data infrastructure at uber | |
| CN104268061A (en) | Storage state monitoring mechanism for virtual machine | |
| US20160171055A1 (en) | Data query interface system in an event historian | |
| CN112099989A (en) | Disaster recovery, migration and recovery method for Kubernetes cloud native application | |
| CN110019138A (en) | A kind of transmission table space Autonomic Migration Framework method and system based on Zabbix | |
| KR20150118963A (en) | Queue monitoring and visualization | |
| Bautista et al. | Shasta log aggregation, monitoring and alerting in HPC environments with Grafana Loki and ServiceNow | |
| Sureddy et al. | A Framework for Monitoring Data Warehousing Applications | |
| Zhang et al. | A big data framework for spacecraft prognostics and health monitoring | |
| US20160171107A1 (en) | Data dictionary system in an event historian | |
| CN111488321A (en) | Management system for storage volume | |
| Xiao et al. | NMSTREAM: A scalable event-driven ETL framework for processing heterogeneous streaming data | |
| US10848546B2 (en) | Direct binary file transfer based network management system free of messaging, commands and data format conversions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |