CN115208930A - Monitoring system and monitoring method based on Kappa architecture - Google Patents
Monitoring system and monitoring method based on Kappa architecture Download PDFInfo
- Publication number
- CN115208930A CN115208930A CN202210422872.1A CN202210422872A CN115208930A CN 115208930 A CN115208930 A CN 115208930A CN 202210422872 A CN202210422872 A CN 202210422872A CN 115208930 A CN115208930 A CN 115208930A
- Authority
- CN
- China
- Prior art keywords
- data
- module
- monitoring
- subsystem
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 15
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000005516 engineering process Methods 0.000 claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000007726 management method Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 3
- 235000014510 cooky Nutrition 0.000 claims description 3
- 238000013480 data collection Methods 0.000 claims description 3
- 230000002085 persistent effect Effects 0.000 claims description 3
- 230000003068 static effect Effects 0.000 claims description 3
- 238000013523 data management Methods 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 238000004364 calculation method Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/141—Setup of application sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/163—In-band adaptation of TCP data exchange; In-band control procedures
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a monitoring system and a monitoring method based on a Kappa framework, comprising a lower subsystem, a middle subsystem and an upper subsystem; the lower subsystem consists of a data acquisition device, a data communicator and a thread controller, and the related contents comprise a Zookeeper monitoring service, a Netty client service and a Future asynchronous callback technology; the middle-layer subsystem consists of a data receiver and a data classifier, and the related contents comprise Netty server-side service, zookeeper distributed coordination service and a Kafka message engine; the upper subsystem consists of a monitoring terminal and a data sensor, and the related technology comprises a SpringBoot server and a Flink stream processing technology. By applying the technical scheme, the problem of packet loss generated by a monitoring system developed based on a UDP protocol and the problem of transmission data packet redundancy generated by the monitoring system developed based on the HTTP protocol can be solved, and the advantages of a Kappa framework can be exerted.
Description
Technical Field
The invention relates to the technical field of server monitoring systems, in particular to a monitoring system and a monitoring method based on a Kappa framework.
Background
The Kappa architecture is a real-time analysis stream processing architecture in a software engineering system architecture, and the core part of the Kappa architecture mainly has two main points: a stream computation engine and a message engine. Compared with the Lambda architecture, the Kappa architecture is simple, takes stream data as the center, has less development workload, and does not need to additionally develop and maintain a huge ETL system and various intermediate report batch processing service systems. However, the system designed by the Kappa architecture also has certain defects that the functionality is not as comprehensive as that of the Lambda architecture, and the system is more suitable for processing various data sources taking time as an important index.
In the Kappa-based stream processing computing service, we generally consider three time indicators, namely event time, entry time, and processing time. The event time refers to the specific time when an event occurs, the entry time refers to the specific time when an event record flows to the message engine, and finally, the processing time refers to the specific time when the event record is formally processed by the stream computing architecture. Most of services of the monitoring system are related to event time, and a large amount of time sequence data can be used as important indexes of service processing, so that the Kappa architecture can be used as a system design architecture of the monitoring system.
The transport layer protocol used by the monitoring system in the market at present mainly comprises a TCP protocol, a UDP protocol and an HTTP protocol, but 3 protocol methods, except the TCP protocol, the UDP protocol and the HTTP protocol have certain defects when used as the transport layer protocol of the monitoring system: the UDP protocol is a connectionless and unreliable transport layer protocol and is characterized in that under the conditions of poor network environment and overlarge load, the warning message cannot be ensured to reach a target host within the specified time; the HTTP protocol contains associated HTTP header information in each packet. These headers are often redundant and useless in monitoring streaming data traffic.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a monitoring system and a monitoring method based on a Kappa architecture, which can alleviate the problem of packet loss generated by a monitoring system developed based on a UDP protocol and the problem of redundancy of transmitted data packets generated by a monitoring system developed based on an HTTP protocol, and can also exert the advantages of the Kappa architecture.
In order to achieve the purpose, the invention adopts the following technical scheme: a monitoring system based on Kappa architecture comprises a lower subsystem, a middle subsystem and an upper subsystem; the lower subsystem consists of a data acquisition device, a data communicator and a thread controller, and the related contents comprise a Zookeeper monitoring service, a Netty client service and a Future asynchronous callback technology; the middle subsystem consists of a data receiver and a data classifier, and the related contents comprise Netty service end service, zookeeper distributed coordination service and a Kafka message engine; the upper subsystem consists of a monitoring terminal and a data sensor, and the related technology comprises a SpringBoot server and a Flink stream processing technology.
In a preferred embodiment, the data collector comprises a configuration file IO module, a log file reading module and a local data management module;
the data communicator comprises a Netty client module, a log file version cache module and a Zookeeper node tree module;
the thread controller comprises a thread distribution module, a thread control module, a script driving module and a thread state rollback module.
In a preferred embodiment, the data receiver comprises a Netty server module and a data preprocessing module;
the data classifier comprises a data classification module and a Kafka issuing end module.
In a preferred embodiment, the data sensor comprises a flow calculation node, a cluster performance calculation node, an index baseline alarm calculation node and each specific service analysis node;
the monitoring terminal comprises a Kafka message subscription end module, a view layer interface module, a cluster service scheduling module, a resource information management module and a monitoring terminal Zookeeper node tree module.
The invention also provides a monitoring method based on the Kappa architecture, which comprises the following steps:
step 1: monitoring data collection, wherein a lower subsystem deployed at a monitored node can regularly collect OS monitoring data, application program log data and service logic log data of the server, and sends the data to a middle subsystem located in an event routing server through a Netty client in a JSON data format; meanwhile, local service data directly from the service client is also transmitted to the middle-layer subsystem through the Cookie;
and 2, step: monitoring data preprocessing, wherein a middle subsystem corrects various collected monitoring stream data and data directly from a user client through a data preprocessing module according to a strategy, corrects missing data and filtered useless data, and collects the processed stream data into a data classifier module of the local computer;
and step 3: monitoring data classification, wherein a data classifier distributes the data to different Kafka subjects according to different categories, emergency degrees and specific functional requirements according to the data processing result and the type of JSON marks of source data;
and 4, step 4: monitoring data processing, namely using a flash cluster based on yarn management to pull data from Kakfa, analyzing various data in real time to generate view information, generating notification information according to an index baseline designed in advance, and distributing different processing results to different Kafka themes;
and 5: storing view layer data and notification data, wherein for the view layer data which can be directly transmitted to a view layer for display, a monitoring terminal developed by using SpringBoot can acquire related data by monitoring a specified KafkaTopic and store the data into a local cache maintained by a JVM (JVM); for the notification data, if the notification data is the notification data of the scheduling class, entering step 7 for processing, and if the notification data is other notification data, caching the notification data to a persistent data layer;
step 6: the method comprises the steps that the data of the view layer are displayed, a front-end system of a monitoring system can obtain static data through an HTTP (hyper text transport protocol) interface, and obtain continuous stream data through a webscolet technology; meanwhile, the monitoring terminal also provides a control interface;
and 7: applying service scheduling, wherein for scheduling notification information, the monitoring terminal can acquire scheduling information by monitoring the appointed Kafkatopic, and also can acquire the scheduling information from a front-end system, and convert the data into corresponding operation on the Zookeeper node tree to change a specific node; and after the lower-layer subsystem monitors the change of the node, the thread controller module is called to schedule the monitored application service.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a monitoring system and a monitoring method based on a Kappa framework, which start from a network transmission layer protocol at the bottom layer, use a TCP protocol, avoid the packet loss problem of a UDP protocol and the redundant data problem of the HTTP protocol, simultaneously combine with a Reactor design mode, and improve the data transmission quantity between servers in a mode of increasing the number of communication threads under the condition of bandwidth allowance. Important information of each server of the monitored cluster can be stored in the Zookeeper, and the information can be redundantly stored in the cluster, so that the problem of memory data loss caused by single-point failure is solved. Due to the partition allocation algorithm of Kafka, data can be evenly distributed and stored in the message engine, and to a certain extent, the data skew phenomenon is prevented from occurring.
Drawings
FIG. 1 is a deployment plan of the various subsystems of the preferred embodiment of the present invention;
FIG. 2 is an exemplary table of lower subsystem functional modules in accordance with a preferred embodiment of the present invention;
FIG. 3 is an exemplary table of the functional modules of the middle subsystem in accordance with the preferred embodiment of the present invention;
fig. 4 is an exemplary table of functional modules of the upper subsystem monitoring terminal system in accordance with the preferred embodiment of the present invention;
FIG. 5 is an exemplary table of sensor function modules of the upper subsystem in accordance with a preferred embodiment of the present invention;
FIG. 6 is a table illustrating the design of Zookeeper nodes in accordance with the preferred embodiment of the present invention;
FIG. 7 is a diagram of an exemplary data sensor stream processing topology in accordance with a preferred embodiment of the present invention;
FIG. 8 is a Kafka theme service storage design example table for each channel in accordance with the preferred embodiment of the present invention;
FIG. 9 is a flow chart of the lower subsystem start-up of the preferred embodiment of the present invention;
fig. 10 is a flow chart of data interaction of the monitoring system according to the preferred embodiment of the present invention.
Detailed Description
The invention is further explained below with reference to the drawings and the embodiments.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application; as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
A monitoring system based on Kappa architecture, referring to fig. 1 to 10, includes a lower subsystem, a middle subsystem and an upper subsystem; the lower subsystem consists of a data acquisition device, a data communicator and a thread controller, and the related contents comprise a Zookeeper monitoring service, a Netty client service and a Future asynchronous callback technology; the middle subsystem consists of a data receiver and a data classifier, and the related contents comprise Netty service end service, zookeeper distributed coordination service and a Kafka message engine; the upper subsystem consists of a monitoring terminal and a data sensor, and the related technology comprises a SpringBoot server and a Flink stream processing technology.
The data acquisition unit comprises a configuration file IO module, a log file reading module and a local data management program module; the data communicator comprises a Netty client module, a log file version cache module and a Zookeeper node tree module; the thread controller comprises a thread distribution module, a thread control module, a script driving module and a thread state rollback module. The data receiver comprises a Netty server module and a data preprocessing module; the data classifier comprises a data classification module and a Kafka issuing end module.
The data sensor comprises a flow calculation node, a cluster performance calculation node, an index baseline alarm calculation node and each specific service analysis node; the monitoring terminal comprises a Kafka message subscriber module, a view layer interface module, a cluster service scheduling module, a resource information management module and a monitoring terminal Zookeeper node tree module.
A monitoring method based on a Kappa architecture comprises the following steps:
step 1: monitoring data collection, wherein a lower subsystem deployed at a monitored node can regularly collect OS monitoring data, application program log data and service logic log data of the server, and sends the data to a middle subsystem located in an event routing server through a Netty client in a JSON data format; meanwhile, local service data directly from the service client is also transmitted to the middle-layer subsystem through the Cookie;
step 2: monitoring data preprocessing, wherein a middle subsystem corrects various collected monitoring stream data and data directly from a user client through a data preprocessing module according to a strategy, corrects missing data and filtered useless data, and collects the processed stream data into a data classifier module of the local computer;
and step 3: monitoring data classification, wherein a data classifier distributes the data to different Kafka topics according to different categories, emergency degrees and specific functional requirements according to the data processing result and the type of JSON marks of source data;
and 4, step 4: monitoring data processing, namely using a flash cluster based on yarn management to pull data from Kakfa, analyzing various data in real time to generate view information, generating notification information according to an index baseline designed in advance, and distributing different processing results to different Kafka themes;
and 5: storing view layer data and notification data, wherein for the view layer data which can be directly transmitted to a view layer for display, a monitoring terminal developed by using SpringBoot can acquire related data by monitoring a specified KafkaTopic and store the data into a local cache maintained by a JVM (JVM); for the notification data, if the notification data is the notification data of the scheduling class, the step 7 is carried out for processing, and if the notification data is other notification data, the notification data is cached to a persistent data layer;
step 6: the method comprises the steps that the data of the view layer are displayed, a front-end system of a monitoring system can obtain static data through an HTTP (hyper text transport protocol) interface, and obtain continuous stream data through a webscolet technology; meanwhile, the monitoring terminal also provides a control interface;
and 7: the application service scheduling, for the scheduling notification information, the monitoring terminal can acquire the scheduling information by monitoring the designated Kafkatopic, can also acquire the scheduling information from a front-end system, and converts the data into corresponding operation on the Zookeeper node tree to change a specific node; and after the lower-layer subsystem monitors the change of the node, the thread controller module is called to schedule the monitored application service.
The invention is applied to a Netty framework, which uses a ByteBuf buffer of a Netty program to perform 'real' read-write operation, the ByteBuf buffer is not positioned in a JVM (Java virtual machine) stack, but is created by a byteBufAllocator single instance through system call of a related malloc () function, so that the byteBuf buffer is also called a Direct byteBuf (Direct byteBuf) and has better performance on write service than a HeapbyteBuf and a DirectteBuf stored in the JVM stack. Considering that the creating and destroying expense of the DirectByteBuf is large, the problem of memory overflow is the same as that of the HeaptByteBuf, and the lower-layer subsystem is mainly used for writing services, the invention proposes to adopt a pooling mode when applying for creating the DirectByteBuf buffer. The Netty program pooling distributor is used for self-management, and the memory overflow can be effectively avoided to a certain extent in a big data environment.
The invention is applied to the Zookeeper distributed coordination system, and the maintained registration information of the system is completely cached in the memory of each Zookeeper server and is stored in a tree structure. Therefore, the official document of the Zookeeper explicitly mentions that the size of each node of the Zookeeper storage tree should not be 1M, and the height of the node tree should not be too large, otherwise the performance of the Zookeeper coordination service may be affected. Therefore, in practical development, the storage content of the Zookeeper node tree needs to be reasonably planned.
The invention is applied to a Kafka message engine architecture, and as all actual read-write operations of Kafka are executed on partitionLeaders, the Kafka has an intelligent partitionLeader election algorithm which can distribute the partitionLeaders to different physical nodes. Therefore, the data load balancing service of the message engine layer is realized, and the problem of data inclination is avoided to a certain extent. If other message engine systems are to be considered, it is ensured that the message engine is at least provided with stream replay techniques, otherwise it would be difficult for a system developed based on the Kappa architecture to perform off-line computation related services.
Before step 1 is executed, the following work needs to be done in sequence: (1) and initializing each module of the lower subsystem. (2) And initializing the connection between the client of the Curator and the Zookeeper server, and registering configuration information. The monitoring terminal can monitor the registration action of the lower subsystem in time and update the local cache content of the monitoring terminal. (3) Initializing a thread controller, inquiring application service information required to be started from the Zookeeper, and sequentially starting application programs required to be monitored. (4) A worker thread listening to the Redis publish/subscribe channel is initialized. (5) Starting a Netty client through a thread controller; after all the steps are completed, the data acquisition unit can normally start working, acquire the required monitoring data and distribute the monitoring data to the middle-layer subsystem.
The starting service of the whole set of system also has a precondition, namely, the Zookeeper cluster and the Kafka cluster must be started first and run normally, otherwise, the information interaction among the 3 subsystems cannot be completed, and the monitoring service cannot be realized normally.
The invention provides a monitoring system and a monitoring method based on a Kappa framework, which start from a network transmission layer protocol at the bottom layer, use a TCP protocol, avoid the packet loss problem of a UDP protocol and the redundant data problem of the HTTP protocol, simultaneously combine with a Reactor design mode, and improve the data transmission quantity between servers in a mode of increasing the number of communication threads under the condition of bandwidth allowance. Important information of each server of the monitored cluster can be stored in the Zookeeper, and the information can be redundantly stored in the cluster, so that the problem of memory data loss caused by single-point failure is solved. Due to the partition allocation algorithm of Kafka, data can be evenly distributed and stored in the message engine, and to a certain extent, the data skew phenomenon is prevented from occurring.
Claims (5)
1. A monitoring system based on a Kappa architecture is characterized in that: comprises a lower subsystem, a middle subsystem and an upper subsystem; the lower subsystem consists of a data acquisition device, a data communicator and a thread controller, and the related contents comprise a Zookeeper monitoring service, a Netty client service and a Future asynchronous callback technology; the middle-layer subsystem consists of a data receiver and a data classifier, and the related contents comprise Netty server-side service, zookeeper distributed coordination service and a Kafka message engine; the upper subsystem consists of a monitoring terminal and a data sensor, and the related technology comprises a SpringBoot server and a Flink stream processing technology.
2. The monitoring system based on the Kappa architecture as claimed in claim 1, wherein the data collector includes a configuration file IO module, a log file reading module and a local data management module;
the data communicator comprises a Netty client module, a log file version cache module and a Zookeeper node tree module;
the thread controller comprises a thread distribution module, a thread control module, a script driving module and a thread state rollback module.
3. The Kappa-based monitoring system as claimed in claim 2, wherein the data receiver includes a Netty server module and a data preprocessing module;
the data classifier comprises a data classification module and a Kafka issuing end module.
4. The Kappa architecture-based monitoring system of claim 3, wherein the data sensors include traffic computation nodes, cluster performance computation nodes, indicator baseline alarm computation nodes, and individual traffic analysis-specific nodes;
the monitoring terminal comprises a Kafka message subscription end module, a view layer interface module, a cluster service scheduling module, a resource information management module and a monitoring terminal Zookeeper node tree module.
5. A monitoring method based on Kappa architecture, characterized in that a monitoring system based on Kappa architecture as claimed in any one of claims 1 to 4 is adopted, comprising the following steps:
step 1: monitoring data collection, wherein a lower subsystem deployed at a monitored node can regularly collect OS monitoring data, application program log data and service logic log data of the server, and sends the data to a middle subsystem located in an event routing server through a Netty client in a JSON data format; meanwhile, local service data directly from the service client is also transmitted to the middle-layer subsystem through the Cookie;
step 2: monitoring data preprocessing, namely, a middle subsystem corrects various collected monitoring stream data and data directly from a user client through a data preprocessing module according to a strategy and corrects missing data and filtered useless data, and meanwhile, the processed stream data is collected into a data classifier module of the local computer;
and step 3: monitoring data classification, wherein a data classifier distributes the data to different Kafka subjects according to different categories, emergency degrees and specific functional requirements according to the data processing result and the type of JSON marks of source data;
and 4, step 4: monitoring data processing, namely using a flash cluster based on yarn management to pull data from Kakfa, analyzing various data in real time to generate view information, generating notification information according to an index baseline designed in advance, and distributing different processing results to different Kafka themes;
and 5: storing view layer data and notification data, wherein for the view layer data which can be directly transmitted to a view layer for display, a monitoring terminal developed by using SpringBoot can acquire related data by monitoring a specified KafkaTopic and store the data into a local cache maintained by a JVM (JVM); for the notification data, if the notification data is the notification data of the scheduling class, the step 7 is carried out for processing, and if the notification data is other notification data, the notification data is cached to a persistent data layer;
step 6: the method comprises the steps that the data of the view layer are displayed, a front-end system of a monitoring system can obtain static data through an HTTP (hyper text transport protocol) interface, and obtain continuous stream data through a webscolet technology; meanwhile, the monitoring terminal also provides a control interface;
and 7: applying service scheduling, wherein for scheduling notification information, the monitoring terminal can acquire scheduling information by monitoring the specified Kafka topoic, can also acquire the scheduling information from a front-end system, and converts the data into corresponding operation on the Zookeeper node tree to change a specific node; and after the lower-layer subsystem monitors the change of the node, the thread controller module is called to schedule the monitored application service.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210422872.1A CN115208930A (en) | 2022-04-21 | 2022-04-21 | Monitoring system and monitoring method based on Kappa architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210422872.1A CN115208930A (en) | 2022-04-21 | 2022-04-21 | Monitoring system and monitoring method based on Kappa architecture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115208930A true CN115208930A (en) | 2022-10-18 |
Family
ID=83575130
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210422872.1A Pending CN115208930A (en) | 2022-04-21 | 2022-04-21 | Monitoring system and monitoring method based on Kappa architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115208930A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105791431A (en) * | 2016-04-26 | 2016-07-20 | 北京邮电大学 | On-line distributed monitoring video processing task scheduling method and device |
| CN109271412A (en) * | 2018-09-28 | 2019-01-25 | 中国-东盟信息港股份有限公司 | The real-time streaming data processing method and system of smart city |
| CN109714192A (en) * | 2018-11-29 | 2019-05-03 | 深圳供电局有限公司 | Monitoring method and system for monitoring cloud platform |
| US20190208365A1 (en) * | 2017-12-31 | 2019-07-04 | Bull Sas | System and method for managing a mass gathering |
| CN113609201A (en) * | 2021-08-10 | 2021-11-05 | 珍岛信息技术（上海）股份有限公司 | Service data processing method and system |
-
2022
- 2022-04-21 CN CN202210422872.1A patent/CN115208930A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105791431A (en) * | 2016-04-26 | 2016-07-20 | 北京邮电大学 | On-line distributed monitoring video processing task scheduling method and device |
| US20190208365A1 (en) * | 2017-12-31 | 2019-07-04 | Bull Sas | System and method for managing a mass gathering |
| CN109271412A (en) * | 2018-09-28 | 2019-01-25 | 中国-东盟信息港股份有限公司 | The real-time streaming data processing method and system of smart city |
| CN109714192A (en) * | 2018-11-29 | 2019-05-03 | 深圳供电局有限公司 | Monitoring method and system for monitoring cloud platform |
| CN113609201A (en) * | 2021-08-10 | 2021-11-05 | 珍岛信息技术（上海）股份有限公司 | Service data processing method and system |
Non-Patent Citations (1)
| Title |
|---|
| 肖睿等: "基于Kappa架构的实时日志分析平台研究与实践", 《中国金融电脑》, 7 August 2021 (2021-08-07), pages 81 - 84 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107018042B (en) | Tracking method and tracking system for online service system | |
| CN109327509A (en) | A kind of distributive type Computational frame of the lower coupling of master/slave framework | |
| CN108471366A (en) | A kind of stereoscopic monitoring system of facing cloud native applications | |
| CN106790718A (en) | Service call link analysis method and system | |
| CN112118174B (en) | Software defined data gateway | |
| CN111970195B (en) | Data transmission method and streaming data transmission system | |
| CN103440244A (en) | Large-data storage and optimization method | |
| CN112003763B (en) | Network link monitoring method, monitoring device, monitoring equipment and storage medium | |
| CN107025222A (en) | A kind of distributed information log acquisition method and device | |
| CN106201754A (en) | Mission bit stream analyzes method and device | |
| CN101277304A (en) | Management system and management method for Web service operational environment based on rules | |
| CN108521339A (en) | A kind of reaction type node failure processing method and system based on cluster daily record | |
| CN105183470A (en) | Natural language processing systematic service platform | |
| CN112543222B (en) | Data processing method and device, computer equipment and storage medium | |
| CN108933798A (en) | Date storage method, storage server and system | |
| CN108599992A (en) | A kind of data processing system and method | |
| CN111984505B (en) | Fortune dimension data acquisition device and acquisition method | |
| CN113190524A (en) | Industrial big data acquisition method and system | |
| CN108900608A (en) | Robot real-time feedback system based on cloud platform | |
| CN112559634A (en) | Big data management system based on computer cloud computing | |
| CN114064211A (en) | Video stream analysis system and method based on end-edge-cloud computing architecture | |
| CN115102986B (en) | Internet of things data distribution and storage method and system in edge environment | |
| CN112965811B (en) | Optimization method of monitoring data and server | |
| CN115208930A (en) | Monitoring system and monitoring method based on Kappa architecture | |
| CN113873033B (en) | Intelligent edge computing gateway platform with fault-tolerant function |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |