CN110633186A - Log monitoring system for electric power metering micro-service architecture and implementation method - Google Patents
Log monitoring system for electric power metering micro-service architecture and implementation method Download PDFInfo
- Publication number
- CN110633186A CN110633186A CN201910760512.0A CN201910760512A CN110633186A CN 110633186 A CN110633186 A CN 110633186A CN 201910760512 A CN201910760512 A CN 201910760512A CN 110633186 A CN110633186 A CN 110633186A
- Authority
- CN
- China
- Prior art keywords
- log
- component
- data
- monitoring component
- monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000007405 data analysis Methods 0.000 claims abstract description 4
- 238000013079 data visualisation Methods 0.000 claims abstract description 4
- 238000004806 packaging method and process Methods 0.000 claims abstract description 4
- 238000013507 mapping Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000005611 electricity Effects 0.000 claims description 3
- 238000003780 insertion Methods 0.000 claims description 3
- 230000037431 insertion Effects 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 2
- 230000003993 interaction Effects 0.000 claims description 2
- 230000009467 reduction Effects 0.000 abstract description 2
- 238000004458 analytical method Methods 0.000 description 10
- 238000013467 fragmentation Methods 0.000 description 5
- 238000006062 fragmentation reaction Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
Abstract
The invention discloses a log monitoring system for a power metering microservice architecture and an implementation method, wherein the system comprises the following steps: the system comprises an Elasticissearch monitoring component, a Logstash monitoring component and a Kibana monitoring component; the Elasticisearch monitoring component adopts a search server based on Lucene to provide a full-text search engine with distributed multi-user capability, the Logstash monitoring component adopts a plug-in framework to collect, analyze and store the power metering data logs, and the Kibana monitoring component adopts a Web platform to provide data analysis and visualization for the Elasticisearch monitoring component; packaging and developing aiming at an Elasticissearch monitoring component, a Logstash monitoring component and a Kibana monitoring component, and matching the architecture mode of a log system of the power metering system. The invention uses the micro-service mode to deploy in each flow of collection, processing, storage and display of the log, supports dynamic capacity expansion and reduction, supports processing and storage of large-scale logs, and meets the log requirement of complex use scenes.
Description
Technical Field
The application relates to the technical field of power cloud computing, in particular to a log monitoring system for a power metering micro-service architecture and an implementation method.
Background
With the development of scientific technology and the popularization of technologies such as cloud computing, the global data volume shows explosive growth, and particularly, with the arrival of a big data era, users make massive data log information through the internet. In the power grid industry, the huge amount of log information generated every day reaches exponential level in various data types such as electricity consumption data, charging data, rule data and the like collected by cities and provinces and regions. The traditional relational database has limited capacity for storing, querying and analyzing the logs, cannot meet the requirement of the explosively increased mass data logs, and cannot solve the problems of information storage capacity, data safety, log searching and analyzing and the like.
Disclosure of Invention
In order to overcome the defects and shortcomings of the prior art, the invention provides a log monitoring system for an electric power metering microservice architecture and an implementation method thereof, which can meet increasingly huge data calculation requirements and business requirements, can explore potential risks in advance by using the log monitoring system, analyze, judge and form qualitative or quantitative descriptions, so that the risks are reduced by taking corresponding measures, the metering system adopts a multi-task distributed technology to analyze and excavate massive logs, and applies analysis methods such as rule association, statistical association and the like, so that the analysis depth of the logs in the power grid industry and the identification accuracy of events are further improved.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a log monitoring system for a power metering microservice architecture, which comprises:
the system comprises an Elasticissearch monitoring component, a Logstash monitoring component and a Kibana monitoring component;
the Elasticisearch monitoring component adopts a search server based on Lucene and is used for providing a full-text search engine with distributed multi-user capability, the Logstash monitoring component adopts a plug-in architecture and is used for collecting, analyzing and storing an electric power metering data log, and the Kibana monitoring component adopts a Web platform and is used for providing data analysis and visualization for the Elasticisearch monitoring component;
packaging and developing aiming at an Elasticissearch monitoring component, a Logstash monitoring component and a Kibana monitoring component, and matching the architecture mode of a log system of the power metering system.
As a preferred technical solution, the Elasticsearch component includes nodes and cluster nodes, Index, distributed system, RESTful support, document, id for identifying document, field, and mapping.
As a preferred technical solution, the logstack monitoring component includes a log data sending component, a data collecting component, and a data writing component.
The invention also provides an implementation method for the log monitoring system of the electric power metering microservice architecture, which comprises the following steps:
and (3) metering system log Request generation and transmission: the method comprises the steps that each micro-service application of a metering system uniformly configures a request interceptor Filter, when an http request is intercepted, the interceptor firstly generates a request ID, then collects calling information and user information of the application and encrypts and outputs the calling information and the user information to a log;
log collection of a metering system: each micro service application server is provided with a log acquisition module which is used for acquiring application logs on each server, wherein the log acquisition module comprises a load balancing layer and a micro service gateway access layer;
log caching of the metering system: dumping the log of the metering system in a preset redis component;
log processing of the metering system: setting a logstack index end to read log data from a redis component, filtering and storing the log data into an ElasticSearch component;
log storage of the metering system: storing the metering system log in an ElasticSearch component, and deploying the ElasticSearch component in a cluster mode;
log display of the metering system; and the Kibana monitoring component searches in the index of the ElasticSearch component, performs data interaction and displays the log table graphs of various dimensions.
As a preferred technical solution, the specific step of dumping the log of the metering system in a preset redis component is as follows:
in redis, data is cached in a user-defined name of a key in a unified manner, a List data type is adopted as a buffer area queue for log collection, the List type adopts a character string linked List ordered according to an insertion sequence, and a message queue is completed by adopting Push/Pop operation of a List.
As a preferred technical solution, the log processing of the metering system further includes a data format conversion step.
As a preferred technical scheme, the logging of the metering system is stored in an ElasticSearch component, and the specific steps are as follows:
yml defines cluster name by setting cluster of the elasticsearch.yml, then sets node for distinguishing cluster node of each Elasticsearch component, which is divided into two types: node.master and node.data;
master: when the value is set to true, the current node becomes a management node of the cluster and is used for maintaining metadata and managing the state of each node of the cluster;
data: when set to true, the current node becomes a data node for storage, querying, and importing of data.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) the invention uses the micro-service mode to deploy in each flow of collection, processing, storage and display of the log, supports dynamic capacity expansion and reduction, supports processing and storage of large-scale logs, and meets the log requirement of complex use scenes.
(2) The method adopts the log monitoring system to explore, analyze, judge and form qualitative or quantitative description in advance, so as to reduce the risk by adopting counter measures, adopts the multitask distributed technology to analyze and excavate massive logs by the metering system, applies analysis methods such as rule association, statistical association and the like, can establish a scientific analysis model, and further improves the analysis depth of the logs in the power grid industry and the identification accuracy of events.
Drawings
Fig. 1 is a schematic diagram of a cluster node used in a log monitoring system of a power metering microservice architecture according to the present embodiment;
fig. 2 is a schematic processing flow diagram of the Logstash monitoring component according to this embodiment;
FIG. 3 is an architecture diagram of an implementation method of the log monitoring system according to this embodiment;
fig. 4 is a schematic diagram illustrating a process flow of generating and transferring a log Request of a metering system according to the present embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Examples
Introduction of metering system microservices is an architectural style for building distributed systems, which means that a single small-sized but business-functional service is developed, each service has its own processing and lightweight communication mechanism and can be deployed on a single or multiple servers. Micro-services also refer to a loosely coupled, service-oriented architecture with some bounded context.
The embodiment provides a log monitoring system for a power metering microservice architecture, which includes:
a metering system log monitoring component-elastic search, a metering system log monitoring component-Logstash and a metering system log monitoring component-Kibana;
the main characteristics are as follows:
1: the processing mode is flexible: elastic search is a real-time full-text index;
2: and (3) linear cluster expansion: whether the Elasticissearch cluster or the Logstash cluster is linearly expandable;
3: the front end is operated dazzlingly: on the Kibana interface, the functions of searching and aggregation can be completed only by clicking a mouse, and a dazzling instrument panel is generated;
4: the retrieval performance is high-efficient: although each query is calculated in real time, the excellent design and implementation can basically achieve the second-level response of the data query all day long;
5: the configuration is simple and easy to operate: the Elasticisch adopts JSON interfaces, and Logstash is a Ruby DSL design and is the most common configuration syntax design in the industry at present; in a metering system log system, an elastic search, Logstash and Kibana open source suite is used as a log system architecture component, and a series of packaging development is carried out on the basis, so that the metering system log system is suitable for an architecture mode of a power grid metering system log system;
metering system log monitoring component-elastic search: the Elasticsearch is a Lucene-based search server. It provides a distributed multi-user capable full-text search engine based on RESTful web interface. The method is used for cloud computing, and has the characteristics of real-time searching, stability, reliability, quickness, convenience in installation and use, zero configuration, Cluster supporting, automatic discovery, automatic fragmentation of indexes, an Index copy mechanism, multiple data sources, automatic searching load and the like, wherein the Elasticissearch component comprises nodes (nodes) and Cluster (Cluster) nodes, Index indexes, distributed system shard ES, RESTful supporting, document identifying id, field and mapping;
wherein, a Node (Node) and a Cluster (Cluster) Node are one elastic instance in the Cluster, and as shown in fig. 1, the Cluster is a group of nodes having a common Cluster name. One of the nodes is an ES process, a plurality of nodes form a cluster, each node generally runs on different operating systems, the ES can automatically form the cluster after relevant cluster parameters are configured, a master node is selected in the cluster through a master selection algorithm of the ES, the cluster can be operated through any node outside, and the cluster has no division of master and slave nodes, is equal in appearance and decentralized and is beneficial to client programming;
index: the index has two layers of meanings in the ES, and as a verb, the index refers to a process of saving a document into the ES, and after indexing the document, a user can search the document by using the ES; as a noun, it refers to a place where a document is saved, and is equivalent to a database concept in a relational database, and a cluster may contain multiple indexes;
slicing and shrard: the ES is a distributed system that stores indices by selecting the appropriate "Primary Shard" (Primary Shard) to store the indices into. The division of the segments is fixed and has to be determined at the time of installation, which cannot be changed later. Since there is a master slice, there must be a "slave" slice, referred to in the ES as a "copy slice" (Replica Shard). Copy fragmentation has two main functions: high availability: if a certain fragmentation node is hung, other copy fragmentation nodes can be walked, and the fragmentation data on the node can be restored to be balanced through other nodes: the ES automatically controls the search route according to the load condition, and the copy fragments can equally share the load;
RESTful support: the ES supports RESTful access, and the HTTP interface of the ES not only can carry out business operation (indexing/searching), but also can carry out some configurations and the like;
document a document is a JSON text stored in es, which can be understood as a row in a relational database table, and each document is stored in an index, having a type and id. A document is a JSON object (hash/associative array in some languages) that contains 0 or more fields (key-value pairs). The original JSON text is stored in a _ source field after being indexed, and the default of a return value after the search is finished is that the return value contains the field;
id is used to identify documents, and the index/type/Id of a document must be unique. Document id is automatically generated (if not specified);
field fields a document contains several fields, or key-value pairs. The value of a field may be a simple (scalar) value (e.g., string, integer, date) or may be a nested structure, such as an array or object. One field is similar to a column in a relational database table. Each field map has a field type that describes the type of value that this field can hold, e.g., integer, string, object. The mapping may also define how the value of a field is analyzed.
mapping a mapping is similar to schema definition in a relational database. There is a mapping for each index that defines each type in the index, and the configuration to which the index is associated. The mapping may display the definition or be automatically created when the document is indexed.
As shown in fig. 2, the metering system log monitoring component-logstack: logstash is based on a command line interface and is oriented to task processing. The software architecture of the Logstash is a plug-in architecture with a 'pipeline-filter' style, has powerful plug-in functions, can collect and analyze logs, store the logs for later use, can access almost any data, can be combined with various external applications, and supports flexible expansion. logstash consists of three main parts: shipper-send log data, Broker-collect data, default built-in Redis and Indexer-data write;
the log monitoring component of the metering system, Logstash, is mainly characterized in that: almost any data can be accessed, and flexible extensions can be supported in conjunction with a variety of external applications.
An input stage: receiving data inflow from different sources, and configuring a codec plug-in unit for simple processing;
and (3) a filtering stage: filtering the inflow data and the like, and transmitting the inflow data to output, wherein an input stage and an output stage are necessary, and a filtering stage is optional;
an output stage: transmitting the data to a message queue, a file system and the like for further processing, and outputting the data to an ElasticSearch index in a log system of the ELK;
the processing tasks of the three phases are asynchronous, and the situation that the cross-phase tasks are executed in the same thread does not exist.
Metering system log monitoring component-Kibana: kibana is a Web platform based on Apache open source protocol, written in JavaScript language, providing analysis and visualization for the Elasticissearch, through which Kibana can query, browse and interact with data stored in the Elasticissearch index, and can also perform advanced data analysis and visualization on the data in various chart, table and map styles, and a browser-based interface enables a dashboard of modified Elasticissearch queries to be quickly created and displayed in real time.
Kibana makes it easy to understand a huge amount of data. The browser-based interface can quickly create and display the dashboard of the modified Elasticsearch query in real time, and Kibana has a simple configuration, and Kibana can be successfully installed and queried for Elasticsearch within a few minutes without any additional infrastructure.
As shown in fig. 3, the embodiment further provides an implementation method for a log monitoring system of a power metering microservice architecture, including the following steps:
generating and transmitting a log Request of a metering system, collecting a log of the metering system, caching the log of the metering system, processing the log of the metering system, storing the log of the metering system and displaying the log of the metering system;
as shown in fig. 4, the specific steps of generating and transmitting the metering system log Request include: the method comprises the steps that a request interceptor Filter is uniformly configured for various micro-service applications of a metering system, when an http request is intercepted, the interceptor firstly generates a request ID (which is globally unique and is not repeated), then collects call information and some user information of the applications and encrypts and outputs the call information and some user information to logs, the request ID is generated to form a call chain between application logs of the metering system, a unique request ID is generated for each request, under the condition that the log quantity of a power grid is huge, the logs with the request ID are subject to troubleshooting, and the logs generated by each request of a user are queried very conveniently and quickly;
log collection of a metering system: a log acquisition module is deployed on each micro-service application server, the log acquisition module is used for acquiring application logs on each server, and the application logs comprise load balance, a micro-service gateway access layer and the like, and the logstack loader defines a unique type for each application to generate a cable reference later;
log caching of the metering system: in the redis, data are uniformly cached in names with keys being logstack (keys can be defined), a List data type is adopted as a buffer queue for log collection, the List type is a character string linked List ordered according to an insertion sequence, the speed of inserting or deleting data from head to tail is very high no matter the data quantity is large, and a message queue can be realized by using the Push/Pop operation of the List;
log processing of the metering system: the log processing stage of the metering system is that the logstack index end reads log data from the redis, performs a series of filtering processes and then stores the log data into the ElasticSearch. In the processing process, data format conversion is carried out, fields such as extraction date and then conversion into a predefined format, extraction log level, RequestId and the like are extracted, message field information is extracted, the filtered field information is deleted or combined according to conditions, the ip location is obtained through GeoIP, filtering conditions are customized according to a system, regular analysis, Json analysis and the like are carried out; capturing a key word from the log information, generating an index of the ElasticSearch according to the type, and writing the log information into the index of the ElasticSearch;
log storage of the metering system: the metering system log is stored in the ElasticSearch, and the ElasticSearch is also deployed using a clustering approach. Name of the cluster is defined by setting cluster of the elastic search. yml, and the cluster names of a plurality of elastic search are to be identical, then name is set to distinguish each elastic search. The ElasticSearch cluster nodes are divided into two types: node. Master: when the node is set to true, the current node becomes a management node (i.e., a master node) of the cluster, and the main functions are to maintain metadata, manage the state of each node of the cluster, and obtain node.data: when the value is set to true, the current node becomes a data node and is mainly responsible for storing, inquiring and importing data;
log display of the metering system: the interface of the metering system log system uses a kibana open source framework, provides an analysis and visualization Web platform for the Elasticissearch, can search and interact data in the index of the Elasticissearch, generates table graphs of various dimensions, and displays the query result of the log, the quantity change trend graph of the log, various chart views generated by the log, an analysis instrument panel and the like.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (7)
1. A log monitoring system for a power metering microservice architecture, comprising:
the system comprises an Elasticissearch monitoring component, a Logstash monitoring component and a Kibana monitoring component;
the Elasticisearch monitoring component adopts a search server based on Lucene and is used for providing a full-text search engine with distributed multi-user capability, the Logstash monitoring component adopts a plug-in architecture and is used for collecting, analyzing and storing an electric power metering data log, and the Kibana monitoring component adopts a Web platform and is used for providing data analysis and visualization for the Elasticisearch monitoring component;
packaging and developing aiming at an Elasticissearch monitoring component, a Logstash monitoring component and a Kibana monitoring component, and matching the architecture mode of a log system of the power metering system.
2. The log monitoring system for electricity metering microservice architecture of claim 1, wherein the Elasticsearch component contains nodes and cluster nodes, Index, distributed systems, RESTful support, document, id identifying document, field, and mapping.
3. The log monitoring system for the electricity metering microservice architecture according to claim 1 or 2, wherein the logstack monitoring component comprises a log data sending component, a data collecting component and a data writing component.
4. An implementation method for a log monitoring system of a power metering microservice architecture is characterized by comprising the following steps:
and (3) metering system log Request generation and transmission: the method comprises the steps that each micro-service application of a metering system uniformly configures a request interceptor Filter, when an http request is intercepted, the interceptor firstly generates a request ID, then collects calling information and user information of the application and encrypts and outputs the calling information and the user information to a log;
log collection of a metering system: each micro service application server is provided with a log acquisition module which is used for acquiring application logs on each server, wherein the log acquisition module comprises a load balancing layer and a micro service gateway access layer;
log caching of the metering system: dumping the log of the metering system in a preset redis component;
log processing of the metering system: setting a logstack index end to read log data from a redis component, filtering and storing the log data into an ElasticSearch component;
log storage of the metering system: storing the metering system log in an ElasticSearch component, and deploying the ElasticSearch component in a cluster mode;
log display of the metering system; and the Kibana monitoring component searches in the index of the ElasticSearch component, performs data interaction and displays the log table graphs of various dimensions.
5. The implementation method for the power metering microservice architecture log monitoring system according to claim 4, wherein the specific step of dumping the log of the metering system in a preset redis component is:
in redis, data is cached in a user-defined name of a key in a unified manner, a List data type is adopted as a buffer area queue for log collection, the List type adopts a character string linked List ordered according to an insertion sequence, and a message queue is completed by adopting Push/Pop operation of a List.
6. The method of claim 4, wherein the log processing of the metering system further comprises a data format conversion step.
7. The implementation method for the power metering microservice architecture log monitoring system according to claim 4, wherein the metering system log is stored in an ElasticSearch component, and the specific steps are as follows:
yml defines cluster name by setting cluster of the elasticsearch.yml, then sets node for distinguishing cluster node of each Elasticsearch component, which is divided into two types: node.master and node.data;
master: when the value is set to true, the current node becomes a management node of the cluster and is used for maintaining metadata and managing the state of each node of the cluster;
data: when set to true, the current node becomes a data node for storage, querying, and importing of data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910760512.0A CN110633186A (en) | 2019-08-16 | 2019-08-16 | Log monitoring system for electric power metering micro-service architecture and implementation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910760512.0A CN110633186A (en) | 2019-08-16 | 2019-08-16 | Log monitoring system for electric power metering micro-service architecture and implementation method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110633186A true CN110633186A (en) | 2019-12-31 |
Family
ID=68970438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910760512.0A Pending CN110633186A (en) | 2019-08-16 | 2019-08-16 | Log monitoring system for electric power metering micro-service architecture and implementation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110633186A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111241049A (en) * | 2020-01-06 | 2020-06-05 | 深圳市同洲电子股份有限公司 | Distributed operation log implementation system based on micro-service architecture |
| CN111309793A (en) * | 2020-01-15 | 2020-06-19 | 北大方正集团有限公司 | Data processing method, device and equipment |
| CN111694721A (en) * | 2020-06-15 | 2020-09-22 | 南方电网科学研究院有限责任公司 | Fault monitoring method and device for microservice |
| CN111736983A (en) * | 2020-05-25 | 2020-10-02 | 南京大学 | Micro-service deployment method facing task and resource perception |
| CN111881161A (en) * | 2020-07-27 | 2020-11-03 | 新华智云科技有限公司 | Index measurement calculation method, system, equipment and storage medium |
| CN111949494A (en) * | 2020-09-16 | 2020-11-17 | 北京浪潮数据技术有限公司 | Task regulation and control method, device and related equipment |
| CN111984507A (en) * | 2020-08-24 | 2020-11-24 | 北京微步在线科技有限公司 | Information processing method for diagnosis log and electronic equipment |
| CN112131196A (en) * | 2020-09-09 | 2020-12-25 | 华人运通(上海)云计算科技有限公司 | Distributed log processing method and device, terminal equipment and storage medium |
| CN112148578A (en) * | 2020-10-12 | 2020-12-29 | 贵州电网有限责任公司 | IT fault defect prediction method based on machine learning |
| CN112445937A (en) * | 2020-11-30 | 2021-03-05 | 成都新潮传媒集团有限公司 | Json log generation method and device and computer readable storage medium |
| CN112612800A (en) * | 2020-12-15 | 2021-04-06 | 福建天泉教育科技有限公司 | Method and terminal for improving data writing efficiency |
| CN112698953A (en) * | 2021-01-11 | 2021-04-23 | 国网电力科学研究院武汉南瑞有限责任公司 | Power grid intelligent operation and detection platform based on micro-service |
| CN112714018A (en) * | 2020-12-28 | 2021-04-27 | 上海领健信息技术有限公司 | Gateway-based ElasticSearch search service method, system, medium and terminal |
| CN114324609A (en) * | 2021-12-09 | 2022-04-12 | 长江勘测规划设计研究有限责任公司 | ELK-based slope acoustic emission monitoring data analysis method |
| CN114363156A (en) * | 2022-01-25 | 2022-04-15 | 南瑞集团有限公司 | Hydropower station computer monitoring system deployment method based on cluster technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| US20180368068A1 (en) * | 2017-06-15 | 2018-12-20 | HyperTrack Inc. | Systems And Methods For Receiving Sensor Data From A Mobile Device |
| CN109344033A (en) * | 2018-09-27 | 2019-02-15 | 浪潮软件股份有限公司 | A kind of cloud log collection method based on distributed structure/architecture |
| CN109376532A (en) * | 2018-10-31 | 2019-02-22 | 云南电网有限责任公司 | Power network security monitoring method and system based on the analysis of ELK log collection |
-
2019
- 2019-08-16 CN CN201910760512.0A patent/CN110633186A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180368068A1 (en) * | 2017-06-15 | 2018-12-20 | HyperTrack Inc. | Systems And Methods For Receiving Sensor Data From A Mobile Device |
| CN107861859A (en) * | 2017-11-22 | 2018-03-30 | 北京汇通金财信息科技有限公司 | A kind of blog management method and system based on micro services framework |
| CN109344033A (en) * | 2018-09-27 | 2019-02-15 | 浪潮软件股份有限公司 | A kind of cloud log collection method based on distributed structure/architecture |
| CN109376532A (en) * | 2018-10-31 | 2019-02-22 | 云南电网有限责任公司 | Power network security monitoring method and system based on the analysis of ELK log collection |
Non-Patent Citations (1)
| Title |
|---|
| 张振，刘俊艳: "基于微服务架构的日志监控系统的设计与实现", 《软件》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111241049A (en) * | 2020-01-06 | 2020-06-05 | 深圳市同洲电子股份有限公司 | Distributed operation log implementation system based on micro-service architecture |
| CN111309793A (en) * | 2020-01-15 | 2020-06-19 | 北大方正集团有限公司 | Data processing method, device and equipment |
| CN111736983A (en) * | 2020-05-25 | 2020-10-02 | 南京大学 | Micro-service deployment method facing task and resource perception |
| CN111736983B (en) * | 2020-05-25 | 2023-12-26 | 南京大学 | Task and resource awareness-oriented micro-service deployment method |
| CN111694721A (en) * | 2020-06-15 | 2020-09-22 | 南方电网科学研究院有限责任公司 | Fault monitoring method and device for microservice |
| CN111881161A (en) * | 2020-07-27 | 2020-11-03 | 新华智云科技有限公司 | Index measurement calculation method, system, equipment and storage medium |
| CN111984507A (en) * | 2020-08-24 | 2020-11-24 | 北京微步在线科技有限公司 | Information processing method for diagnosis log and electronic equipment |
| CN112131196A (en) * | 2020-09-09 | 2020-12-25 | 华人运通(上海)云计算科技有限公司 | Distributed log processing method and device, terminal equipment and storage medium |
| CN111949494B (en) * | 2020-09-16 | 2022-06-10 | 北京浪潮数据技术有限公司 | Task regulation and control method, device and related equipment |
| CN111949494A (en) * | 2020-09-16 | 2020-11-17 | 北京浪潮数据技术有限公司 | Task regulation and control method, device and related equipment |
| CN112148578A (en) * | 2020-10-12 | 2020-12-29 | 贵州电网有限责任公司 | IT fault defect prediction method based on machine learning |
| CN112445937B (en) * | 2020-11-30 | 2023-11-14 | 成都新潮传媒集团有限公司 | Json log generation method and device and computer readable storage medium |
| CN112445937A (en) * | 2020-11-30 | 2021-03-05 | 成都新潮传媒集团有限公司 | Json log generation method and device and computer readable storage medium |
| CN112612800A (en) * | 2020-12-15 | 2021-04-06 | 福建天泉教育科技有限公司 | Method and terminal for improving data writing efficiency |
| CN112612800B (en) * | 2020-12-15 | 2022-07-05 | 福建天泉教育科技有限公司 | Method and terminal for improving data writing efficiency |
| CN112714018A (en) * | 2020-12-28 | 2021-04-27 | 上海领健信息技术有限公司 | Gateway-based ElasticSearch search service method, system, medium and terminal |
| CN112698953A (en) * | 2021-01-11 | 2021-04-23 | 国网电力科学研究院武汉南瑞有限责任公司 | Power grid intelligent operation and detection platform based on micro-service |
| CN114324609A (en) * | 2021-12-09 | 2022-04-12 | 长江勘测规划设计研究有限责任公司 | ELK-based slope acoustic emission monitoring data analysis method |
| CN114363156A (en) * | 2022-01-25 | 2022-04-15 | 南瑞集团有限公司 | Hydropower station computer monitoring system deployment method based on cluster technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633186A (en) | Log monitoring system for electric power metering micro-service architecture and implementation method | |
| CN106611046B (en) | Spatial data storage processing middleware system based on big data technology | |
| US20230144450A1 (en) | Multi-partitioning data for combination operations | |
| Shah et al. | A framework for social media data analytics using Elasticsearch and Kibana | |
| CN106708993B (en) | Method for realizing space data storage processing middleware framework based on big data technology | |
| CN107819824B (en) | Urban data opening and information service system and service method | |
| CN104160394B (en) | Scalable analysis platform for semi-structured data | |
| CN105122243B (en) | Expansible analysis platform for semi-structured data | |
| Padhy | Big data processing with Hadoop-MapReduce in cloud systems | |
| CN110659790A (en) | Experiment management and data analysis system | |
| CN105138661A (en) | Hadoop-based k-means clustering analysis system and method of network security log | |
| CN111324610A (en) | Data synchronization method and device | |
| CN104102710A (en) | Massive data query method | |
| US9753960B1 (en) | System, method, and computer program for dynamically generating a visual representation of a subset of a graph for display, based on search criteria | |
| US20150032743A1 (en) | Analyzing files using big data tools | |
| WO2010042238A1 (en) | System and method for data warehousing and analytics on a distributed file system | |
| US10614087B2 (en) | Data analytics on distributed databases | |
| CN109710767B (en) | Multilingual big data service platform | |
| CN112148718A (en) | Big data support management system for city-level data middling station | |
| US10216862B1 (en) | Predictive estimation for ingestion, performance and utilization in a data indexing and query system | |
| CN110245134B (en) | Increment synchronization method applied to search service | |
| CN114416855A (en) | Visualization platform and method based on electric power big data | |
| CN112948486A (en) | Batch data synchronization method and system and electronic equipment | |
| CN103226608A (en) | Parallel file searching method based on folder-level telescopic Bloom Filter bit diagram | |
| CN114297173A (en) | Knowledge graph construction method and system for large-scale mass data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191231 |