CN111352901A - Monitoring data storage method based on computer - Google Patents
Monitoring data storage method based on computer Download PDFInfo
- Publication number
- CN111352901A CN111352901A CN202010210016.0A CN202010210016A CN111352901A CN 111352901 A CN111352901 A CN 111352901A CN 202010210016 A CN202010210016 A CN 202010210016A CN 111352901 A CN111352901 A CN 111352901A
- Authority
- CN
- China
- Prior art keywords
- monitoring data
- data set
- attribute information
- storage node
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/137—Hash-based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a monitoring data storage method based on a computer, which comprises the steps of obtaining at least one monitoring data of a monitored device and data attribute information corresponding to each monitoring data; dividing monitoring data based on data attribute information, wherein each data attribute information corresponds to one monitoring data set, and each monitoring data set comprises at least one monitoring data; respectively determining a preset storage node corresponding to each monitoring data set by utilizing a consistent hash algorithm; and storing the data attribute information corresponding to each monitoring data set and each monitoring data to the corresponding preset storage node. The monitoring data with the same data attribute information is stored in the corresponding storage nodes, the storage mode is relatively orderly and not disordered, when the monitoring data with a certain data attribute needs to be read, the monitoring data only needs to be read from the corresponding storage nodes, the reading timeliness and the reliability are high, and the reliable and quick reading of the monitoring data is realized.
Description
Technical Field
The invention relates to a monitoring data storage method based on a computer.
Background
When the monitored equipment is monitored, various monitoring data of the monitored equipment can be acquired and stored in the related memory. In addition, in order to comprehensively and accurately acquire the operating state and performance of the monitored equipment, the monitoring data needs to be read, so that high reading timeliness and reliability are required, but the existing monitoring data storage methods simply store the monitoring data into a memory, do not perform related data management, cause storage to be disordered, and further cannot achieve reliable and rapid reading of the monitoring data.
Disclosure of Invention
The invention aims to provide a monitoring data storage method based on a computer, which is used for solving the problems that the storage is messy and the reliable and quick reading of monitoring data cannot be realized in the existing monitoring data storage method.
In order to solve the problems, the invention adopts the following technical scheme:
a computer-based monitoring data storage method, comprising:
acquiring at least one monitoring data of monitored equipment and data attribute information corresponding to each monitoring data;
dividing the monitoring data based on the data attribute information, wherein each data attribute information corresponds to one monitoring data set, and each monitoring data set comprises at least one monitoring data;
respectively determining a preset storage node corresponding to each monitoring data set by utilizing a consistent hash algorithm;
and storing the data attribute information corresponding to each monitoring data set and each monitoring data to the corresponding preset storage node.
Optionally, the dividing the monitoring data based on the data attribute information includes:
and dividing the monitoring data with the same data attribute information into the same data set to obtain a corresponding monitoring data set.
Optionally, the determining, by using a consistent hash algorithm, the preset storage node corresponding to each monitoring data set includes:
respectively calculating a first hash value corresponding to each monitoring data set and a second hash value corresponding to each preset storage node by using a preset hash algorithm;
respectively mapping the first hash value and the second hash value on a hash ring with a preset number of digital spaces according to a preset mapping rule to obtain a first mapping relation between the first hash value and the second hash value;
and respectively determining the preset storage node corresponding to each monitoring data set according to the first mapping relation.
Optionally, after the storing the data attribute information and the monitoring data corresponding to each monitoring data set to the corresponding preset storage node, the method includes:
respectively establishing index information corresponding to each monitoring data set;
generating a second mapping relation between the index information and the preset storage node;
correspondingly, when the reading request information is detected, the monitoring data set corresponding to the index information in the reading request information is read from the preset storage node based on the second mapping relation.
The invention has the beneficial effects that: the method comprises the steps of obtaining monitoring data and corresponding data attribute information of monitored equipment, dividing the monitoring data based on the data attribute information, wherein each data attribute information corresponds to one monitoring data set, each monitoring data set comprises at least one monitoring data, then respectively determining a preset storage node corresponding to each monitoring data set by utilizing a consistent Hash algorithm, and storing the data attribute information and each monitoring data corresponding to each monitoring data set to the corresponding preset storage node. The monitoring data with the same data attribute information is stored in the corresponding storage nodes, the storage mode is relatively orderly and not disordered, when the monitoring data with a certain data attribute needs to be read, the monitoring data only needs to be read from the corresponding storage nodes, the reading timeliness and the reliability are high, and the reliable and quick reading of the monitoring data is realized.
Drawings
In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings needed to be used in the embodiment will be briefly described as follows:
FIG. 1 is a schematic flow diagram of a computer-based method of monitoring data storage;
fig. 2 is a schematic structural diagram illustrating a structure in which the first hash value and the second hash value are respectively mapped on the hash ring.
Detailed Description
The embodiment provides a monitoring data storage method based on a computer, the execution subject of the monitoring data storage method is computer equipment or server equipment, and the monitoring data storage method is realized through computer software.
As shown in fig. 1, the monitoring data storage method includes:
acquiring at least one monitoring data of monitored equipment and data attribute information corresponding to each monitoring data:
the embodiment does not limit the monitored device, and may be a factory production device, a communication device, an electrical device, or the like. The monitored device usually comprises at least one kind of monitoring data, which in the case of a transformer device in an electrical device comprises voltage data, current data, active power data, reactive power data, and various fault data, among others.
At least one kind of monitoring data of the monitored equipment and data attribute information corresponding to the monitoring data are obtained. Each kind of monitoring data has corresponding data attributes, the data attributes can be understood as data types or data types, for the monitoring data of the transformer equipment, the data attributes of voltage data and current data are basic electrical parameters, the data attributes of active power data and reactive power data are electric energy quality parameters, and the data attributes of various fault data are fault parameters. Of course, the data attribute of each monitoring data can be manually defined or divided according to actual needs.
Dividing the monitoring data based on the data attribute information, wherein each data attribute information corresponds to a monitoring data set, and each monitoring data set comprises at least one monitoring data:
the dividing of the monitoring data based on the data attribute information is specifically as follows: and dividing the monitoring data with the same data attribute information into the same data set to obtain a corresponding monitoring data set. Then, the monitoring data of the same data attribute information is divided into the same monitoring data set, each data attribute information corresponds to one monitoring data set, and each monitoring data set includes at least one type of monitoring data, and the specific number of the included monitoring data is determined by the specific data attribute.
Respectively determining the preset storage nodes corresponding to each monitoring data set by utilizing a consistent hash algorithm:
based on the utilization of a consistent hash algorithm, respectively determining a preset storage node corresponding to each monitoring data set, a specific implementation process is given as follows:
(1) and respectively calculating a first hash value corresponding to each monitoring data set and a second hash value corresponding to each preset storage node by using a preset hash algorithm.
In this embodiment, the preset hash algorithm is an MD5 algorithm, and the MD5 algorithm is used to calculate a first hash value corresponding to each monitoring data set, for example, the first hash values are respectively represented as: key1, key2, …, key (n-1), wherein n is an integer.
Respectively calculating a second hash value corresponding to each preset storage node by using the MD5 algorithm, where the second hash values are respectively expressed as: KEY1, KEY2, … and KEY (n-k), wherein k is less than or equal to n.
(2) And mapping the first hash value and the second hash value on a hash ring with a preset number of digital spaces according to a preset mapping rule to obtain a first mapping relation between the first hash value and the second hash value.
The preset mapping rule is to determine a start position of mapping, and then clockwise search a preset digital space corresponding to the hash ring from the start position, for example, the hash ring has 2n digital spaces, and map the first hash value and the second hash value in the corresponding digital spaces respectively to obtain a first mapping relationship between the first hash value and the second hash value.
Specifically, as shown in fig. 2, the structure diagram is a schematic diagram of mapping the first hash value and the second hash value on the hash ring respectively. As shown in fig. 2, 0 is the starting position of the mapping, corresponding to the first digit space; 2n is the end position of the mapping and corresponds to the 2 n-th digit space. For ease of observation, the first hash value and the second hash value are not mapped at the 2n position.
As can be seen from fig. 2, the first hash value and the second hash value have a one-to-one mapping relationship. When the number of the monitoring data sets exceeds the number of the preset digital spaces of the hash ring, the preset digital spaces corresponding to the hash ring are searched clockwise from the initial position again, and the first hash values corresponding to the redundant monitoring data sets are remapped on the hash ring to cover the previous first hash values.
(3) And respectively determining the preset storage node corresponding to each monitoring data set according to the first mapping relation.
Each monitoring data set corresponds to a unique first hash value, and each preset storage node also corresponds to a unique second hash value, so that the preset storage node corresponding to each monitoring data set can be respectively determined according to the first mapping relation.
Storing the data attribute information corresponding to each monitoring data set and each monitoring data to a corresponding preset storage node:
storing the data attribute information and the monitoring data corresponding to each monitoring data set to a corresponding preset storage node, for example: the number of the preset storage nodes can be the same as that of the data attribute information, the preset storage nodes correspond to the data attribute information one to one, and for monitoring data of the transformer equipment, the number of the storage nodes is three, namely a first storage node, a second storage node and a third storage node, wherein the data attribute information stored in the first storage node is a basic electrical parameter, and the specifically stored monitoring data are voltage data and current data; the data attribute information stored in the second storage node is an electric energy quality parameter, and the specifically stored monitoring data is active power data and reactive power data; the data attribute information stored in the third storage node is a fault parameter, and the specifically stored monitoring data is various fault data.
In this embodiment, after the data attribute information and the monitoring data corresponding to each monitoring data set are stored in the corresponding preset storage node, the monitoring data storage method further includes the following steps:
respectively establishing index information corresponding to each monitoring data set:
in this embodiment, the index information corresponding to each monitoring data set may be data attribute information or a first hash value corresponding to each monitoring data set, or may be identified by a preset index word, which is not limited herein.
Generating a second mapping relation between the index information and the preset storage node:
because the preset storage node and the monitoring data sets have a first mapping relationship, after the index information corresponding to each monitoring data set is established, a second mapping relationship between the index information and the preset storage node can be generated according to the first mapping relationship and the index information corresponding to each monitoring data set.
By establishing the index information corresponding to each monitoring data set and generating the second mapping relation between the index information and the preset storage node, the corresponding monitoring data set can be read from the preset storage node based on the second mapping relation in the reading process, and the reading timeliness and reliability are improved.
Correspondingly, when the reading request information is detected, reading a monitoring data set corresponding to the index information in the reading request information from the preset storage node based on the second mapping relation:
when the reading request information is detected, the monitoring data set corresponding to the index information in the reading request information can be read from the preset storage node based on the obtained second mapping relation, and finally the corresponding monitoring data is obtained.
The above-mentioned embodiments are merely illustrative of the technical solutions of the present invention in a specific embodiment, and any equivalent substitutions and modifications or partial substitutions of the present invention without departing from the spirit and scope of the present invention should be covered by the claims of the present invention.
Claims (4)
1. A computer-based monitoring data storage method, comprising:
acquiring at least one monitoring data of monitored equipment and data attribute information corresponding to each monitoring data;
dividing the monitoring data based on the data attribute information, wherein each data attribute information corresponds to one monitoring data set, and each monitoring data set comprises at least one monitoring data;
respectively determining a preset storage node corresponding to each monitoring data set by utilizing a consistent hash algorithm;
and storing the data attribute information corresponding to each monitoring data set and each monitoring data to the corresponding preset storage node.
2. The computer-based monitoring data storage method of claim 1, wherein the partitioning the monitoring data based on the data attribute information comprises:
and dividing the monitoring data with the same data attribute information into the same data set to obtain a corresponding monitoring data set.
3. The computer-based monitoring data storage method according to claim 1 or 2, wherein the determining the preset storage node corresponding to each monitoring data set by using a consistent hashing algorithm respectively comprises:
respectively calculating a first hash value corresponding to each monitoring data set and a second hash value corresponding to each preset storage node by using a preset hash algorithm;
respectively mapping the first hash value and the second hash value on a hash ring with a preset number of digital spaces according to a preset mapping rule to obtain a first mapping relation between the first hash value and the second hash value;
and respectively determining the preset storage node corresponding to each monitoring data set according to the first mapping relation.
4. The computer-based monitoring data storage method according to claim 3, wherein after storing the data attribute information corresponding to each monitoring data set and each monitoring data to the corresponding preset storage node, the method comprises:
respectively establishing index information corresponding to each monitoring data set;
generating a second mapping relation between the index information and the preset storage node;
correspondingly, when the reading request information is detected, the monitoring data set corresponding to the index information in the reading request information is read from the preset storage node based on the second mapping relation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010210016.0A CN111352901A (en) | 2020-03-23 | 2020-03-23 | Monitoring data storage method based on computer |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010210016.0A CN111352901A (en) | 2020-03-23 | 2020-03-23 | Monitoring data storage method based on computer |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111352901A true CN111352901A (en) | 2020-06-30 |
Family
ID=71198097
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010210016.0A Withdrawn CN111352901A (en) | 2020-03-23 | 2020-03-23 | Monitoring data storage method based on computer |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111352901A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100122026A1 (en) * | 2008-09-19 | 2010-05-13 | Oracle International Corporation | Selectively reading data from cache and primary storage |
| CN103490937A (en) * | 2013-10-12 | 2014-01-01 | 北京奇虎科技有限公司 | Method and device for filtering monitoring data |
| CN104216708A (en) * | 2014-08-20 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Method and device for data monitoring |
| CN105303456A (en) * | 2015-10-16 | 2016-02-03 | 国家电网公司 | Method for processing monitoring data of electric power transmission equipment |
| CN106156328A (en) * | 2016-07-06 | 2016-11-23 | 中国银行股份有限公司 | A kind of bank's running log data monitoring method and system |
-
2020
- 2020-03-23 CN CN202010210016.0A patent/CN111352901A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100122026A1 (en) * | 2008-09-19 | 2010-05-13 | Oracle International Corporation | Selectively reading data from cache and primary storage |
| CN103490937A (en) * | 2013-10-12 | 2014-01-01 | 北京奇虎科技有限公司 | Method and device for filtering monitoring data |
| CN104216708A (en) * | 2014-08-20 | 2014-12-17 | 北京奇艺世纪科技有限公司 | Method and device for data monitoring |
| CN105303456A (en) * | 2015-10-16 | 2016-02-03 | 国家电网公司 | Method for processing monitoring data of electric power transmission equipment |
| CN106156328A (en) * | 2016-07-06 | 2016-11-23 | 中国银行股份有限公司 | A kind of bank's running log data monitoring method and system |
Non-Patent Citations (1)
| Title |
|---|
| 鞠瑞 等: "面向云平台的异构监控数据存储方法", 《山东大学学报(理学版)》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108205577B (en) | Array construction method, array query method, device and electronic equipment | |
| CN106874348B (en) | File storage and index method and device and file reading method | |
| CN107657019B (en) | Network topology structure obtaining method and system of power grid system | |
| CN111475105B (en) | Monitoring data storage method, monitoring data storage device, monitoring data server and storage medium | |
| CN110908590A (en) | Distributed storage method and system for transformer substation data | |
| CN111177143B (en) | Key value data storage method and device, storage medium and electronic equipment | |
| WO2014000578A1 (en) | Method and apparatus for processing database data in distributed database system | |
| CN105320771A (en) | Hash ring based time sequence database service cluster implementation method and system | |
| WO2013143278A1 (en) | Method, device and system for querying data index | |
| CN108388613B (en) | Cache data updating method | |
| CN110928690B (en) | Method for synchronizing and verifying block chain data in 5G network environment | |
| CN111046081A (en) | Access method and system for industrial time sequence data | |
| CN114398567A (en) | Multi-time-state power grid topological graph data management method and device and computer equipment | |
| CN111352901A (en) | Monitoring data storage method based on computer | |
| CN107590160B (en) | Method and device for monitoring internal structure of radix tree to realize test | |
| CN110020001A (en) | Storage, querying method and the corresponding equipment of string data | |
| CN110032586B (en) | Storage method, query method and acquisition and storage system for energy storage cell data | |
| CN116842012A (en) | Method, device, equipment and storage medium for storing Redis cluster in fragments | |
| CN107463540B (en) | Electric energy quality data processing method and electric energy quality monitoring device | |
| CN114996535A (en) | Data retrieval method and system for power equipment in power grid and electronic equipment | |
| CN108959581B (en) | Power equipment state data processing method oriented to space-time fusion analysis | |
| CN116126928A (en) | Information searching system based on variable fingerprint cuckoo filter | |
| CN111538804A (en) | HBase-based graph data processing method and equipment | |
| CN117271480B (en) | Data processing method, device, electronic equipment and medium | |
| CN111553040A (en) | Power grid topology analysis high-performance computing method and device based on GPU acceleration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200630 |
|
| WW01 | Invention patent application withdrawn after publication |