CN114528341A - Power distribution station low-voltage monitoring method and device based on big data technology - Google Patents
Power distribution station low-voltage monitoring method and device based on big data technology Download PDFInfo
- Publication number
- CN114528341A CN114528341A CN202210091696.8A CN202210091696A CN114528341A CN 114528341 A CN114528341 A CN 114528341A CN 202210091696 A CN202210091696 A CN 202210091696A CN 114528341 A CN114528341 A CN 114528341A
- Authority
- CN
- China
- Prior art keywords
- voltage
- data
- limit
- archive
- power distribution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 89
- 238000009826 distribution Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000005516 engineering process Methods 0.000 title claims abstract description 37
- 238000004458 analytical method Methods 0.000 claims abstract description 50
- 238000004364 calculation method Methods 0.000 claims abstract description 45
- 230000002159 abnormal effect Effects 0.000 claims abstract description 25
- 238000005259 measurement Methods 0.000 claims abstract description 22
- 230000002776 aggregation Effects 0.000 claims abstract description 21
- 238000004220 aggregation Methods 0.000 claims abstract description 21
- 238000013500 data storage Methods 0.000 claims abstract description 21
- 230000003203 everyday effect Effects 0.000 claims abstract description 8
- 238000003908 quality control method Methods 0.000 claims abstract description 8
- 238000003860 storage Methods 0.000 claims description 21
- 238000004140 cleaning Methods 0.000 claims description 15
- 230000002354 daily effect Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 8
- 238000012797 qualification Methods 0.000 claims description 6
- 230000003442 weekly effect Effects 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 4
- 238000007726 management method Methods 0.000 claims description 2
- 230000004927 fusion Effects 0.000 abstract description 2
- 238000007405 data analysis Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
- G01R19/00—Arrangements for measuring currents or voltages or for indicating presence or sign thereof
- G01R19/0084—Arrangements for measuring currents or voltages or for indicating presence or sign thereof measuring voltage only
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
- Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
- Y04S10/00—Systems supporting electrical power generation, transmission or distribution
- Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications
- Y04S10/52—Outage or fault management, e.g. fault detection or location
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Remote Monitoring And Control Of Power-Distribution Networks (AREA)
Abstract
The invention discloses a power distribution area low-voltage monitoring method and a device based on a big data technology, wherein the method comprises the following steps: the voltage monitoring example collects the metering voltage of the distribution transformer area and finally uploads the metering voltage to the data storage cluster, and in the calculation cluster, a monitoring analysis task reads the metering data and the archive data, establishes archive thermal data cache, fuses the metering and archive data, and precomputes an abnormal voltage and an out-of-limit voltage mark. And then carrying out distributed parallel aggregation based on the fusion data, carrying out voltage abnormal rate calculation on one part, carrying out voltage out-of-limit condition calculation on the other part, finally summarizing results, outputting basic analysis data of the same measurement point voltage every day, generating a report according to the data, and subsequently carrying out voltage quality control according to the report. The method solves the problem of low-voltage monitoring of the power distribution station area, and can complete calculation and provide analysis results in a reasonable time in the face of billion-level data volume per day.
Description
Technical Field
The invention relates to the technical field of electric power big data analysis, in particular to a power distribution area low-voltage monitoring method and device based on big data technology.
Background
The traditional domestic voltage quality monitoring function is single, the traditional monitoring system adopts a table form, the page display and expression are backward, the full automation is not realized, some places even manual meter reading is adopted, and the real-time performance and the accuracy are poor.
With the acceleration of the digital process of the electric power industry in China, a single machine monitoring and analyzing mode gradually appears, at the moment, domestic power grids are mainly sampled and monitored, the single machine analyzing mode is adopted to face the situation that the data volume is not large, but at present, when the big data era comes, a power grid monitoring network is gradually changed from sampling monitoring to full-coverage monitoring, each distribution transformer is provided with an intelligent electric meter capable of collecting voltage metering data, and only daily data of Guangdong provincial level city reaches hundred million level. The data volume managed by the current power grid data center is increased from GB to TB level or even PB level, and the data composition is more and more complex, and the data composition not only comprises conventional relational structured data, but also comprises semi-structured or unstructured data. The data structure is also compounded more and more, and the acquired data is also changed from single metering data into multi-domain fused data, such as the data not only contains basic metering current and voltage, but also contains information such as region codes, measuring point marks and the like. In the face of increasingly complex huge data sets and increasingly rich data, higher requirements are placed on analysis of voltage data acquired by the distribution transformer. Some past schemes based on a single machine cannot fully utilize calculation power to analyze big electric power data, and are not satisfactory in the aspect of time consumption.
The development trend of electric power big data analysis is to use a distributed storage network and a distributed parallel computing scheme, i.e. a plurality of storage and data processing devices are connected through a high-speed communication line to cooperatively provide storage and computing services. The storage of mass power monitoring data usually uses a distributed storage network, and one data set is stored to different devices in a data block form, so that the capacity of each server can be fully utilized, backup can be provided, and the disaster tolerance capability of the data is improved. The data is calculated by using a distributed parallel computing platform, such as a disk-based scheme MapReduce, a memory-based scheme Spark, a Flink and the like, so that the limitation of a stand-alone device is eliminated. In the disk-based scheme MapReduce, a calculation result is stored back to the disk after each calculation, the calculation speed is slowed down due to the I/O of the disk, a large amount of time is consumed in the whole calculation process on the I/O of the disk, the calculation speed is slow, and for a low-voltage monitoring scene, the data size is not so large that the disk-based scheme is used, so that a memory-based calculation method is more suitable. The memory-based schemes Spark and Flink are iterated in the memory, so that the calculation speed is much higher than that of MapReduce, and the memory-based schemes Spark and Flink are common schemes for big data analysis at present. Practice shows that data in a big data storage system has different access heat degrees, file data in the power industry can not be easily changed usually, the file data is read-write-more-less-type thermal data, the access frequency is high, the updating rate is low, and the method is particularly suitable for cache reuse after a scene is regularly cleaned.
With the acceleration of the domestic industrial digitization process and the arrival of the big data era, the combination of the two is more and more compact. At present, a complete system is not formed in domestic low-voltage monitoring, a distributed parallel voltage monitoring, analyzing and calculating method based on big data is gradually explored, a theoretical framework for solving low-voltage monitoring based on big data is not systematically provided, certain limitations are achieved, most of voltage monitoring, analyzing and calculating schemes based on a distributed calculating framework are specifically applied, and universality is not achieved.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a power distribution area low-voltage monitoring method and device based on a big data technology.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides a power distribution area low-voltage monitoring method based on a big data technology, which comprises the following steps:
monitoring and acquiring the voltage state of the distribution transformer at time intervals, and uploading the voltage state to a corresponding voltage monitoring controller;
the voltage monitoring controller organizes the collected voltage data into a batch of tissues according to the same collecting time point and then uploads the organized data to the data storage cluster;
the computing manager configures task related parameters and starts an analysis task in the computing cluster;
in the computing cluster, an analysis task reads metering data and archive data from a data storage cluster, establishes archive thermal data cache, fuses the metering and archive data, and calculates abnormal voltage marks and voltage out-of-limit time;
in distributed parallel tasks of the computing cluster, parallel aggregation is carried out, one part is used for carrying out voltage abnormal rate computation, the other part is used for carrying out voltage out-of-limit condition computation, the computation results of the two parts are summarized, and the computation results are output as daily same-measurement-point voltage basic analysis data;
and generating a report according to the voltage basic analysis data of the same measurement point every day, and performing voltage quality control according to the report.
Preferably, the voltage state data is expressed as a voltage measurement vector Vi(voltage, point, code), wherein voltage is a three-phase voltage vector, point is a measurement point identifier, and code is a region code.
As a preferred technical solution, the establishing of the archive data cache specifically includes:
before the calculation of the archive data, regular cleaning is needed to remove some invalid data and unnecessary data, the method has the characteristic of reading more and updating less, hot data cache after cleaning is established, the efficiency can be effectively improved, and the cache is abandoned for cleaning again when the archive is updated. The cleaning process is carried out on the data computing cluster in a distributed parallel mode, and the result is output to a specified directory of the data storage cluster by the data computing cluster in a file form with the same format as the archive data and is used as archive hot data cache;
the hot data cache is established in a data computing cluster, in a Filter operator, specific service rule conditions are combined, cleaning is carried out in a distributed parallel mode, and the result is output to a specified directory of the data storage cluster by the data computing cluster in a file with the same format as archive data, so that the hot data cache is established.
As a preferred technical solution, the performing the abnormal voltage marking and the calculation of the voltage out-of-limit time specifically includes:
voltage anomaly determination, defined as follows:
wherein, x is the determination result of the voltage value, normal is normal voltage, abnormal is abnormal voltage, ref is the reference voltage of the voltage grade corresponding to the current distribution transformer, voltage is the voltage value of the current distribution transformer, and lower _ rate and upper _ rate respectively represent the ratio of the lower limit to the upper limit;
and judging whether the voltage exceeds the upper limit or the lower limit, wherein the definition is as follows:
wherein, ylAnd yuThe method is characterized by representing the time length of an upper limit and the time length of an upper limit, wherein voltage is input voltage, gap _ time is sampling interval of a voltage monitor, lower is lower limit voltage, upper is upper limit voltage, and the definition is as follows:
wherein ref is a reference voltage of a corresponding voltage class of the current distribution transformer, lowerLimit is a lower limit index of the corresponding voltage class of the distribution transformer, and upperLimit is an upper limit index of the corresponding voltage class of the distribution transformer.
As a preferred technical scheme, the voltage abnormality rate calculation is to count the ratio relationship between the number of voltage abnormality points and the number of normal points in the same measurement point;
and the voltage out-of-limit condition calculation is to aggregate voltage data identified by the same measuring point based on pre-calculated out-of-limit time in the Join operator, and calculate a voltage extreme value and time, out-of-limit time and voltage out-of-limit rate.
As a preferred technical scheme, when two parts of calculation results are summarized, one part of voltage basic analysis data is generated for each measuring point, wherein the voltage basic analysis data comprises a qualification rate, qualification time, total overrun time and total monitoring duration; each piece of the voltage basic analysis data corresponds to a single measuring point, a daily report is generated for the measuring points with the same regional code, extra records are carried out on the measuring points with low percent of pass, the daily report is collected into a weekly report and a monthly report, and the long-time unqualified measuring points are tracked and processed.
The invention provides a power distribution area low-voltage monitoring system based on a big data technology, which is applied to the power distribution area low-voltage monitoring method based on the big data technology and comprises a voltage state acquisition module, a data uploading module, a task configuration module, a data processing module, a parallel aggregation module and a voltage quality management module;
the voltage state acquisition module is used for monitoring and acquiring the voltage state of the distribution transformer at time intervals and uploading the voltage state to the corresponding voltage monitoring controller;
the data uploading module is used for uploading the collected voltage data to the data storage cluster after the voltage monitoring controller organizes the collected voltage data into a batch of tissues according to the same collecting time point;
the task configuration module is used for configuring task related parameters by the computing manager and starting an analysis task in the computing cluster;
the data processing module is used for reading the metering data and the file data from the data storage cluster by the analysis task in the calculation cluster, establishing a file thermal data cache, fusing the metering data and the file data, and calculating the abnormal voltage mark and the voltage out-of-limit time length;
the parallel aggregation module is used for performing parallel aggregation in distributed parallel tasks of the computing cluster, performing voltage abnormal rate computation on one part, performing voltage out-of-limit condition computation on the other part, summarizing computation results of the two parts, and outputting the computation results as daily same-measurement-point voltage basic analysis data;
and the voltage quality control module is used for generating a report according to the voltage basic analysis data of the same measurement point every day and controlling the voltage quality according to the report.
Still another aspect of the present invention provides a computer-readable storage medium storing a program, which when executed by a processor, implements the power distribution substation low voltage monitoring method based on big data technology.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1) according to the technical scheme, aiming at the characteristics that the reading amount of the scene file data in the power industry is large and the writing amount of the scene file data is small, and large-scale regular cleaning is needed before use, the distribution transformer file data after regular cleaning is cached based on the HDFS, only when the file is updated, the file is cleaned again, otherwise, the cache is directly read, and the overall calculation efficiency is improved.
2) Compared with the traditional single-machine data analysis, the scheme of the invention provides a distributed parallel computing method for solving the low-voltage monitoring data analysis, the throughput is high, and the computing efficiency is effectively improved.
3) Compared with a scheme based on MapReduce, the method is based on memory calculation, overcomes the defect of slow MapReduce calculation, reduces the consumption on disk I/O, and is more suitable for being applied to a power distribution station area low-voltage monitoring scene.
4) According to the power distribution area low-voltage monitoring framework based on the big data, disclosed by the invention, the calculation power and the storage resources are fully utilized by combining the Internet of things equipment and the big data technology, the voltage monitoring and analyzing problem under the existing big data condition can be effectively solved, and meanwhile, the framework has extremely strong flexibility and is convenient for the transverse expansion of the voltage monitoring and analyzing task.
Drawings
Fig. 1 is a flowchart of a power distribution area low voltage monitoring method based on big data technology in an embodiment of the present invention.
Fig. 2 is a diagram of a large data-based power distribution substation low voltage monitoring architecture in an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a thermal data caching process for files according to an embodiment of the present invention.
Fig. 4 is a flowchart of task computation based on the Flink embodiment in the embodiment of the present invention.
FIG. 5 is a flowchart of a parallel aggregation method according to an embodiment of the present invention.
Fig. 6 is a structural diagram of a power distribution station low voltage monitoring system based on big data technology.
Fig. 7 is a structural diagram of a computer-readable storage medium of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings of the present invention. It is to be understood that the embodiments described are only a few embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention utilizes a distributed storage system-based HDFS as a storage technology to build a cluster storage network for data storage, wherein the HDFS cluster stores metering data collected by distribution transformers in various places and related archive data, such as the archive data of the distribution transformers, and the storage cluster can carry out disaster recovery backup on all data.
As shown in fig. 1, the power distribution area low voltage monitoring method based on the big data technology in this embodiment includes the following specific implementation steps:
and S1, monitoring and acquiring the voltage measurement data of the distribution transformer by the intelligent electric meter in the distribution area according to time intervals, and uploading the voltage measurement data to the corresponding voltage monitoring controller.
As shown in fig. 2, a large data-based distribution area low-voltage monitoring architecture, at present, more and more TB-level power grid data centers and even PB-level power grid data centers emerge, the development of a power large data analysis technology is promoted by massive data analysis requirements, data analysis is performed in a conventional single machine mode, computing power and storage resources cannot be fully utilized due to too large data volume, accordingly, a large data technology-based low-voltage monitoring analysis architecture is proposed, in the architecture, voltage monitoring examples are various internet of things devices with collection and measurement functions at an edge end of a power grid, a plurality of voltage monitoring examples are monitored by a voltage monitoring manager, one voltage monitoring manager is responsible for collecting collection and measurement data of the voltage monitoring examples managed by the voltage monitoring manager, and the voltage monitoring controller periodically uploads batch data to a distributed storage HDFS. At this time, a user interacts with the calculation manager, configures parameters and starts a task, the calculation manager of this embodiment starts a Flink cluster as a voltage analysis calculation example for a Web page providing parameter configuration, and reads collected metering data and distribution transformer archive data from the data storage cluster HDFS as a Flink calculation source.
Further, the voltage data result is expressed as a voltage measurement vector Vi(voltage, point, code), wherein voltage is a three-phase voltage vector, point is a measurement point identifier, and code is a region code.
And S2, collecting a batch of data by the voltage monitoring controller, uploading the data to the distributed storage system HDFS, and organizing and storing the data according to the region codes.
And S3, setting related parameters through a calculation task entry program, and starting a Flink calculation task.
And S4, reading in the hot cache archive data and the collected metering data from the HDFS, performing data fusion, and pre-calculating the fused data to complete the abnormal voltage marking and the upper and lower limit time exceeding marking.
Furthermore, because of the data quality problem, the archive data can be regularly cleaned before calculation to remove some invalid data and unnecessary data, so that the archive hot data cache has the characteristic of reading more and updating less, the efficiency can be effectively improved by establishing the cleaned hot data cache, and the cache is abandoned for cleaning again when the archive is updated. The hot data cache is established in a data computing cluster, in a Filter operator, specific service rule conditions are combined, cleaning is carried out in a distributed parallel mode, and the result is output to a specified directory of the data storage cluster by the data computing cluster in a file with the same format as archive data, so that the hot data cache is established.
As shown in fig. 3, the distribution transformer thermal data cache caches regularly cleaned distribution transformer archive data, and distribution transformer archives in a power analysis scene have the characteristics of large-scale cleaning, less reading, writing, low update frequency and the like. The scheme for checking the update uses the file flag bit to access a specific file on the cluster server to acquire the update condition of the current file. If the data is not updated, the cached hot data can be directly read and enter the Flink operator for calculation, so that the overall calculation efficiency is improved.
As shown in fig. 4, the core computation step, in this embodiment, uses a distributed storage HDFS as storage and a Flink big data computation engine as computation. Reading the distribution transformer archive data to be analyzed and collecting metering data from the HDFS, enabling the two data to enter a Join operator, connecting the two data based on the measuring point identification in the Join operator, and expanding non-metering domain data of each voltage data according to the measuring point identification for subsequent aggregation calculation. And simultaneously, pre-calculating the expanded data, marking abnormal voltage and calculating out the time of upper and lower limits. At the moment, the pre-calculated data respectively enter two aggregation operators representing different analysis requirements for aggregation calculation, wherein one is to Aggregate the voltage anomaly rate and count the ratio relation between the number of voltage anomaly points and the number of normal points in the same measurement point; and secondly, aggregating the situation that the voltage exceeds the upper limit and the lower limit, aggregating the voltage data identified by the same measuring point based on the pre-computed time for exceeding the upper limit and the lower limit in the Join operator, and computing a voltage extreme value, time, the time duration of exceeding the limit, the voltage exceeding rate and the like.
As shown in fig. 5, the voltage analysis parallel aggregation method utilizes the distributed parallel feature of the computation cluster to perform parallel aggregation on the voltage data, and can be expanded in the horizontal direction in this part. Under a voltage monitoring scene, aggregation calculation is common, and the parallel aggregation scheme is used for the scene, so that the distributed cluster calculation power can be fully utilized, and the overall calculation efficiency is improved. The voltage analysis requirements in this embodiment are listed as two, including the aggregate calculated voltage anomaly rate and the aggregate calculated out-of-limit condition, but the embodiment of the present invention is not limited by this embodiment, and can be flexibly expanded according to the actual situation.
Further, the abnormal voltage determination and the out-of-limit time period are calculated as:
s41, voltage abnormality determination, which is defined as follows:
where x is the determination result of the voltage value, normal is the normal voltage, and abnormal is the abnormal voltage. ref is the reference voltage of the current distribution transformer, voltage is the voltage value of the current distribution transformer, and lower _ rate and upper _ rate represent the ratio of the lower limit and the upper limit, respectively.
S42, judging whether the voltage exceeds the upper limit and the lower limit, wherein the definition is as follows:
wherein, ylAnd yuRepresenting the time (minutes) of the lower limit and the time (minutes) of the upper limit, voltage is the input voltage, and gap _ time is the sampling interval of the voltage monitor. lower is the lower limit voltage and upper is the upper limit voltage, and they are defined as follows:
wherein ref is a reference voltage of the distribution transformer corresponding to the voltage class, lowerLimit is a lower limit index of the distribution transformer corresponding to the voltage class, and upperLimit is an upper limit index of the distribution transformer corresponding to the voltage class.
And S5, based on the data in the fourth step, in the Flink subtask, one part of the Flank subtasks is used for calculating the voltage abnormal rate, and the other part of the Flank subtasks is used for calculating the voltage out-of-limit condition.
And S6, summarizing data of all operators, generating low-voltage basic analysis data for different measuring point identifications, wherein the low-voltage basic analysis data are one part per day, outputting daily reports, weekly reports and monthly reports through one or more analysis, and treating the distribution transformer according to a subsequent report.
Further, when the result is summarized, a part of voltage basic analysis data is generated for each measuring point, and the voltage basic analysis data comprises voltage monitoring analysis summarized data such as qualification rate, qualification time, total overrun time, total monitoring time and the like. Each piece of voltage basic analysis data corresponds to a single measuring point, a daily report is generated for the measuring points with the same regional code, extra records are carried out on the measuring points with low percent of pass, the daily report is collected into a weekly report and a monthly report, and the long-time unqualified measuring points are tracked and processed.
More specifically, Join merging is performed on the calculation results of multiple Aggregate operators again, one daily basic analysis data is generated for each measuring point, the Sink operator outputs the generated basic analysis data to the HDFS to be stored as a result, and based on the result, a single machine analysis scheme can be directly used subsequently. And summarizing the daily reports into weekly reports and monthly reports, and tracking and processing the long-time unqualified measuring points.
It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present invention is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present invention.
Based on the same idea as the power distribution area low-voltage monitoring method based on the big data technology in the embodiment, the invention further provides a power distribution area low-voltage monitoring system based on the big data technology, and the system can be used for executing the power distribution area low-voltage monitoring method based on the big data technology. For convenience of explanation, the structural schematic diagram of the power distribution station area low voltage monitoring system based on the big data technology only shows the part relevant to the embodiment of the present invention, and those skilled in the art can understand that the illustrated structure does not constitute a limitation to the device, and may include more or less components than those illustrated, or combine some components, or arrange different components.
As shown in fig. 6, in another embodiment of the present application, a power distribution station low voltage monitoring system 100 based on big data technology is provided, which includes a voltage status acquisition module 101, a data uploading module 102, a task configuration module 103, a data processing module 104, a parallel aggregation module 105, and a voltage quality governance module 106;
the voltage state acquisition module 101 is used for monitoring and acquiring the voltage state of the distribution transformer at time intervals and uploading the voltage state to the corresponding voltage monitoring controller;
the data uploading module 102 is used for the voltage monitoring controller to organize the collected voltage data and upload the organized data to the data storage cluster;
the task configuration module 103 is configured to configure task-related parameters by the computing manager, and start an analysis task in the computing cluster;
the data processing module 104 is configured to, in the computing cluster, read the metering data and the archive data from the data storage cluster by the analysis task, establish an archive thermal data cache, merge the metering and archive data, and perform abnormal voltage marking and voltage out-of-limit duration calculation;
the parallel aggregation module 105 is configured to perform parallel aggregation in a distributed parallel task of a computing cluster, perform voltage anomaly rate computation on one part, perform voltage out-of-limit condition computation on the other part, summarize computation results of the two parts, and output the summarized computation results as daily voltage basic analysis data of the same measurement point;
and the voltage quality control module 106 is configured to generate a report according to the voltage basic analysis data of the same measurement point every day, and perform voltage quality control according to the report.
It should be noted that, the power distribution area low voltage monitoring system based on the big data technology of the present invention corresponds to the power distribution area low voltage monitoring method based on the big data technology one to one, and the technical features and the advantages thereof described in the above-mentioned embodiments of the power distribution area low voltage monitoring method based on the big data technology are both applicable to the embodiments of the power distribution area low voltage monitoring system based on the big data technology, and specific contents may refer to the description in the embodiments of the method of the present invention, and are not described herein again, and thus, the present invention is stated.
In addition, in the implementation of the power distribution station area low voltage monitoring system based on the big data technology in the above embodiment, the logical division of each program module is only an example, and in practical applications, the above function distribution may be performed by different program modules according to needs, for example, due to the configuration requirements of corresponding hardware or the convenience of implementation of software, that is, the internal structure of the power distribution station area low voltage monitoring system based on the big data technology is divided into different program modules to perform all or part of the above described functions.
As shown in fig. 7, in an embodiment, a computer-readable storage medium 200 is provided, which stores a program in a memory 201, and when the program is executed by a processor 202, the program implements the power distribution area low voltage monitoring method based on big data technology, specifically:
monitoring and acquiring the voltage state of the distribution transformer at time intervals, and uploading the voltage state to a corresponding voltage monitoring controller;
the voltage monitoring controller organizes the acquired voltage data and uploads the organized data to the data storage cluster;
the computing manager configures task related parameters and starts an analysis task in the computing cluster;
in the computing cluster, an analysis task reads metering data and archive data from a data storage cluster, establishes archive thermal data cache, fuses the metering and archive data, and calculates abnormal voltage marks and voltage out-of-limit time;
in the distributed parallel task of the computing cluster, parallel aggregation is carried out, one part is used for carrying out voltage abnormal rate computation, the other part is used for carrying out voltage out-of-limit condition computation, the computation results of the two parts are summarized, and the computation results are output as voltage basic analysis data of the same measurement point every day;
and generating a report according to the voltage basic analysis data of the same measurement point every day, and performing voltage quality control according to the report.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (8)
1. A power distribution station low-voltage monitoring method based on a big data technology is characterized by comprising the following steps:
monitoring and acquiring the voltage state of the distribution transformer at time intervals, and uploading the voltage state to a corresponding voltage monitoring controller;
the voltage monitoring controller organizes the collected voltage data into a batch of tissues according to the same collecting time point and then uploads the organized data to the data storage cluster;
the computing manager configures task related parameters and starts an analysis task in the computing cluster;
in the computing cluster, an analysis task reads metering data and archive data from a data storage cluster, establishes archive thermal data cache, fuses the metering and archive data, and calculates abnormal voltage marks and voltage out-of-limit time;
in distributed parallel tasks of the computing cluster, parallel aggregation is carried out, one part is used for carrying out voltage abnormal rate computation, the other part is used for carrying out voltage out-of-limit condition computation, the computation results of the two parts are summarized, and the computation results are output as daily same-measurement-point voltage basic analysis data;
and generating a report according to the voltage basic analysis data of the same measurement point every day, and performing voltage quality control according to the report.
2. The big data technology based power distribution area low voltage monitoring method according to claim 1,the voltage state data is represented as a voltage measurement vector Vi(voltage, point, code), wherein voltage is a three-phase voltage vector, point is a measurement point identifier, and code is a region code.
3. The power distribution area low-voltage monitoring method based on the big data technology as claimed in claim 1, wherein the establishing of the archival data cache specifically comprises:
before the calculation of the archive data, regular cleaning is needed to remove some invalid data and unnecessary data, the method has the characteristic of reading more and updating less, hot data cache after cleaning is established, the efficiency can be effectively improved, and the cache is abandoned for cleaning again when the archive is updated. The cleaning process is carried out on the data computing cluster in a distributed parallel mode, and the result is output to a specified directory of the data storage cluster by the data computing cluster in a file form with the same format as the archive data and is used as archive hot data cache;
the hot data cache is established in a data computing cluster, in a Filter operator, specific service rule conditions are combined, cleaning is carried out in a distributed parallel mode, and the result is output to a specified directory of the data storage cluster by the data computing cluster in a file with the same format as archive data, so that the hot data cache is established.
4. The power distribution area low-voltage monitoring method based on the big data technology according to claim 1, wherein the abnormal voltage marking and the voltage out-of-limit duration calculation are specifically:
voltage anomaly determination, defined as follows:
wherein, x is the determination result of the voltage value, normal is normal voltage, abnormal is abnormal voltage, ref is the reference voltage of the voltage grade corresponding to the current distribution transformer, voltage is the voltage value of the current distribution transformer, and lower _ rate and upper _ rate respectively represent the ratio of the lower limit to the upper limit;
and judging whether the voltage exceeds the upper limit or the lower limit, wherein the definition is as follows:
wherein, ylAnd yuThe method is characterized by representing the time length of an upper limit and the time length of an upper limit, wherein voltage is input voltage, gap _ time is sampling interval of a voltage monitor, lower is lower limit voltage, upper is upper limit voltage, and the definition is as follows:
wherein ref is a reference voltage of a corresponding voltage class of the current distribution transformer, lowerLimit is a lower limit index of the corresponding voltage class of the distribution transformer, and upperLimit is an upper limit index of the corresponding voltage class of the distribution transformer.
5. The power distribution area low-voltage monitoring method based on the big data technology as claimed in claim 1, wherein the voltage anomaly rate calculation is to count the ratio relationship between the number of voltage anomaly points and the number of normal points in the same measurement point;
and the voltage out-of-limit condition calculation is to aggregate voltage data identified by the same measuring point based on pre-calculated out-of-limit time in the Join operator, and calculate a voltage extreme value and time, out-of-limit time and voltage out-of-limit rate.
6. The power distribution area low-voltage monitoring method based on the big data technology as claimed in claim 1, wherein when two parts of calculation results are summarized, one part of voltage basic analysis data is generated for each measuring point, and the voltage basic analysis data comprises a qualification rate, a qualification time, a total overrun time and a total monitoring time; each piece of the voltage basic analysis data corresponds to a single measuring point, a daily report is generated for the measuring points with the same regional codes, the measuring points with low percent of pass are additionally recorded, the daily reports are collected into a weekly report and a monthly report, and the measuring points which are unqualified for a long time are tracked.
7. The power distribution area low-voltage monitoring system based on the big data technology is characterized by being applied to the power distribution area low-voltage monitoring method based on the big data technology in any one of claims 1-6, and comprising a voltage state acquisition module, a data uploading module, a task configuration module, a data processing module, a parallel aggregation module and a voltage quality management module;
the voltage state acquisition module is used for monitoring and acquiring the voltage state of the distribution transformer at time intervals and uploading the voltage state to the corresponding voltage monitoring controller;
the data uploading module is used for uploading the collected voltage data to the data storage cluster after the voltage monitoring controller organizes the collected voltage data into a batch of tissues according to the same collecting time point;
the task configuration module is used for configuring task related parameters by the computing manager and starting an analysis task in the computing cluster;
the data processing module is used for reading the metering data and the file data from the data storage cluster by the analysis task in the calculation cluster, establishing a file thermal data cache, fusing the metering data and the file data, and calculating the abnormal voltage mark and the voltage out-of-limit time length;
the parallel aggregation module is used for performing parallel aggregation in distributed parallel tasks of the computing cluster, performing voltage abnormal rate computation on one part, performing voltage out-of-limit condition computation on the other part, summarizing computation results of the two parts, and outputting the computation results as daily same-measurement-point voltage basic analysis data;
and the voltage quality control module is used for generating a report according to the voltage basic analysis data of the same measurement point every day and controlling the voltage quality according to the report.
8. A computer-readable storage medium storing a program, wherein the program, when executed by a processor, implements the big data technology-based distribution substation low voltage monitoring method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210091696.8A CN114528341B (en) | 2022-01-26 | 2022-01-26 | Distribution area low-voltage monitoring method and device based on big data technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210091696.8A CN114528341B (en) | 2022-01-26 | 2022-01-26 | Distribution area low-voltage monitoring method and device based on big data technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114528341A true CN114528341A (en) | 2022-05-24 |
| CN114528341B CN114528341B (en) | 2024-04-02 |
Family
ID=81623447
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210091696.8A Active CN114528341B (en) | 2022-01-26 | 2022-01-26 | Distribution area low-voltage monitoring method and device based on big data technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114528341B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180131185A1 (en) * | 2015-03-27 | 2018-05-10 | Kabushiki Kaisha Toshiba | Power distribution monitoring and control device |
| CN110601214A (en) * | 2019-10-30 | 2019-12-20 | 国网浙江浦江县供电有限公司 | Distribution network reactive voltage governance monitoring system |
| CN112622690A (en) * | 2020-11-09 | 2021-04-09 | 特瓦特能源科技有限公司 | Monitoring method and system for power battery safe charging |
| CN113642933A (en) * | 2021-09-06 | 2021-11-12 | 广东电网有限责任公司电力科学研究院 | Power distribution station low-voltage diagnosis method and device |
-
2022
- 2022-01-26 CN CN202210091696.8A patent/CN114528341B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180131185A1 (en) * | 2015-03-27 | 2018-05-10 | Kabushiki Kaisha Toshiba | Power distribution monitoring and control device |
| CN110601214A (en) * | 2019-10-30 | 2019-12-20 | 国网浙江浦江县供电有限公司 | Distribution network reactive voltage governance monitoring system |
| CN112622690A (en) * | 2020-11-09 | 2021-04-09 | 特瓦特能源科技有限公司 | Monitoring method and system for power battery safe charging |
| CN113642933A (en) * | 2021-09-06 | 2021-11-12 | 广东电网有限责任公司电力科学研究院 | Power distribution station low-voltage diagnosis method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114528341B (en) | 2024-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110022226B (en) | Object-oriented data acquisition system and acquisition method | |
| CN107402976B (en) | Power grid multi-source data fusion method and system based on multi-element heterogeneous model | |
| CN109286188B (en) | 10kV power distribution network theoretical line loss calculation method based on multi-source data set | |
| CN104317800A (en) | Hybrid storage system and method for mass intelligent power utilization data | |
| CN112559475B (en) | Data real-time capturing and transmitting method and system | |
| CN111523004B (en) | Storage method and system for edge computing gateway data | |
| CN111147959A (en) | Processing method and device for change information of centralized meter reading system equipment and computer equipment | |
| CN110941617A (en) | Power grid GIS data storage method based on version control | |
| CN116500385B (en) | Power transmission network monitoring and checking method, device, equipment and medium | |
| CN107862459B (en) | Metering equipment state evaluation method and system based on big data | |
| CN106849064B (en) | Regional power grid load prediction management system based on meteorological data | |
| CN105162622A (en) | Storage method and system | |
| CN104601374A (en) | Network fault processing method and device for numerical control machine tool | |
| CN114528341A (en) | Power distribution station low-voltage monitoring method and device based on big data technology | |
| CN117453695A (en) | Method, system, medium and equipment for storing meter reading data based on low-voltage transformer area | |
| CN112381583A (en) | Power consumption calculation method and device based on distributed memory calculation technology | |
| CN112148893A (en) | Energy analysis knowledge graph construction method and energy analysis visualization method | |
| CN110134690B (en) | Method and system for rapidly storing oversized data of Oracle database | |
| Jin et al. | Research on Wide-area Distributed Power Quality Data Fusion Technology of Power Grid | |
| CN114418237B (en) | Distribution network power supply safety capability evaluation standard quantification method, system, equipment and medium | |
| CN113537824B (en) | Full-link monitoring system for power grid data | |
| CN111625190B (en) | Method and device for realizing electric quantity storage based on power distribution characteristics | |
| Zhou et al. | Big data storage and parallel analysis of grid equipment monitoring system | |
| CN112614017A (en) | Power failure information management system, power failure information management method, and storage medium | |
| CN112380164B (en) | Electric power system scene data management method, device and system based on snapshot technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |