KR20190050682A - Pattern Tagging based Power Data Storing and Management System - Google Patents

Abstract

Provided are a method and a system for storing and managing power data based on a pattern tagging technology. The method for storing and managing power data based on a pattern tagging technology comprises the following steps: selecting multiple types of data among entire data samples; extracting patterns through unsupervised learning of average point movement clustering using a correlation value for the multiple types of selected data; classifying the entire data samples into the most similar patterns after extracting the patterns; inserting and storing the classified patterns together with tags into the data; and visualizing power usage data and pattern data using the tags of the inserted data.

Description

{Pattern Tagging based Power Data Storing and Management System}

The present invention relates to a method and system for managing power data storage based on pattern tagging technology.

Currently, the IoT market is exploding. By 2015, more than 4.9 billion devices have been installed, and by 2020 there will be around 20.8 billion devices installed.

Power-related Advanced Metering Infrastructure (AMI) is a small part of the IoT market. Approximately 65 million power-related advanced meter devices are deployed in the United States in 2015. This means that smart metering devices account for more than 1.32% of the total deployed IoT devices. Considering the diversity of devices in the IoT market and the number of power smart metering devices deployed in the US, 1.32% indicates that the power industry is a large part of IoT. Considering that IoT devices include personal phones, computers, etc., one of the 100 IoT devices is a power measurement device.

IoT data is generated and collected quickly and continuously from sensor networks consisting of a large number of sensors. As a new kind of data, its characteristics are different from existing data. The amount of data generated / collected is vast, irregular, and streaming. Depending on the industry and device type, the characteristics of each data and the customized processing method for each application are still used in analysis and management stages.

In the storage and management of existing IoT power data, the characteristics of the data are stored and managed in the same way as the non-IoT data. Although the customized processing and analysis methods that are tailored to the characteristics of each industry, field, and purpose have been continuously studied and used in data use, methods and systems suited to the characteristics of IoT data are not available when storing and managing data. In the case of power data, the data values are stored in a single row on the time axis based on the user and date when storing and managing the collected power usage.

The existing data storage management method has the following problems. Currently, many IoT data storage methods collect data, store it without analysis at the time of storage, apply it later, and use it appropriately through analysis when using it. Unlike the non-IoTs, the IoT data, which is high in quantity and irregular in streaming, is stored in the same manner as the non-IoT data, so that it is difficult to search for the management, to grasp the meaning and to use it. Therefore, in IoT, analysis should be performed before the real-time value of the data disappears, and analysis through preprocessing is required before data is stored.

SUMMARY OF THE INVENTION The present invention is directed to a technology for efficiently storing and managing smart metering devices and power data in an IoT environment. In the data gathering and storing step, pattern tags are inserted through preprocess analysis, We use pattern tagging technology which makes it easy to search, manage, and use data by inserting tag and inserting tag by dividing data into each pattern. Power data storage management method and system.

According to an aspect of the present invention, there is provided a method for managing a power data storage based on a pattern tagging technique, comprising: selecting a plurality of data from among all data samples; Extracting the patterns, extracting the patterns and dividing the entire data samples into the most similar patterns, inserting and storing the divided patterns together with the tags in the data, and using the tags of the inserted data And visualizing power usage data and pattern data.

The step of extracting patterns through learning of averaging point clustering non-affine mapping using the correlation value for the selected plurality of data includes a step of extracting a plurality of power use values used by the user each day as one vector having a plurality of features And calculate the correlation value by measuring the similarity of each vector, and apply the average moving clustering.

The step of dividing all the data samples into the most similar patterns after extracting the patterns includes learning the distribution diagrams of the plurality of data, and moving the clusters using the correlation values. The vectors having high similarity among the patterns extracted through the learning of the non- .

After forming the clusters and performing outlier handling, each cluster centroid is extracted using each of the final clusters as one pattern.

The step of inserting and storing the divided patterns into the data together with the tags may include inserting the corresponding pattern tags by distinguishing patterns of existing data and newly generated data using the information of the patterns, To update an existing database or to construct a new database.

According to another aspect of the present invention, there is provided a method of managing a power data storage based on pattern tagging technology, comprising: a data generation unit for selecting a plurality of data among all data samples; A data analyzing unit for extracting patterns through learning of moving clustering non-affine mapping, extracting the patterns, and then dividing the entire data samples into the most similar patterns; storing the divided patterns in data together with the tags in the data storage unit; And a visualization unit for visualizing the power usage data and the pattern data using the data structure unit and the tag of the inserted data.

The data analysis unit defines a plurality of power usage values used by a user in one day as one vector having a plurality of features, calculates a correlation value by measuring similarity of each vector, and applies average mobile clustering.

The data analyzer learns a distribution diagram of a plurality of data and forms clusters among vectors having high similarity among the patterns extracted through the averaging shift clustering non-index learning using correlation values.

The data analyzer forms the clusters and extracts cluster centroids of each of the final clusters as one pattern after outlier handling.

The data structure unit divides patterns of existing data and newly generated data by using the information of the patterns, inserts the corresponding pattern tags, and updates the existing database by issuing the tag-inserted data, Build the base.

According to embodiments of the present invention, pattern recognition and extraction of data is a fundamental step of data analysis, and can be widely used in various application fields of power data, and is expected to be utilized in various fields. Usage forecasting, power trading modeling, anomaly detection systems, user analysis, and energy management. Various service models such as power consumption customized service development and management improvement system using pattern tag are possible.

FIG. 1 is a flowchart illustrating a method of managing a power data storage based on a pattern tagging technique according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining learning of a moving average clustering busy according to an exemplary embodiment of the present invention. Referring to FIG.
FIG. 3 is a diagram for explaining an averaging shift clustering beacon learning using a correlation value according to an embodiment of the present invention. Referring to FIG.
4 and 5 are diagrams for explaining a process of storing power data based on a pattern tagging technique according to an embodiment of the present invention.
6 is a diagram illustrating a configuration of a power data storage management system based on a pattern tagging technology according to an embodiment of the present invention.
7 is a view for explaining a visualization process according to an embodiment of the present invention.

As the number of connected devices increases, the IoT market continues to grow explosively, and electric power-related smart metering is a small part of it. The energy-related market is also growing with Smart City / IoT.

According to the embodiment of the present invention, efficiency in the storage, management, processing and visualization of energy-related IoT data is enhanced in an environment where power data is generated geometrically in proportion to the growing IoT, the energy market and the increasing number of smart metering devices And the cost reduction effect can be expected.

In addition, when IoT, smart buildings, cars, and cities are in the spotlight, new business models based on extracted patterns can be derived and commercialization is highly possible.

In addition, it can be applied to various companies such as electric power companies that handle power data, building companies that build smart buildings, and IoT companies that research, develop and produce IoT devices. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a flowchart illustrating a method of managing a power data storage based on a pattern tagging technique according to an embodiment of the present invention.

The proposed pattern tagging technology based power data storage management method includes a step 110 of selecting a plurality of data among all data samples, a step of calculating a pattern using a correlation point, Extracting the patterns and dividing the entire data samples into the most similar patterns 130, inserting and storing the separated patterns in the data together with the tags 140, And visualizing the power consumption data and the pattern data using the tag of the power consumption data.

At step 110, a plurality of data are selected from among the entire data samples.

In step 120, patterns are extracted through averaging shift clustering bitmap learning using a correlation value for the selected plurality of data.

A plurality of power usage values used by a user for a day are defined as one vector having a plurality of features, and average mobile clustering is applied by calculating a correlation value by measuring similarity of each vector.

FIG. 2 is a diagram for explaining learning of a moving average clustering busy according to an exemplary embodiment of the present invention. Referring to FIG.

The non-background learning according to the embodiment of the present invention is a type of machine learning and is used in various fields. When the distribution of data or the actual label value of the data is unknown, it is used to grasp the hidden characteristics and structure of the data, and the hidden patterns of the data can be found by utilizing the identified important features. The pattern referred to here is a group of data having any common attribute, and the data in one pattern does not need to be the same, but consists of data having properties most similar to each other in some criterion.

Mean-shift clustering according to an exemplary embodiment of the present invention begins with a single cluster (i.e., a pattern), moves clusters to a place where data density is high, .

Gaussian kernel density estimate can be calculated using the following equation for learning the average moving clustering likelihood.

는 총 N개의 데이터를 포함하는 데이터 셋 중 n번째 즉 하나의 데이터 포인트를 뜻하며,

는 전체 데이터 셋의 분포를 나타낸다. K(t)는 커널 함수를 뜻하며(가우시안의 경우

), 위의 식에서는 가우시안 커널이 사용되었다. σ는 자유 파라미터로써 데이터들의 이동시 이동 반경을 제한하는 파라미터이다. 최종적으로 데이터 셋의 분포 변동이 없어지는 지점(

)까지 계속하여 데이터 포인트 이동 함수의 식(

)을 통하여 각 데이터 포인트(

)들을 이동시켜준다.here,

Denotes an n-th data point, i.e., one data point, of a data set including a total of N data,

Represents the distribution of the entire dataset. K (t) is the kernel function (Gaussian case

), Gaussian kernel was used in the above equation. σ is a free parameter and is a parameter that limits the moving radius when moving data. Finally, at the point where the distribution variation of the dataset disappears (

) To the data point shift function (

) To each data point (

).

Referring to FIG. 2, the cluster 1 211 moves to a higher density region and is divided into a pattern 1 212, a cluster 2 221 moves to a higher density region and is divided into a pattern 2 222, 3 (231) moves to a higher density region and is divided into a pattern 3 (232).

In step 130, after extracting the patterns, all data samples are divided into the most similar patterns. Learning the distribution of a plurality of data and moving averaging points using correlation values. Clustering among the vectors with high similarity among the patterns extracted through the learning of the beacon is formed. Clusters are formed, and after each outlier handling, each cluster centroid is extracted using each of the final clusters as a pattern.

FIG. 3 is a diagram for explaining an averaging shift clustering beacon learning using a correlation value according to an embodiment of the present invention. Referring to FIG.

The averaged point clustering based on the correlation value, that is, the correlation distance, is performed by considering n power use values of one user per day as one vector having n features.

Referring to FIG. 3, N power usage values per day for user M, K power usage values per day for each user L, and L power usage values per day for user M are shown as one vector 340. Then, the correlation values are calculated by similarity measurement of the vector, and the average moving clustering is applied, and the distribution diagram of the data is learned to form a cluster of vectors having high similarity. After outlier handling, each of the final clusters is represented as a pattern, and each cluster centroid is extracted as a pattern.

We can calculate the correlation coefficient and the distance measurement for the average point clustering non-geographic learning using the correlation distance using the following equation.

또는

는 d길이를 가지는 하나의 벡터 데이터 또는 d차원의 하나의 데이터 포인트로 해석될 수 있으며,

는

의 분산 그리고

는 두 데이터간의 공분산을 뜻한다. 피어슨 상관계수(

)에 기반한 두 데이터 포인트의 거리 함수(

)를 이용하여 비유사도를 측정한다.

or

Can be interpreted as one vector data having a length d or one data point having a dimension d,

The

And

Is the covariance between two data. Pearson correlation coefficient (

) Based on the distance function of two data points (

).

In step 140, the divided patterns are inserted into the data together with the tags and stored. By using the information of the patterns, the existing pattern data is distinguished from the pattern of the newly generated data, the corresponding pattern tag is inserted, and the inserted data is issued to update the existing database or construct a new database.

4 and 5 are diagrams for explaining a process of storing power data based on a pattern tagging technique according to an embodiment of the present invention.

In the preprocessing step, a pattern of IoT power data is automatically detected through learning of average moving clustering busy information according to an embodiment of the present invention, and a pattern is inserted and stored together with a tag when data is stored, Can be made easy and efficient.

Referring to FIG. 4, a daily power consumption graph 410 of one user is shown as a data set 400 from Day 1 to Day from the user 1 (User 1) to the user L (User L). Patterns 420 (pattern1, pattern2, pattern3, pattern4, pattern5, pattern6, ..., patternN) identified using the pattern classification through the non-background learning are shown for these data sets.

Finally, the pattern thus classified can be inserted into the data together with the pattern tag 430 and stored, and the existing data base can be updated or the new data 440 can be constructed.

Referring to FIG. 5, a part of data is randomly selected (510) in the entire data sample 511 and pattern extraction 521 is performed through non-feature learning. After extracting the pattern, the entire data sample is divided into a suitable pattern (that is, the similarity is high), and the pattern tag 540 is stored by inserting the divided pattern into the data together with the tag (530).

In step 150, the power usage data and the pattern data can be visualized using the tag of the inserted data. The power consumption data can be visualized using the inserted data tag, and the pattern data can be visualized using the inserted data tag.

6 is a diagram illustrating a configuration of a power data storage management system based on a pattern tagging technology according to an embodiment of the present invention.

The proposed pattern tagging technology based power data storage management system includes a data management unit 600 including a data generation unit 610, a data analysis unit 620 and a data structure unit 630, a data storage unit 640, (650).

The data generation unit 610 selects a plurality of data from among all the data samples.

The data analyzer 620 extracts patterns by learning an average moving clustering likelihood using a correlation value for a plurality of selected data, extracts the patterns, and divides all data samples into the most similar patterns.

A plurality of power usage values used by a user for a day are defined as one vector having a plurality of features, and average mobile clustering is applied by calculating a correlation value by measuring similarity of each vector.

Then, a distribution diagram of a plurality of data is learned, and vectors having high similarity among patterns extracted through the averaging shift clustering non-degree learning using a correlation value are formed into clusters. Clusters are formed, and after each outlier handling, each cluster centroid is extracted using each of the final clusters as a pattern.

The data analyzer 620 includes a pattern recognizer 621, a pattern storage 622, and a pattern issuer 623.

The pattern recognizer 621 receives the information for each database stored in the data storage unit 640 and executes pattern recognition and extraction by learning clustering bitmap.

In the plurality of data, randomly selected data is selected and a preprocessing process for standardizing the selected data is performed. Then, average mobile clustering is applied.

After extracting the patterns, we divide the whole data samples into the most similar patterns. Learning the distribution of a plurality of data and moving averaging points using correlation values. Clustering among the vectors with high similarity among the patterns extracted through the learning of the beacon is formed. Clusters are formed to perform post-processing of outlier handling, and each cluster centroid is extracted using each of the final clusters as one pattern.

The pattern storage unit 622 stores each cluster center value in a pattern of each of the last clusters. In other words, the recognized and extracted patterns are stored separately for each database.

The pattern issuer 623 issues and transmits a stored pattern to provide pattern information to other modules.

The data structure unit 630 inserts the divided patterns together with the tags into the data, and stores the inserted patterns in the data storage unit 640. By using the information of the patterns, the existing pattern data is distinguished from the pattern of the newly generated data, the corresponding pattern tag is inserted, and the inserted data is issued to update the existing database or construct a new database.

The data structure unit 630 includes a data tag 631 and a tagged data issuer 632.

The data tagger 631 classifies the entire data into an appropriate pattern and inserts the pattern tag into the original data. Using the pattern information received from the data analysis unit 620, separates the existing data from the newly generated data pattern and inserts the corresponding pattern tag.

The tagged data issuer 632 issues the tagged data and updates the existing database or builds a new database.

The data storage unit 640 inserts the generated patterns together with the tags and stores the inserted data.

The visualization unit 650 visualizes the power consumption data and the pattern data using the tag of the inserted data.

The visualization unit 650 includes a data visualizer 651 and a pattern visualizer 652. The data visualizer 651 visualizes the power consumption data using the inserted data tag, and the pattern visualizer 652 can visualize the pattern data using the inserted data tag.

7 is a view for explaining a visualization process according to an embodiment of the present invention.

The visualization unit according to an embodiment of the present invention firstly displays the recognized power usage pattern as a graph 710 and displays the inserted pattern for each data 720. Then, the pattern distribution ratio for each user and business can be displayed (730) and visualized.

The IoT market continues to grow, with a growing number of smart metering devices associated with the power industry and the resulting financial savings. The storage, management, processing, and visualization markets for energy-related IoT data are also growing in an environment where power data is geometrically generated in proportion to the growing IoT market and the growing number of smart metering devices. Pattern extraction and classification are the basic steps of data analysis, and related technologies are expected to be used in various fields of analysis of energy data. Examples include usage forecasting, abnormal event prediction or detection, missing data interpolation, user analysis, power trading, and energy management. Therefore, it is also possible to derive a new business model based on the extracted pattern utilization.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device As shown in FIG. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (12)

Comprising: selecting a plurality of data from among all data samples;
Extracting patterns by learning an average moving clustering bitmap using a correlation value for the selected plurality of data;
Dividing all data samples into the most similar patterns after extracting the patterns; And
Inserting and storing the divided patterns together with the tags in the data
The power data storage management method comprising: The method according to claim 1,
The step of extracting patterns by learning the average point moving clustering likelihood using a correlation value for the selected plurality of data,
A plurality of power usage values that a user uses in one day are defined as one vector having a plurality of features, a correlation value is calculated by measuring similarity of each vector, and average mobile clustering is applied
Power data storage management method. The method according to claim 1,
Dividing the entire data samples into the most similar patterns after extracting the patterns,
Learning the distribution of a plurality of data and moving averaging points using correlation values, vectors having high similarity among the patterns extracted through the learning of non-mapping of clusters form clusters
Power data storage management method. The method of claim 3,
After forming the clusters and performing outlier handling, each cluster centroid is extracted using each of the final clusters as one pattern
Power data storage management method. The method according to claim 1,
The step of inserting and storing the divided patterns together with the tags into the data may include:
Using the information of the patterns, it is possible to distinguish patterns of existing data and newly generated data, inserting the corresponding pattern tag, and updating the existing database by issuing the tag-inserted data or building a new database
Power data storage management method. The method according to claim 1,
Visualizing power usage data and pattern data using the tags of the inserted data
Further comprising the steps of: A data generation unit for selecting a plurality of data from all the data samples;
A data analyzer for extracting patterns by learning averaging point clustering non-affinity learning using a correlation value for a plurality of selected data, dividing all data samples into the most similar patterns after extracting the patterns; And
A data structuring unit for inserting the divided patterns together with tags into data and storing the data in a data storage unit,
And the power data storage management system. 8. The method of claim 7,
The data analysis unit may include:
A plurality of power usage values that a user uses in one day are defined as one vector having a plurality of features, a correlation value is calculated by measuring similarity of each vector, and average mobile clustering is applied
Power data storage management system. 8. The method of claim 7,
The data analysis unit may include:
Learning the distribution of a plurality of data and moving averaging points using correlation values, vectors having high similarity among the patterns extracted through the learning of non-mapping of clusters form clusters
Power data storage management system. 10. The method of claim 9,
The data analysis unit may include:
After forming the clusters and performing outlier handling, each cluster centroid is extracted using each of the final clusters as one pattern
Power data storage management system. 8. The method of claim 7,
The data structure unit may include:
Using the information of the patterns, it is possible to distinguish patterns of existing data and newly generated data, inserting the corresponding pattern tag, and updating the existing database by issuing the tag-inserted data or building a new database
Power data storage management system. 8. The method of claim 7,
A visualization unit for visualizing power consumption data and pattern data using tags of the inserted data
The power data storage management system further comprising:

Images