CN111027612B - Energy metering data feature reduction method and device based on weighted entropy FCM - Google Patents
Energy metering data feature reduction method and device based on weighted entropy FCM Download PDFInfo
- Publication number
- CN111027612B CN111027612B CN201911226594.7A CN201911226594A CN111027612B CN 111027612 B CN111027612 B CN 111027612B CN 201911226594 A CN201911226594 A CN 201911226594A CN 111027612 B CN111027612 B CN 111027612B
- Authority
- CN
- China
- Prior art keywords
- data
- reduction
- feature
- fcm
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000009467 reduction Effects 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 28
- 238000000605 extraction Methods 0.000 claims abstract description 15
- 239000000203 mixture Substances 0.000 claims abstract description 8
- 230000002159 abnormal effect Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 7
- 238000011946 reduction process Methods 0.000 claims description 4
- 238000006467 substitution reaction Methods 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims description 2
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 238000012216 screening Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 27
- 238000004422 calculation algorithm Methods 0.000 description 16
- 230000008569 process Effects 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 239000006185 dispersion Substances 0.000 description 5
- 238000003860 storage Methods 0.000 description 3
- 238000007476 Maximum Likelihood Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Complex Calculations (AREA)
Abstract
The invention provides an energy metering data feature reduction method and device based on weighted entropy FCM, and relates to the technical field of power system automation, and the technical characteristics are that the method comprises the following steps: step 1: acquiring data to be processed; step 2: carrying out feature extraction on the data information of the data to be processed by adopting a Gaussian mixture model; step 3: performing reduction processing on the features extracted in the step 2 based on the FCM function; step 4: under the condition that the data reduction convergence is determined, finishing feature reduction, and exiting from the execution logic; and under the condition that the reduction of the number is not converged, the method of updating the weight is adopted to remove smaller features, and then the step 3 is executed, so that the reduction is carried out. The invention is beneficial to accurately and real-timely screening the characteristic data.
Description
Technical Field
The invention relates to the technical field of power system automation, in particular to a data feature reduction method and device based on feature weighted entropy FCM.
Background
The power industry is in the digital information age, the information volume is in explosive growth, and the large data age of electric power is accompanied by the explosion, so people enjoy the information and convenience brought by the information, and simultaneously face the huge challenge brought by the huge information volume, namely the availability of the information. The power big data has the characteristics of 4"V', namely large Volume, rapid growth (speed), multiple categories (Variety) and thin Value (Value) density. According to 4"V' characteristics of large electric power data, to obtain decision information beneficial to power grid operation from mass data which are rapidly generated, multi-source, heterogeneous and have low value, the decision information needs to be processed, namely, the inherent relevance among all attribute data is found out, and useful information beneficial to power grid decision is screened from the mass information. The data reduction is a key link in the big data preprocessing, reduces the data processing amount in the big data analysis process, and improves the data processing efficiency.
Traditional data reduction modes comprise a PCA manifold learning dimension reduction method and a RELIEF algorithm, but both algorithms have certain limitations: the PCA manifold learning dimension reduction method cannot solve the uncertainty of the mapping of data points in the high-dimensional space to the low-dimensional space, and the corresponding inverse mapping cannot be given, so that the method can only be used for the potential low-dimensional structure of the data set. The RELIEF algorithm can process discrete data and continuous data, but cannot be applied to various problems, and the algorithm does not consider the situation of data loss and cannot remove redundant characteristics. Therefore, the algorithm cannot cope with probability calculation with high complexity, and probability value is difficult to estimate accurately.
Therefore, in view of the above drawbacks, there is a need to develop a new data feature reduction method.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides an energy metering data feature reduction method and device with high accuracy and good real-time performance based on weighted entropy FCM, so as to solve the technical problem that the existing data processing mode cannot effectively screen out feature data.
In order to achieve the above purpose, the present invention provides the following technical solutions:
the data feature reduction method based on the feature weighted entropy FCM provided by the invention comprises the following steps:
step 1: acquiring data to be processed;
step 2: extracting data information characteristics of the data to be processed by adopting a Gaussian mixture model;
step 3: performing reduction processing on the features extracted in the step 2 based on the FCM function;
step 4: under the condition that the data reduction convergence is determined, finishing feature reduction, and exiting from the execution logic; under the condition that the reduction of the number is not converged, the reduction treatment is carried out again after the smaller features are removed by adopting a weight updating method, namely, the step 3 is repeated;
furthermore, the pretreatment step between the step 1 and the step 2 further comprises the following pretreatment steps:
(1) Judging abnormal data of the data to be processed by adopting a 3 sigma principle and correcting the abnormal data;
(2) Establishing a difference function by using known points, and carrying out interpolation substitution processing on the missing values in the corrected data;
(3) And (3) smoothing technology is used for the corrected and interpolated data set to reduce data noise.
The specific method for extracting the data features in the step 3 is as follows:
GMM modeling is carried out on the data, the best GMM fitting parameters are calculated, the GMM model parameters are extracted as characteristic variables of the original data, so that characteristic extraction is carried out, and the built GMM model is as follows:
wherein x is the input sample variable; pi i Is a mixed weight and hasd is the dimension of the input variable; p is p i (x；μ i ,Σ i ) An ith gaussian component of GMM; mu (mu) i Is the mean value of the ith Gaussian component; sigma (sigma) i Is the covariance matrix of the ith gaussian component.
The specific method for the feature reduction processing in the step 4 is as follows:
FCM is improved by reducing features in the feature weighted entropy, the objective function is:
wherein:δ j for controlling feature weights, w j As the feature weight of the jth feature, W is the matrix formed by the features, U is the membership matrix, and V is the clustering center.
The invention also provides an energy metering data feature reduction device based on the weighted entropy FCM, which comprises:
the data acquisition unit is used for acquiring data to be processed;
the feature extraction unit is used for carrying out feature extraction processing on the data to be processed acquired by the data acquisition unit based on the Gaussian mixture model;
the reduction unit is used for carrying out reduction processing on the features extracted by the feature extraction unit based on the FCM function, and completing feature reduction under the condition that the convergence of data reduction is determined, and exiting from the execution logic; and under the condition that the reduction of the number is not converged, adopting a method for updating the weight to remove smaller features and returning to the reduction process.
The invention has the advantages and beneficial effects that:
1. the invention adopts the fuzzy C-means clustering (FRFCM) characteristic reduction method based on weighted entropy, and can add the weighted entropy into the FRFCM objective function, thereby automatically reducing the characteristic quantity and generating good clustering effect.
2. According to the FRFCM algorithm, the characteristic reduction behavior with the characteristic weight is embedded in the FCM process, the performance of the FCM can be improved, important characteristics can be selected through weighting, the number of the characteristics is reduced by discarding unimportant characteristics, and the purpose of characteristic reduction is achieved.
3. The calculation method adopted by the invention can utilize the reduced important features to perform clustering, can ensure the accuracy and simultaneously give consideration to the real-time performance, and can greatly reduce the calculation time.
4. The calculation method selected by the invention can estimate the parameters used in the FRFCM objective function in the learning process, so that the FRFCM algorithm is not limited by parameter selection, and the use is more flexible and convenient.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a process flow diagram of the present invention;
FIG. 2 is a flow chart of step 2 of the present invention;
fig. 3 is a flow chart of step 3 in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, based on the examples herein, which are within the scope of the invention as defined by the claims, will be within the scope of the invention as defined by the claims.
Embodiments of the invention are described in further detail below with reference to the attached drawing figures:
an energy metering data characteristic reduction method based on weighted entropy FCM comprises the following steps:
step 1: acquiring data to be processed; the data to be processed is energy metering data;
step 2: extracting data information characteristics of the data to be processed by adopting a Gaussian mixture model;
the specific method for extracting the characteristics in the step 2 is as follows:
GMM modeling is carried out on the data, the best GMM fitting parameters are calculated, the GMM model parameters are extracted as characteristic variables of the original data, so that characteristic extraction is carried out, and the built GMM model is as follows:
wherein x is the input sample variable; pi i Is a mixed weight and hasd is the dimension of the input variable; p is p i (x；μ i ,Σ i ) An ith gaussian component of GMM; mu (mu) i Is the mean value of the ith Gaussian component; sigma (sigma) i Is the covariance matrix of the ith gaussian component.
When in use, the GMM model generally adopts a maximum likelihood estimation method, and likelihood functions are as follows:
L(λ|x)＝p(X|λ)＝Πp(x i |λ)
the purpose of training is to find a set ofSo that the likelihood functionThe number L (λ|x) takes the maximum value, namely:
maximum likelihood estimation is a problem of nonlinear optimization, generally by iterative solution of a expectation maximization algorithm (EM algorithm), which is mainly divided into two steps: step E is desirable and step M is maximized. E, calculating an expected value of a likelihood function of the complete data by using the current parameter set; and step M, obtaining new parameters through maximization of an expected function, and iterating the step E and the step M until convergence.
To illustrate the EM algorithm, first define the Q function:
where y is a hidden signal, indicating the sequence number of the gaussian component currently being generated, and if it is generated by the ith gaussian component, y=i, λ is the existing model parameter,is a new parameter to be calculated.
The Q function is therefore rewritten as follows:
after defining the Q function, it is necessary to set the initial value of the EM algorithmAnd iteration termination conditions, the solving process is simply described as follows:
1) Solving for the data in the training dataset to fall in the assumed hidden stateThe probability of (2) isThen: />
2) Maximizing the Q function, solvingRelative to pi i ；μ i ；Σ i I=1, 2.. i ；μ i ；∑ i 。
The features extracted by the method can be used for feature reduction processing.
It should be noted that, in the case where there may be a missing value or an erroneous value in the data to be processed, the feature reduction process may be performed by a preprocessing step, where the preprocessing step includes:
(1) Judging abnormal data of the data to be processed by adopting a 3 sigma principle and correcting the abnormal data;
the following formula is selected for abnormal number judgment:
wherein: epsilon is a threshold, typically 1-1.5;
the following formula is selected for the abnormal number correction:
wherein: p's' i,t The load data corrected at the ith time t of the user; α, β, γ are weight coefficients, and satisfy α+β+γ=1;and->Is the distance p i,t Recently, it has been proposed toIs a daily load of the observation.
(2) Establishing a difference function by using known points, and carrying out interpolation substitution processing on the missing values in the corrected data; the specific method comprises the following steps:
processing missing values Using a tri-spline interpolation to process missing values, i.e., the known function y=f (x) is in the interval [ a, b ]]N+1 nodes above: a=x 0 ＜x 1 ＜…＜x n-1 ＜x n Value y on =b i ＝f(x i ) (i=0, 1, …, n), the difference function S (x) is calculated such that:
A：S(x)＝y i (i＝0,1,…,n)；
b: between each cell [ x i ,x i+1 ]S (x) on (i=0, 1, …, n-1) is a cubic polynomial denoted as S i (x)；
C: s (x) is continuously differentiable in second order on [ a, b ];
at this time, the function S (x) is called a cubic spline difference function of f (x).
The condition B can be expressed as: s (x) = { S i (x),x∈[x i ,x i+1 ],i＝0,1,…,n-1}，S i (x)＝a i x 3 +b i x 2 +c i x+d, wherein: a, a i ,b i ,c i ,d i For the undetermined coefficients, 4n in total.
From the condition C, it can be seen that:
meanwhile, the total of 4n-2 equations can be known according to the formulas used for data preprocessing, and two boundary conditions are needed to be given for determining 4n undetermined parameters of S (x).
The boundary conditions of the usual cubic spline function are of three types:
1)S'(a)＝y' 0 ，S'(b)＝y' n . The spline interpolation function established by such boundary conditions is called the complete spline difference function of f (x). In particular y' 0 ＝y' n When=0, the spline curve is in a horizontal state at the end points.
2)S”(a)＝y” 0 ，S”(b)＝y” n . In particular, y' 0 ＝y” n =0, called natural boundary condition.
3) S '(a+0) =s' (b-0), S "(a+0) =s" (b-0), this condition is referred to as a cycle condition.
Any formula is selected according to the actual situation to determine the boundary condition.
The interpolation substitution process is to supplement the missing values in the data.
(3) The smoothing technology is used for reducing data noise of the corrected and interpolated data after replacing processing, and the specific method is as follows:
wherein p is i,t The original load data at the time t of the ith day of the user; n is the total number of observation days;and->The average load and variance at time t of the user, respectively.
In the preprocessing step, the operation sequences of (1) and (2) can be interchanged, namely, the missing value in the data can be interpolated to replace the processing, and then the correction processing can be carried out on the abnormal data in the obtained data.
Step 3: the extracted features are reduced according to the fuzzy C-means clustering mode of the feature weighted entropy, namely the features extracted in the step 2 are reduced based on the FCM function;
the specific method for the reduction treatment in the step 3 is as follows:
FCM is improved by reducing features in the feature weighted entropy, the objective function is:
wherein,δ j for controlling the feature weights.
It should be noted that, in order to ensure that the calculation result meets the requirement, δ j Is important.
The FRFCM objective function has two items, wherein the first item is the sum of the weighted distances between the data points and the characteristic of the cluster center, and when the distance between the data points and the center is smaller, the weighted distances between the characteristic and the cluster center are the smallest; the second term is the feature weight entropyVariations of (2), e.gBecause of delta j At->And->Variables for controlling feature weights, so delta j Is important.
For more convenient estimation of delta j The invention also provides a learning program.
In probability theory and statistics, standard deviation and variance are used to measure the dispersion of data. Another measured dispersion index is a well-known variance-to-mean ratio (VMR), defined as vmr=σ 2 And/u. VMR can be used to observe scattered or aggregated data sets, with smaller dispersion indicating that the data set is closer to the cluster center; the larger the dispersion, the farther the data set is from the cluster center. Because of the need to preserve features with small dispersion, then discard the ionsFeatures of large divergence, the inverse of VMR, i.e., mean-to-variance ratio (MVR), should be considered for use in the algorithm. That is, δ can be considered j The estimation is as follows:
to make delta j Can play the screening role and normalize the screening role to delta j '＝δ j /max(δ j ) So that delta j ' E (0, 1), replaces the original delta j 。
In performing the FRFCM calculation, three minimization steps may be used to do so:
the first step: fixingAnd->Then minimize with respect to U considerations>Taking into account the lagrangian function:
the Lagrangian function is applied to u ik The partial derivative is set to zero, resulting in:
thereby obtaining u ik The update equation of (2) is as follows:
and a second step of: fixingAnd->Then minimize with respect to V considerations>The method comprises the following steps:
thereby obtaining v ik The update equation of (2) is as follows:
and a third step of: fixingAnd->Then consider minimize +.>The method comprises the following steps:
thereby obtaining w j The update equation of (2) is as follows:
in addition, to maintain constraintsIt is necessary to add w j Adjust to->
It should be noted that during actual data processing, some insignificant features (i.e., small weight features) need to be culled because the FRFCM algorithm belongs to feature reduction. For better data rejection, a suitable threshold value needs to be selected: if w j In the update equation of (2)Too large, then molecule->Becomes too small to approach a value of 0, resulting in excessive feature weights being lost in the update step; if, converselyToo small, the molecule->It approaches 1 resulting in a difficult to discard feature during the update step. To avoid this, a suitable constant is set in the FRFCM clustering algorithm to control. Because in the FRFCM clustering algorithm, one goal is to cluster a dataset (with n data points) into c clusters. The numbers n and c are two generally given constants, so the constant n/c can be used to controlSo as not to present the above-mentioned extreme cases. Review +.>The constant n/c is chosen for the treatment of +.>
In order to determine whether the feature reduction converges, a certain threshold value needs to be set for determination.
The data set to be processed has n data points, wherein each data point has d characteristic components, and the constraint condition of the characteristic weights is thatIf d is large, the feature reduction threshold is intuitively chosen to be 1/d.
In order to further improve the applicability of the algorithm, it is necessary to consider the data number n as another influencing factor, considering that the value of d may be small. Is known to beIn order to balance between a large d and a small d, a data n is chosen to replace one d, so that it becomes +.>When the data point data n is much larger than the bit component number d, select +.>Instead of 1/d, inaccuracy is achieved, in which case +.>And more accurate instead of 1/d.
The invention also provides an energy metering data feature reduction device based on the weighted entropy FCM, which comprises:
the data acquisition unit is used for acquiring data to be processed;
the feature extraction unit is used for carrying out feature extraction processing on the data to be processed acquired by the data acquisition unit based on the Gaussian mixture model;
the reduction unit is used for carrying out reduction processing on the features extracted by the feature extraction unit based on the FCM function, and completing feature reduction under the condition that the convergence of data reduction is determined, and exiting from the execution logic; and under the condition that the reduction of the number is not converged, adopting a method for updating the weight to remove smaller features and returning to the reduction process.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical aspects of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the above embodiments, it should be understood by those of ordinary skill in the art that: modifications and equivalents may be made to the specific embodiments of the invention without departing from the spirit and scope of the invention, which is intended to be covered by the claims.
Claims (7)
1. The energy metering data characteristic reduction method based on the weighted entropy FCM is characterized by comprising the following steps of:
step 1: acquiring data to be processed;
step 2: carrying out feature extraction on the data information of the data to be processed by adopting a Gaussian mixture model;
step 3: performing reduction processing on the features extracted in the step 2 based on the FCM function;
step 4: under the condition that the data reduction convergence is determined, finishing feature reduction, and exiting from the execution logic; under the condition that the reduction of the number is not converged, the method of updating the weight is adopted to remove smaller features, and then the step 3 is executed again to perform reduction treatment;
the data information feature extraction of the data to be processed by adopting a Gaussian mixture model comprises the following steps:
GMM modeling is conducted on the data to be processed, optimal GMM fitting parameters are calculated, and GMM model parameters are extracted to serve as feature variables of original data to conduct feature extraction;
the built GMM model is as follows:
wherein x is the input sample variable; pi i Is a mixed weight and hasd is the dimension of the input variable; p is p i (x；μ i ,Σ i ) An ith gaussian component of GMM; mu (mu) i Is the mean value of the ith Gaussian component; sigma and method for producing the same i Covariance matrix of ith Gaussian component;
and (3) performing reduction processing on the features extracted in the step (2) based on the FCM function, wherein the reduction processing comprises the following steps:
FCM is improved by reducing features in the feature weighted entropy, the objective function is:
wherein,
δ j for controlling feature weights, w j As the feature weight of the jth feature, W is the matrix formed by the features, U is the membership matrix, and V is the clustering center.
2. The energy metering data feature reduction method based on weighted entropy FCM according to claim 1, further comprising a preprocessing step between the step 1 and the step 2, wherein the preprocessing step comprises:
(1) Abnormal data judgment is carried out on the data to be processed by adopting a 3 sigma principle, and the abnormal data is corrected;
(2) Establishing a difference function by using known points, and carrying out interpolation substitution processing on the missing values in the corrected data;
(3) And (3) smoothing technology is used for the corrected and interpolated data set to reduce data noise.
3. According to claim 1The energy metering data characteristic reduction method based on weighted entropy FCM is characterized in that the delta j estimated value is that
4. The weighted entropy FCM-based energy metering data feature reduction method of claim 1, wherein δ j ' replace the delta j Wherein delta' j ＝δ j /max(δ j ) And delta' j ∈(0，1)。
5. The method for feature reduction of energy metering data based on weighted entropy FCM according to claim 1, wherein the constraint condition of feature weights is that
6. The method for reducing characteristics of energy metering data based on weighted entropy FCM according to claim 5, wherein when the number of data points n.ltoreq.d in the data set, the reduction threshold of the characteristic weight is set to be
When n > d, the reduction threshold of the feature weight is set asThe reduction threshold is used to determine whether the feature reduction converges.
7. The reduction device of the weighted entropy FCM-based energy metering data feature reduction method according to any one of claims 1 to 6, comprising:
the data acquisition unit is used for acquiring data to be processed;
the feature extraction unit is used for carrying out feature extraction processing on the data to be processed acquired by the data acquisition unit based on the Gaussian mixture model;
the reduction unit is used for carrying out reduction processing on the features extracted by the feature extraction unit based on the FCM function, and completing feature reduction under the condition that the convergence of data reduction is determined, and exiting from the execution logic;
and under the condition that the reduction of the number is not converged, adopting a method for updating the weight to remove smaller features and returning to the reduction process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911226594.7A CN111027612B (en) | 2019-12-04 | 2019-12-04 | Energy metering data feature reduction method and device based on weighted entropy FCM |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911226594.7A CN111027612B (en) | 2019-12-04 | 2019-12-04 | Energy metering data feature reduction method and device based on weighted entropy FCM |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111027612A CN111027612A (en) | 2020-04-17 |
| CN111027612B true CN111027612B (en) | 2024-01-30 |
Family
ID=70208059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911226594.7A Active CN111027612B (en) | 2019-12-04 | 2019-12-04 | Energy metering data feature reduction method and device based on weighted entropy FCM |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111027612B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117727373B (en) * | 2023-12-01 | 2024-05-31 | 海南大学 | Sample and feature double weighting-based intelligent C-means clustering method for feature reduction |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102129568A (en) * | 2011-04-29 | 2011-07-20 | 南京邮电大学 | Method for detecting image-based spam email by utilizing improved gauss hybrid model classifier |
| CN103921743A (en) * | 2014-05-08 | 2014-07-16 | 长春工业大学 | Automobile running working condition judgment system and judgment method thereof |
| CN106897679A (en) * | 2017-02-13 | 2017-06-27 | 长江水利委员会长江科学院 | A kind of semantic change detecting method and system based on improvement fuzzy C-means clustering |
| CN107784015A (en) * | 2016-08-30 | 2018-03-09 | 中国电力科学研究院 | A kind of Data Reduction method based on the online historical data of power system |
| CN108319984A (en) * | 2018-02-06 | 2018-07-24 | 北京林业大学 | The construction method and prediction technique of xylophyta leaf morphology feature and photosynthesis characteristics prediction model based on DNA methylation level |
| CN108334994A (en) * | 2018-03-20 | 2018-07-27 | 哈尔滨工业大学 | A kind of heating network flow and pressure monitoring point optimization placement method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942755B (en) * | 2013-01-23 | 2017-11-17 | 深圳市腾讯计算机系统有限公司 | Brightness of image adjusting method and device |
| PT3161527T (en) * | 2014-06-30 | 2018-10-19 | Siemens Ag | Solar power forecasting using mixture of probabilistic principal component analyzers |
| WO2018126185A1 (en) * | 2016-12-29 | 2018-07-05 | Agrian, Inc. | Classification technique for multi-band raster data for sorting and processing of colorized data for display |
-
2019
- 2019-12-04 CN CN201911226594.7A patent/CN111027612B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102129568A (en) * | 2011-04-29 | 2011-07-20 | 南京邮电大学 | Method for detecting image-based spam email by utilizing improved gauss hybrid model classifier |
| CN103921743A (en) * | 2014-05-08 | 2014-07-16 | 长春工业大学 | Automobile running working condition judgment system and judgment method thereof |
| CN107784015A (en) * | 2016-08-30 | 2018-03-09 | 中国电力科学研究院 | A kind of Data Reduction method based on the online historical data of power system |
| CN106897679A (en) * | 2017-02-13 | 2017-06-27 | 长江水利委员会长江科学院 | A kind of semantic change detecting method and system based on improvement fuzzy C-means clustering |
| CN108319984A (en) * | 2018-02-06 | 2018-07-24 | 北京林业大学 | The construction method and prediction technique of xylophyta leaf morphology feature and photosynthesis characteristics prediction model based on DNA methylation level |
| CN108334994A (en) * | 2018-03-20 | 2018-07-27 | 哈尔滨工业大学 | A kind of heating network flow and pressure monitoring point optimization placement method |
Non-Patent Citations (2)
| Title |
|---|
| 一种改进的混合高斯学习自适应背景建模算法;邓利平;李明东;邹海洋;;西华师范大学学报(自然科学版)(03);第349-353页 * |
| 李新玉，徐桂云，任世锦，杨茂云.基于可靠性的正则化加权软k-均值的子空间聚类.《南京大学学报(自然科学)》.2017,第525-536页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111027612A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738512B (en) | Short-term power load prediction method based on CNN-IPSO-GRU hybrid model | |
| CN107038292B (en) | Multi-wind-field output correlation modeling method based on self-adaptive multivariate nonparametric kernel density estimation | |
| CN108898154A (en) | A kind of electric load SOM-FCM Hierarchical clustering methods | |
| CN112884236B (en) | Short-term load prediction method and system based on VDM decomposition and LSTM improvement | |
| CN112700326A (en) | Credit default prediction method for optimizing BP neural network based on Grey wolf algorithm | |
| CN111709454A (en) | Multi-wind-field output clustering evaluation method based on optimal copula model | |
| CN112990420A (en) | Pruning method for convolutional neural network model | |
| CN116187835A (en) | Data-driven-based method and system for estimating theoretical line loss interval of transformer area | |
| CN110987436A (en) | Bearing fault diagnosis method based on excitation mechanism | |
| CN113869442A (en) | Industrial scene working condition clustering method, system, equipment and storage medium based on Markov random field model | |
| CN115051929A (en) | Network fault prediction method and device based on self-supervision target perception neural network | |
| CN111027612B (en) | Energy metering data feature reduction method and device based on weighted entropy FCM | |
| CN116226689A (en) | Power distribution network typical operation scene generation method based on Gaussian mixture model | |
| CN117420803A (en) | Multi-process technological parameter fusion optimization method based on improved particle swarm optimization algorithm | |
| CN117709712A (en) | Situation prediction method and terminal for power distribution network based on hybrid neural network | |
| CN113408622A (en) | Non-invasive load identification method and system considering characteristic quantity information expression difference | |
| CN113763710B (en) | Short-term traffic flow prediction method based on nonlinear adaptive system | |
| CN116992954A (en) | UMAP data dimension reduction-based similarity measurement transfer learning method | |
| CN109636014B (en) | Cotton blending method based on finished yarn quality prediction | |
| CN116865255A (en) | Short-term wind power prediction method based on improved entropy weight method and SECEEMD | |
| CN111476402A (en) | Wind power generation capacity prediction method coupling meteorological information and EMD technology | |
| CN116882272A (en) | Statistical analysis method for machining errors of compressor blades | |
| CN113536685B (en) | Neural network extrapolation-based wind speed probability model modeling method | |
| CN115017988A (en) | Competitive clustering method for state anomaly diagnosis | |
| CN102236898A (en) | Image segmentation method based on t mixed model with infinite component number |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |