CN114168268A - Container technology-based intelligent distribution data acquisition and fusion method and system - Google Patents

Container technology-based intelligent distribution data acquisition and fusion method and system Download PDF

Info

Publication number
CN114168268A
CN114168268A CN202111535291.0A CN202111535291A CN114168268A CN 114168268 A CN114168268 A CN 114168268A CN 202111535291 A CN202111535291 A CN 202111535291A CN 114168268 A CN114168268 A CN 114168268A
Authority
CN
China
Prior art keywords
data
container
data acquisition
distribution
different
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111535291.0A
Other languages
Chinese (zh)
Inventor
孙文川
崔冬晓
黄凯
申佃涛
王毅
吴哲
张囝
程梓航
于洋
叶亮
邱晨
杨雷鹏
傅乐
李艾民
董革放
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zibo Power Supply Co of State Grid Shandong Electric Power Co Ltd
Original Assignee
Zibo Power Supply Co of State Grid Shandong Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zibo Power Supply Co of State Grid Shandong Electric Power Co Ltd filed Critical Zibo Power Supply Co of State Grid Shandong Electric Power Co Ltd
Priority to CN202111535291.0A priority Critical patent/CN114168268A/en
Publication of CN114168268A publication Critical patent/CN114168268A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45575—Starting, stopping, suspending or resuming virtual machine instances
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45587—Isolation or security of virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Storage Device Security (AREA)

Abstract

The disclosure belongs to the field of electric power systems and automation thereof, and provides a container technology-based intelligent station region operation and distribution data acquisition and fusion method and system, wherein the method comprises the following steps: acquiring the operation and distribution data of the transformer area; preprocessing the data by adopting a Spark-based data parallel preprocessing method; completing data fusion of the preprocessed data through a unified information model; the integrated data are respectively placed in different containers which are subjected to safety isolation according to task identifiers, comprehensive and high-efficiency data acquisition of the intelligent distribution area considering multi-side perception requirements is achieved by breaking through data barriers between marketing and power distribution service systems, the complaint rate of users is reduced, potential safety hazards of a power distribution network are found in time, power failure management is enhanced, and the service level and the power supply safety and reliability of the users are improved.

Description

Container technology-based intelligent distribution data acquisition and fusion method and system
Technical Field
The invention belongs to the field of electric power systems and automation thereof, and particularly relates to a container technology-based intelligent transformer district operation and distribution data acquisition and fusion method and system.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Along with the development of smart power distribution network construction, a smart distribution area is an integral part of a smart power distribution network, and is developing towards the overall direction and the overall target of reliable and efficient power distribution network grid structure, high-reliability and high-safety communication network, high-permeability distributed power access, and rapid simulation and self-healing control of a power distribution system.
Although the smart platform area has a good development prospect, the smart platform area also has problems. Distribution network marketing and distribution service system are relatively independent, lack data interaction mechanism and system interface between the system, information isolated island phenomenon is serious, marketing, relevance and complementarity between the distribution data do not obtain make full use of, be unfavorable for integrating distribution network data resource and realizing the quick analysis of distribution network multisource data, and along with wisdom platform district electricity, gas, cold, heat supply network and distributed photovoltaic, the energy storage, electric automobile fills the occupation ratio increase of all kinds of networking confession energy supply equipment such as electric pile and intelligent household electrical appliances, current marketing distribution informationization integration system has failed to realize the terminal electricity distribution holographic perception of platform district adapted and state identification demand, and the not comprehensive scheduling problem of data access type has appeared.
Disclosure of Invention
In order to solve the problems, the comprehensive and high-efficiency data acquisition of the intelligent transformer area considering the multi-side perception requirement is realized by breaking through the data barriers between marketing and power distribution service systems, the complaint rate of users is reduced, the potential safety hazard of a power distribution network is discovered in time, the power failure management is enhanced, and the service level and the power supply safety and reliability of the users are improved.
In order to achieve the purpose, the following technical scheme is adopted in the disclosure:
in a first aspect, the intelligent platform region operation and distribution data acquisition and fusion method based on the container technology comprises the following steps:
acquiring operation and distribution data of a transformer area;
preprocessing the marketing and distribution data by adopting a Spark-based data parallel preprocessing method;
completing data fusion of the preprocessed data through a unified information model;
and respectively placing the fused data into different containers which are safely isolated according to the task identifiers.
As a further limitation, the process of preprocessing the data includes:
reducing the dimension of the data and eliminating the blank/filling in the data based on the map function and the reduce;
denoising data based on an amplitude limiting jitter elimination filtering algorithm;
and (4) screening and removing the data outlier records based on the K-means clustering.
By way of further limitation, the construction process of the unified information model includes the following steps:
taking the preprocessed data as an input sample;
introducing transformed nonlinear mapping and kernel function, and classifying the sample formula by mapping to a high-dimensional space;
solving a sample classification formula according to the KKT optimal condition and a least square method to obtain a weight vector and a bias vector in an input sample set space, and obtaining a decision quantity according to the weight vector and the bias vector;
and carrying out normalization processing on the decision quantity to obtain a consistency coefficient.
As a further limitation, the security isolation method between the containers is to determine the partition manner of the object address space according to a security isolation policy rule set, and different subjects access different address spaces according to different permissions.
By way of further limitation, the set of security quarantine policy rules includes rule 1 and rule 2; rule 1 represents the address space in which any container body can read operations, and rule 2 represents the address space in which any container body can write.
As a further limitation, the method for determining the partition of the object address space according to the security isolation policy rule set is as follows: and newly building a container, distributing a container main body and an object address space for the container, carrying out any operation on the object in the address space range in the container by the container main body, initiating system call to a host main body, applying for access of the corresponding address space, and accessing different address spaces by different main bodies according to different authorities.
By way of further limitation, the securely isolated containers are classified into different security levels, each different security level containing a different container and each container having a piece of information, and the containers are classified into corresponding security levels by container multi-level access control.
In a second aspect, a container technology-based intelligent platform region marketing and distribution data acquisition and fusion system is provided, which includes:
a data acquisition module configured to: acquiring operation and distribution data of the transformer area through a uniform data interface;
a pre-processing module configured to: preprocessing the marketing and distribution data by adopting a Spark-based data parallel preprocessing method;
a data fusion module configured to: completing data fusion of the preprocessed data through a unified information model;
a data storage module configured to: and respectively placing the fused data into different containers which are safely isolated according to the task identifiers.
In a third aspect, a computer-readable storage medium is provided, in which a plurality of instructions are stored, the instructions being adapted to be loaded by a processor of a terminal device and to execute the steps of the method for collecting and fusing data of a smart platform region based on container technology.
In a fourth aspect, a terminal device is provided that includes a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the intelligent platform region operation data acquisition and fusion method based on the container technology.
Compared with the prior art, the beneficial effect of this disclosure is:
(1) the marketing and distribution data acquisition and fusion method provided by the invention can eliminate possible conflicts when each sensor acquires data, different data can be uniformly described by constructing a uniform information model, the problem of data model inconsistency and the model conversion workload in data exchange can be effectively avoided, and because of uniform modeling and uniform storage, redundancy does not exist in the data, and further, the data exchange, comparison and other work in two systems are not required, and the development and maintenance workload is greatly reduced.
(2) The invention can effectively isolate the containers without direct or indirect information exchange through the division of the container address space, and any container main body can not directly read the address space and the content written by other container main bodies, thereby ensuring the safety of data.
(3) The invention can effectively limit the communication of the unauthorized container through the multilevel access control of the container, enhances the safety isolation between the containers, prevents the operating environment of the container and the container from being tampered and attacked, and ensures the operating legality in the container.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and are not to limit the disclosure.
FIG. 1 is a flow chart of a method for collecting and fusing operation and configuration data of an intelligent platform based on container technology according to the present embodiment;
fig. 2 is a division relationship of address spaces corresponding to objects in the convergence terminal of this embodiment;
fig. 3 is a container security level organization chart of the present embodiment.
The specific implementation mode is as follows:
the present disclosure is further described with reference to the following drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Interpretation of terms:
marketing and distribution: the 'operation' means marketing and management, and the 'distribution' means distribution of electricity.
Example one
Fig. 1 is a flowchart illustrating a method for collecting and fusing operation and configuration data of an intelligent station based on container technology. As shown in fig. 1, the method for collecting and fusing operation and configuration data of an intelligent station based on container technology of the present embodiment includes the following steps:
s1: acquiring operation and distribution data of the transformer area through a uniform data interface;
s2: preprocessing the data by adopting a Spark-based data parallel preprocessing method;
the invention solves the problems of repeated redundancy and non-uniform data of data acquired from marketing and power distribution systems by adopting a Spark-based data parallel preprocessing method, and mainly comprises the following steps:
s201: reducing the dimension of the data and eliminating the blank/filling in the data based on the map function and the reduce;
the method specifically comprises the following steps:
(1) importing original data into Spark to form an elastic distributed data set for parallel computation;
(2) removing invalid fields for data analysis through feature selection;
(3) and taking the transformer ID of the transformer area and the recorded date as the duplicate checking data of the map function, and marking repeated data records. Merging the data according to two key fields by adopting database operation provided by Spark SQL, finally returning the position where the number of the merged data records is more than 1, and deleting the corresponding row in the original array by using a reduce function to obtain a new data set with the repeated data removed.
(4) And calculating the deletion proportion of the attributes in the single record by using a map function, removing the blank of the record with the deletion proportion more than 30%, and supplementing the deletion of the record with the deletion proportion less than 30% by using a Lagrange interpolation method.
S202: denoising data based on an amplitude limiting jitter elimination filtering algorithm;
the method specifically comprises the following steps:
determining the maximum deviation value allowed by twice sampling of load data, comparing the difference between two data of continuous time slices, and judging whether the data error is in the range of the maximum deviation value, if not, replacing the data of the following time slice with a reasonable value, wherein the reasonable value can be the copy of the value of the previous time point, and can also be a data record statistical result;
the specific method combining Spark is that firstly, the map function is used for carrying out one-by-one inspection on the data records, and then the reduce function is used for carrying out replacement processing on the screened data records larger than the deviation value according to the latest sampling result.
And determining the maximum deviation value allowed by twice sampling of the load data according to expert experience.
S203, screening and removing the data outlier records based on the K-means clustering;
the method specifically comprises the following steps:
(1) converting the data into a structured DataFrame to facilitate API computation;
(2) importing a Spark MLlib algorithm library, and realizing load clustering by calling a KMeans function; the hyper-parameters of the clustering algorithm are set through expert experience;
(3) and deleting the undivided data records by using a drop function according to the clustering result.
Because the Spark ecosystem provides a machine learning algorithm library, the data can be directly analyzed through the API calling function provided by Spark without realizing a complex load clustering algorithm through a large amount of codes.
S3: completing data fusion of the preprocessed data through a unified information model; the construction process of the unified information model is as follows:
let the processed data set be X ═ X1,x2,x3,…,xnA certain data xkCan be expressed as:
T＝{(x1,y1),…,(xk,yk)}∈(Rn×y) (1)
wherein xi∈RnTo input samples, yiE y R is output sample, i 1,2, …, n is to find out real decision giThe decision quantity is a regression equation for inferring y by x:
y＝gi＝(w·x)+b (2)
in the formula, w is RnA weight vector in space; b is a bias vector;
after introducing the nonlinear mapping of the transformation x ═ Φ (x) and the kernel function K (x, x '), (Φ (x) · Φ (x')), the classification is performed by mapping to a high-dimensional space, and the classification formula can be expressed as:
Figure BDA0003412992380000071
s.t.yi(w·Φ(xi))+b+ηi,i＝1,2,…,l (4)
in the formula, eta is [. eta. ]1,η2,…,ηl]Is an error variable; and C is a penalty coefficient. From the lagrange function:
Figure BDA0003412992380000081
wherein α ═ α1,α2,…,αi,…,αl]TRepresenting a Lagrange multiplier, derived from the above equation according to KKT optimal conditions:
Figure BDA0003412992380000082
and (5) solving a and b according to a least square method. Is provided with
Figure BDA0003412992380000083
Is any solution of the above problem, then (w) is the solution for (w, b)*,b*) Can be calculated according to the following formula:
Figure BDA0003412992380000084
Figure BDA0003412992380000085
Figure BDA0003412992380000086
carrying out normalization processing on the decision quantity:
Figure BDA0003412992380000087
Figure BDA0003412992380000088
in the formula, qiIs a consistency factor.
The method has the advantages that conflicts which may exist when the sensors collect data can be eliminated, different data are described in a unified mode through constructing a unified information model, the problem of data model inconsistency and the model conversion workload in data exchange can be effectively avoided, due to the fact that unified modeling and unified storage are adopted, redundancy does not exist in the data, data exchange, comparison and other work in two sets of systems are not needed, and development and maintenance workloads are greatly reduced.
S4: respectively placing the fused data in different containers which are safely isolated according to task identifiers;
each container has an independent task identifier mechanism, and when the containers are mapped to the fusion terminal kernel, the containers and the task identifiers are combined into a complete task identifier. The marketing and distribution data fusion terminal isolates the service software from the operating system through a container technology, and the distribution service APP and the utilization service APP are isolated while the safety of the system is improved.
Further, the safety isolation method between the containers comprises the following steps: and constructing a security isolation policy rule set, determining a division mode of the object address space according to the security isolation policy rule set, and accessing different address spaces by different subjects according to different authorities.
To pair
Figure BDA0003412992380000091
j, 1. ltoreq. i.ltoreq.n, 1. ltoreq. j.ltoreq.n, i.noteq.j, with R (Co)i)∩W(Coi)＝φ。
Wherein R is S → 2LAn object address space indicating that the subject S can perform a read operation; w: S → 2LAn object address space indicating that the host S can perform a write operation; coi(1. ltoreq. i. ltoreq. n) represents the container i body.
Determining a partition mode of the object address space according to a security isolation policy rule set;
the set of rules includes:
rule 1:
Figure BDA0003412992380000092
rule 2:
Figure BDA0003412992380000101
in the formula, H represents the address space corresponding to the object that the host can operate arbitrarily, and let HxRepresenting a first type of address space in H, HroIndicating the second type of address space in H, Hic(1 ≦ i ≦ n) for the third type of address space in H.
Wherein, rule 1 indicates that the address space of any container i (i is more than or equal to 1 and less than or equal to n) subject capable of performing reading operation includes the container address space C corresponding to all objects in the container ii(i is more than or equal to 1 and less than or equal to n) and address space H corresponding to the object of which all subjects can only carry out reading operationroAnd an object address space H corresponding to the container i and the subject in the H and capable of reading and writingic(1≤i≤n)。
Rule 2 indicates that the writable address space of any container i (1 ≦ i ≦ n) subject includes the container address space C corresponding to all objects in the container ii(i is more than or equal to 1 and less than or equal to n) and an object address space H corresponding to the container i in the H and allowing the host to read and writeic(1≤i≤n)。
The application provides an address space division method meeting the limiting conditions in the rule 1 and the rule 2, namely different subjects access different address spaces according to different authorities, and all containers in a fusion terminal are safely isolated.
Namely to
Figure BDA0003412992380000102
j, 1. ltoreq. i.ltoreq.n, 1. ltoreq. j.ltoreq.n, i.noteq.j, with R (Co)i)∩W(Coi)＝(Ci∪Hro∪Hic)＝(Cj∪Hjc)＝φ。
The partition method for determining the object address space according to the security isolation policy rule set is shown in fig. 2, a container is newly created, a container subject and an object address space in a certain range are allocated to the container, the container subject performs any operation on the object in the address space range in the container, system call can be initiated to the subject according to needs, access of the corresponding address space is applied, and different subjects access different address spaces according to different permissions. Q represents the address space corresponding to all objects, the address space corresponding to all objects can be regarded as one (n +1) tuple according to the security isolation policy rule set, n represents the number of containers in the subject, i.e. Q ═ { H, C ═1、C2、…、CnAn object address space where a host can operate arbitrarily can be regarded as an (n +2) tuple, i.e., H ═ Hx、Hro、H1c、H2c、…、Hnc}。
The scheme has the advantages that direct or indirect information exchange does not exist between the containers through the division of the container address space, and any container main body cannot directly read the address space and the content written by other container main bodies, so that the safety of data is ensured.
S5: the containers are classified into different safety classes, denoted L.
To avoid unauthorized communication between containers, the present invention classifies containers into different security levels, where L ═ L1,L2,L3,L4,L5,L6,L7The relationship between the levels is shown in fig. 3:
in container multi-level access control, each different security level LiComprising different containers LiCiAnd mixing LiC is denoted by LiThe set of containers contained in the security level, then
LiC＝{LiC1,LiC2,…,LiCn} (12)
And each container has a message LiCimiBy LiCiM is recorded as a set of container information, then
LiCiM＝{LiC1m1,LiC2m2,…,LiCnmn} (13)
For each container, the administrator can classify the container into corresponding security levels according to the needs of the administrator.
Main body LiThe ciphertext c may be obtained by encrypting the information m by the following relation:
c＝memodNi (14)
if L isjIs LiAncestor of (1), subject LjThe ciphertext c may be parsed by the following relationship to obtain the information m:
Figure BDA0003412992380000111
to ensure data security, any piece of information m is encrypted by using K ═ e, N >, denoted by [ m, K ], and defined as:
[m,<e,N>]＝memod N (16)
the key K being equal to<e,N>Decrypting key K-1By ordered pairs<d,N>And satisfies ed 1mod (N).
For decryption key K-1＝<d,N>Knowing the ciphertext c, the plaintext m can be obtained by the following relation,with [ c, K ]-1]To show that:
m＝[c,K-1]＝cdmod N (17)
for any information m, when K ═<e,N>And K is-1＝<d,N>When there is
[[m,K],K-1]＝m (18)
In the multi-level relation of the container, the information m can be encrypted by using the encryption key K provided by the hierarchy, and the provided decryption key K can also be used-1And encrypting the encrypted information to restore the information m.
Each container has an independent network environment. Tasks in the container are communicated with the outside through a network, the container is provided with a port, an IP address, a socket and the like, and the marketing and power distribution double-master-station butt joint can be realized through optical fibers or remote communication modes such as 4G.
Example two
In this embodiment, a container technology-based intelligent platform region operation and distribution data acquisition and fusion system is disclosed, comprising:
a data acquisition module configured to: acquiring operation and distribution data of the transformer area through a uniform data interface;
a pre-processing module configured to: preprocessing the data by adopting a Spark-based data parallel preprocessing method;
a data fusion module configured to: completing data fusion of the preprocessed data through a unified information model;
a data storage module configured to: and respectively placing the fused data into different containers which are safely isolated according to the task identifiers.
EXAMPLE III
In this embodiment, a computer readable storage medium is disclosed, in which a plurality of instructions are stored, the instructions being adapted to be loaded by a processor of a terminal device and to execute the steps of the container technology-based smart station configuration data acquisition and fusion method disclosed in embodiment one.
Example four
In this embodiment, a terminal device is disclosed that includes a processor and a computer-readable storage medium, the processor configured to implement instructions; the computer readable storage medium is used for storing a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the intelligent platform region operation data acquisition and fusion method based on the container technology.
The invention can effectively limit the communication of the unauthorized container through the multilevel access control of the container, enhances the safety isolation between the containers, prevents the operating environment of the container and the container from being tampered and attacked, and ensures the operating legality in the container.
As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Although the present disclosure has been described with reference to specific embodiments, it should be understood that the scope of the present disclosure is not limited thereto, and those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present disclosure.

Claims (10)

1. An intelligent distribution data acquisition and fusion method based on container technology is characterized in that: the method comprises the following steps:
acquiring the operation and distribution data of the transformer area;
preprocessing the distribution data of the distribution room by adopting a Spark-based data parallel preprocessing method;
completing data fusion of the preprocessed data through a unified information model;
and respectively placing the fused data into different containers which are safely isolated according to the task identifiers.
2. The container technology-based intelligent platform region operation and configuration data acquisition and fusion method as claimed in claim 1, wherein: the process of preprocessing the data comprises the following steps:
reducing the dimension of the data and eliminating the blank/filling in the data based on the map function and the reduce;
denoising data based on an amplitude limiting jitter elimination filtering algorithm;
and (4) screening and removing the data outlier records based on the K-means clustering.
3. The container technology-based intelligent platform region operation and configuration data acquisition and fusion method as claimed in claim 1, wherein: the construction process of the unified information model comprises the following steps:
taking the preprocessed data as an input sample;
introducing transformed nonlinear mapping and kernel function, and classifying the sample formula by mapping to a high-dimensional space;
solving a sample classification formula according to the KKT optimal condition and a least square method to obtain a weight vector and a bias vector in an input sample set space, and obtaining a decision quantity according to the weight vector and the bias vector;
and carrying out normalization processing on the decision quantity to obtain a consistency coefficient.
4. The container technology-based intelligent platform region operation and configuration data acquisition and fusion method as claimed in claim 1, wherein: the safety isolation method among the containers is to determine the partition mode of the object address space according to the safety isolation strategy rule set, and different subjects access different address spaces according to different authorities.
5. The container technology-based intelligent platform region operation and configuration data acquisition and fusion method as claimed in claim 1, wherein: the security isolation policy rule set comprises rule 1 and rule 2; rule 1 represents the address space in which any container body can read operations, and rule 2 represents the address space in which any container body can write.
6. The intelligent platform region operation and distribution data acquisition and fusion method based on container technology as claimed in claim 4, wherein: the method for determining the partition mode of the object address space according to the security isolation policy rule set is as follows: and newly building a container, distributing a container main body and an object address space for the container, carrying out any operation on the object in the address space range in the container by the container main body, initiating system call to a host main body, applying for access of the corresponding address space, and accessing different address spaces by different main bodies according to different authorities.
7. The container technology-based intelligent platform region operation and configuration data acquisition and fusion method as claimed in claim 1, wherein: the safety isolated containers are divided into different safety levels, each different safety level comprises different containers, each container has one information, and the containers are divided into corresponding safety levels through container multilevel access control.
8. Intelligence platform district marketing and distribution data acquisition fuses system based on container technique, characterized by includes:
a data acquisition module configured to: acquiring the operation and distribution data of the transformer area;
a pre-processing module configured to: preprocessing the distribution data of the distribution room by adopting a Spark-based data parallel preprocessing method;
a data fusion module configured to: completing data fusion of the preprocessed data through a unified information model;
a data storage module configured to: and respectively placing the fused data into different containers which are safely isolated according to the task identifiers.
9. A computer-readable storage medium characterized by: in which a plurality of instructions are stored, said instructions being adapted to be loaded by a processor of a terminal device and to carry out the steps of the intelligent platform configuration data acquisition and fusion method based on container technology according to any one of claims 1 to 7.
10. A terminal device is characterized in that: the system comprises a processor and a computer readable storage medium, wherein the processor is used for realizing instructions; a computer readable storage medium for storing a plurality of instructions adapted to be loaded by a processor and to perform the steps of the intelligent station operation data acquisition and fusion method based on container technology according to any one of claims 1 to 7.
CN202111535291.0A 2021-12-15 2021-12-15 Container technology-based intelligent distribution data acquisition and fusion method and system Pending CN114168268A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111535291.0A CN114168268A (en) 2021-12-15 2021-12-15 Container technology-based intelligent distribution data acquisition and fusion method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111535291.0A CN114168268A (en) 2021-12-15 2021-12-15 Container technology-based intelligent distribution data acquisition and fusion method and system

Publications (1)

Publication Number Publication Date
CN114168268A true CN114168268A (en) 2022-03-11

Family

ID=80486703

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111535291.0A Pending CN114168268A (en) 2021-12-15 2021-12-15 Container technology-based intelligent distribution data acquisition and fusion method and system

Country Status (1)

Country Link
CN (1) CN114168268A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827209A (en) * 2022-05-07 2022-07-29 南京四维智联科技有限公司 Data acquisition method and device, electronic equipment and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827209A (en) * 2022-05-07 2022-07-29 南京四维智联科技有限公司 Data acquisition method and device, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
Bhattarai et al. Big data analytics in smart grids: state‐of‐the‐art, challenges, opportunities, and future directions
Ghorbanian et al. Big data issues in smart grids: A survey
Sahebi et al. Modeling the enablers for blockchain technology adoption in renewable energy supply chain
Hao et al. Robust vehicle pre‐allocation with uncertain covariates
McKee et al. Survey of advances and challenges in intelligent autonomy for distributed cyber‐physical systems
Green et al. Applications and trends of high performance computing for electric power systems: Focusing on smart grid
CN108446396B (en) Power data processing method based on improved CIM model
Li et al. Research on QoS service composition based on coevolutionary genetic algorithm
CN114626807A (en) Nuclear power scene management method, system, device, computer equipment and storage medium
Liu et al. A contingency-aware method for N-2 security-constrained transmission expansion planning
CN112364008A (en) Equipment portrait construction method for intelligent terminal of power internet of things
CN114168268A (en) Container technology-based intelligent distribution data acquisition and fusion method and system
Priyanka et al. Fundamentals of wireless sensor networks using machine learning approaches: Advancement in big data analysis using Hadoop for oil pipeline system with scheduling algorithm
CN117171711B (en) Cloud platform-based enterprise internal and external data fusion sharing method and system
CN103514412B (en) Build the method and Cloud Server of access control based roles system
CN104346575A (en) Software defined security architecture
CN115237592B (en) Privacy-aware hybrid cloud service flow scheduling method
CN116680090A (en) Edge computing network management method and platform based on big data
Zhu A hybrid indicator many‐objective optimization algorithm for the selection and delivery of disaster relief materials problem
CN101477647A (en) Service process exhibition control system
CN114329127A (en) Characteristic box dividing method, device and storage medium
Bernabé-Moreno When digitalization becomes an essential part of our energy transition
Moodley et al. Using self‐organising maps to predict and contain natural disasters and pandemics
Gong et al. Qualitative Analysis of Commercial Services in MEC as Phased‐Mission Systems
Madasi et al. N‐Cubic q‐Rung Orthopair Fuzzy Sets: Analysis of the Use of Mobile App in the Education Sector

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination