CN113127515A - Power grid-oriented regulation and control data caching method and device, computer equipment and storage medium - Google Patents
Power grid-oriented regulation and control data caching method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113127515A CN113127515A CN202110391693.1A CN202110391693A CN113127515A CN 113127515 A CN113127515 A CN 113127515A CN 202110391693 A CN202110391693 A CN 202110391693A CN 113127515 A CN113127515 A CN 113127515A
- Authority
- CN
- China
- Prior art keywords
- regulation
- data
- control data
- cache
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000000605 extraction Methods 0.000 claims abstract description 32
- 230000015654 memory Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 17
- 230000001105 regulatory effect Effects 0.000 claims description 17
- 238000013500 data storage Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000007787 long-term memory Effects 0.000 description 4
- 230000006403 short-term memory Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Economics (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a power grid-oriented regulation and control data caching method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring regulation and control data to be stored, extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space; and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency. By adopting the method, the speed and the accuracy of accessing and inquiring the regulation and control data in the power grid mass data can be improved.
Description
Technical Field
The present application relates to the field of power grid data caching, and in particular, to a power grid-oriented data caching method and apparatus, a computer device, and a storage medium.
Background
In order to support the construction of a new generation of dispatching control system, the challenges brought by the rapid increase of information scale, application functions and system users under the architecture of physical distribution and logic unification are met, the requirement of high-speed access of power grid regulation and control mass data is met, and the high-efficiency access technology research of power grid regulation and control multi-element data based on high-speed cache is required to be developed. The overall requirements are directed by the efficient access requirement of the power grid regulation and control multi-element data, and the power grid regulation and control multi-element data storage architecture and the index technology are adapted to a new generation regulation and control system architecture by combining a cache technology and using the successful experience of the IT industry as a reference.
The cache network model is used for analyzing the performance of network cache by modeling a scene network. The theory of cache correlation and performance analysis of its cache network is essential to understanding network caching. Theoretical modeling of a cache network often requires consideration of the topology of the cache network, the model of the distribution of arrival of packets, and the popularity of data requested by a user. For the current cache performance optimization method, the method mainly includes three aspects of a cache decision strategy, a cache replacement strategy and cache capacity. Cache decision policies, which are used to determine which data objects are placed on which cache nodes, are an active and possibly the hottest area of research in caches. There are three main categories of cache placement: content popularity, cache partitioning, selfish. Content popularity is the caching of only popular content on the network to reduce inter-domain traffic. The content popularity is calculated by the total number of requests for content that remains stable for a short time. When the content storage stores information larger than the maximum cache space of the content storage, a cache replacement policy is needed to decide which contents are stored in the content storage and which contents are discarded. As is well known, the most common cache replacement policy is the least recently used method, discarding the least recently accessed content, which performs well and increases the probability of a cache hit by storing the most recent data for some more time. Another important cache replacement policy is the least common algorithm, where the least common content is discarded first. The caching decision may be a replacement decision based on the content arriving at the router and its popularity. In order to obtain better network performance, we should not delete content directly from the cache, and can move it one level up in the cache hierarchy for caching. We further classify cache replacement as content popularity based and content priority based, where less popular and lower priority content is replaced first. How to evaluate the performance of the cache scheme is important, the following evaluation indexes are very important.
(1) The hit rate of the server: refers to the number of content requests that the server satisfies, thereby indicating the load of the server. In order to determine the server load savings due to caching, the server hit metric is defined as a fraction, called the server hit rate, normalized by the number of content-capable requests represented in the network, and the priority caching strategy should conclude in reducing this metric criterion that all experiments in this context use the same day's request sequence, the total number of requests is the same, and therefore not in the form of a fraction, and directly count the number of times data is obtained from the source node as an evaluation index.
(2) Cache hit: meaning that the number of content requests satisfied by the cache in the node is not the same as the source node. To determine the server load savings due to caching, the cache hit metric is defined as a fraction, called the cache hit rate, normalized by the number of content requests represented in the network. The hit rate and cache rate of the server are disputed indexes, and the preferred cache strategy should correspond to the increase of the measurement standard.
(3) Local cache eviction amount: refers to the number of local cache replacements that occur on a node using a current cache replacement policy. An increase in this metric means that the utilization of the cache contents is low.
(4) Cache efficiency: is defined as the number of content requests that the cache satisfies and is calculated as the score of cache hits on a node and the number of content stored in the cache of the same node.
(5) Cache frequency: is defined as the amount of content cached on a node, and the amount of content traversed by the same node.
(6) Cache redundancy: the cache redundancy metric refers to the amount of identical content stored in the network.
However, the types of the power grid regulation data are various and complex, the requirement on access speed is high, and it is very difficult to accurately and timely find the required data in massive, complex, various and heterogeneous data.
Disclosure of Invention
In view of the foregoing, there is a need to provide a power grid-oriented regulation and control data caching method, device, computer device and storage medium capable of quickly and accurately finding target data.
A regulation and control data caching method facing a power grid comprises the following steps:
acquiring regulation and control data to be stored;
extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency.
Optionally, the first cache space is configured to store hash values corresponding to the stored regulatory data, and use the hash values as an index directory;
and the second cache space is used for storing the regulation and control data with the reading frequency within the preset ranking.
Optionally, the regulation data caching method further includes querying the stored regulation data, and the querying method includes:
acquiring regulation and control data to be inquired;
calculating the reading frequency of the regulation and control data by using the frequency prediction network, and if the reading frequency is within the preset ranking, inquiring the corresponding regulation and control data in the second cache;
if the reading frequency is not within the preset ranking, extracting semantic features of the regulation and control data by using the feature extraction network, and calculating corresponding hash values according to the semantic features;
and after finding the corresponding hash value in the index directory in the first cache space according to the hash value, acquiring corresponding regulation and control data from the database or in real time according to the hash value, and reading the regulation and control data into a third cache space.
Optionally, when the third cache space already stores the same regulation and control data as the newly read data, the newly read regulation and control data will replace the existing regulation and control data.
Optionally, the type of the regulation data includes text, image, and time series data;
the feature extraction network projects the different types of regulation and control data to the same semantic space, so that the extracted semantic features have uniformity.
Optionally, when the acquired to-be-stored regulation and control data is a plurality of data of different types, after extracting semantic features corresponding to the regulation and control data by using the feature extraction network, fusing the semantic features and calculating corresponding hash values.
Optionally, the frequency prediction network includes a plurality of sub-network models and a converged network model, and the sub-network models include:
the time sequence long-short term memory network model is used for identifying the regulation and control data of the time sequence type;
the image processing and identifying network model is used for identifying the regulation and control data of the image type; and
and the long-short term network model is used for identifying the regulation data of the text type.
The present application further provides a regulation and control data cache device for a power grid, including:
the data acquisition module is used for acquiring the regulation and control data to be stored;
the semantic feature extraction module is used for extracting semantic features of the regulation and control data by using the trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and the data storage module is used for calculating the reading frequency of the regulation and control data by utilizing the trained frequency prediction network and storing the regulation and control data into a second cache space or a database according to the reading frequency.
The present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:
acquiring regulation and control data to be stored;
extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency.
The present application further provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of:
acquiring regulation and control data to be stored;
extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency.
According to the power grid-oriented regulation and control data cache method, the device, the computer equipment and the storage medium, the semantic features of the regulation and control data to be stored are extracted, the hash values of the semantic features are calculated and stored in the first cache space, the regulation and control data are placed in the corresponding second cache space or the corresponding database after the reading frequency is predicted through the deep learning model, so that when the subsequent query is carried out, the hash values can be searched in the first cache space to determine whether the data to be queried is stored, and if more data can be queried in the second cache space, the target data can be accurately and quickly extracted from mass data.
Drawings
FIG. 1 is a flow diagram illustrating a method for regulating data caching in accordance with one embodiment;
FIG. 2 is a schematic flow chart diagram illustrating a method for high-speed query of regulatory data in one embodiment;
FIG. 3 is a block diagram of an embodiment of a cross-modal binary learning framework based on fused similarity hashing;
FIG. 4 is a block diagram of an apparatus for regulating a data caching method in accordance with an embodiment;
FIG. 5 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
By analyzing the request for the control data, that is, the access and query history data and the correlation of the requested data, it is found that there is a great difference in the requested data, that is, some types of data or some locations of data are frequently called, and other data are frequently called rarely, that is, the calling frequency is very different. The relevant conditions of the regulation and control data comprise the power grid position where the requested data is located, the requested data type, the time interval of the requested data, where the data comes from, the power grid service type expressed by the data and the like.
The access characteristics of such data can make some caching strategies perform poorly, such as first-in-first-out (FIFO) based on historical data access order and Least Recently Used (LRU) based on locality time. Since many power grid regulation data are acquired in real time, statistics on historical various power grid regulation data shows that many real-time acquired data have relatively constant values, that is, they do not change all the time. The characteristic can provide convenience for a cache strategy, and the cache node can be sent to a database or power grid regulation hardware to collect data in real time at a certain time interval, so that the cache strategy can greatly save resource consumption caused by reading the data and reduce the pressure of storage bandwidth.
Based on the characteristics of the power grid, as shown in fig. 1, a power grid-oriented regulation and control data caching method is provided, which includes the following steps:
step S101, acquiring regulation and control data to be stored;
step S102, extracting semantic features of the regulation and control data by using the trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
step S103, calculating the reading frequency of the regulation data by using the trained frequency prediction network, and storing the regulation data into a second cache space or a database according to the reading frequency.
In this embodiment, when storing the regulation and control data, the data needs to be simultaneously input into two trained deep neural networks, and corresponding semantic features and predicted reading frequencies are respectively extracted. After the semantic features are extracted, the hash value is calculated according to the semantic features and then stored in the first cache space as an index, so that the corresponding hash value can be searched in the first cache space to determine whether the data exists and the storage position. More or less of the reading frequency determines the position of regulating data storage, so that the data can be accessed quickly in the following process.
In one embodiment, the first cache space is used for storing hash values corresponding to the regulatory data, and each hash value is used as an index directory, so as to facilitate statistics and query on the stored data.
In one embodiment, the second buffer space is used for storing the regulation and control data with the reading frequency within the preset ranking. That is, the second cache space is used to store data with the highest access frequency. For example, when the preset rank is 100, the regulation data with the reading frequency within the rank of 100 is stored in the second cache space. And the data ranked beyond 100 is stored in a database, or sensor.
Specifically, the data stored in the second cache space is determined according to historical data, and needs to be dynamically updated according to the running of time.
A timing dynamic updating method can be adopted, and the time interval can be set. For example, the frequency prediction network is set to update once a week or a few days, the network parameters are updated by the frequency prediction network according to the data reading condition from a recent period of time, the updated network recalculates the data reading frequency, then gives a ranking, and updates the regulation and control data in the second cache space according to the ranking.
As shown in fig. 2, the regulation data caching method further includes querying the stored regulation data, and the querying method includes the following steps:
step S201, acquiring regulation and control data to be inquired;
step S202, calculating the reading frequency of the regulation and control data by using a frequency prediction network, and if the reading frequency is within a preset rank, inquiring the corresponding regulation and control data in a second cache;
step S203, if the reading frequency is not within the preset rank, extracting semantic features of the regulation and control data by using a feature extraction network, and calculating corresponding hash values according to the semantic features;
step S204, after finding the corresponding hash value in the index directory in the first cache space according to the hash value, acquiring corresponding regulation and control data from the database or in real time according to the hash value, and reading the regulation and control data into a third cache space.
In this embodiment, when querying data stored in the power grid, it may be first queried whether the regulatory data to be queried is data that is read frequently, and if the regulatory data is within a preset rank, the regulatory data may be directly searched in the second cache space, as in the step described in step S202. If not, semantic feature extraction is carried out on the data to be queried, then the hash value is calculated, the hash value obtained through calculation is searched in the first cache space, then the corresponding regulation and control data are obtained from the database or in a real-time acquisition mode, and the regulation and control data are read into the third cache space for access.
In one embodiment, the regulatory data to be queried is already stored in the database in advance, or can be obtained through a timely reading function. However, since some real-time read data may not change within a certain time, the data can be directly read from the last read value, and then the steps from S202 to S204 are performed.
In one embodiment, the third buffer space is used for storing the pre-read regulation and control data. The regulation and control data in the power grid comprise statistical information and data which need to be collected in real time. And when the regulation data are stored, the regulation data of the statistical information class are stored into a second cache space or a database according to the reading frequency. And the data which needs to be collected in real time cannot be put into the database and cannot be put into the second cache space, and the data which needs to be collected in real time and the data in the database are read in and stored in the third cache space only when the data are inquired.
In one embodiment, when the third cache space stores the same regulation and control data as the newly read data, the newly read regulation and control data replaces the existing regulation and control data.
Once the data is read in the third cache space, the original data needs to be replaced, and the replacement is performed according to a replacement strategy of the power grid regulation data. Specifically, a first-in-first-out strategy may be employed to perform the updating of the data. The pre-reading strategy is mainly based on the continuity of data, and once a certain block of data is read, the adjacent data can be pre-read.
In the above power grid-oriented regulation data storage method and query method, both a frequency prediction network and a feature extraction network are utilized, and the architectures and training methods of the two networks are further described below.
In this embodiment, it is important to predict the reading frequency of the regulation and control data, and it is found that the request frequency of the power grid regulation and control data is related to time and is different with time when the data request frequency of the power grid regulation and control data is analyzed. The data time required by different services in different regions is different mainly determined by the service characteristics of power grid regulation. To better represent the frequency of such data requests, in one embodiment, a poisson equation expressed by a plurality of different parameters is constructed, as follows:
Xq(t)＝{M(t1),M(t2),M(t3),M(t4),M(t5)}
wherein Xq(t) is the data read frequency, M (t)1) Is the Poisson equation, and t1Are parameters in the poisson equation.
In one embodiment, in order to better predict the request frequency of the power grid regulation data, a deep neural network of multi-modal data, namely a frequency prediction network, is provided.
In this embodiment, the types of the control data include text, image, and time series data.
Aiming at different types of regulation and control data, the frequency prediction network can perform frequency prediction of data requests on text data, image data and time sequence data.
In one embodiment, the frequency prediction network includes a plurality of sub-network models and a converged network model, and the sub-network models include: the time sequence long-short term memory network model is used for identifying the regulation and control data of the time sequence type; the image processing and identifying network model is used for identifying the regulation and control data of the image type; and the long-short term network model is used for identifying the regulation data of the text type.
According to the characteristics of power grid regulation and control data, the time sequence long-term and short-term memory network and the text long-term and short-term memory network models all comprise attention mechanisms, the attention mechanisms learn to integrate data information and the like of power grid regulation and control services together into important attention, and reweigh the important attention to emphasize the attention, so that the data reading frequency is accurately predicted.
Specifically, in the time sequence long-term and short-term memory model, the data characteristics of the power grid regulation and control service are input to learn attention weights, and the attention weights are reweighed to identify.
Specifically, in the text long-short term memory network model, the relationship between texts is learned according to the attention characteristics of text data, and the characteristics are weighted again for prediction.
In one embodiment, the power grid regulation and control data passes through the corresponding sub-network model and then is input into the fusion network model. And the converged network model comprises a sharing layer, a first full connection layer, a pooling layer and a second full connection layer which are connected once, and the predicted reading frequency is finally output.
After the frequency prediction network is constructed, in one embodiment, it can be trained in an off-line manner. The frequency prediction network is trained by using the historical request data of the power grid regulation and control to establish the reading frequency prediction of the power grid regulation and control data, and the trained frequency prediction network is very small.
In the training process, the sub-network models corresponding to different data types in the frequency prediction network can be trained together or separately. And putting the trained frequency prediction network into a cache module to perform reading frequency prediction of the request data. When the trained frequency prediction network is used for reading frequency prediction of single request data, only very little memory space is needed, meanwhile, little time is spent, and the requirement on hardware resources is very low.
In order to improve the speed of inquiring and accessing the regulation and control data in the power grid, the unified data expression and storage of the regulation and control data of the power grid are needed, a semantic feature unified expression deep neural network model, namely a feature extraction model, needs to be constructed, and the feature extraction network is utilized to project the regulation and control data of different types to the same semantic space, so that the extracted semantic features have uniformity. Then, the semantic features of the data are subjected to Hash semantic cataloging, and unified semantic cataloging combined with a Hash table is adopted.
The semantic cataloging is to form a catalog by utilizing a semantic relation, and is favorable for classified storage cataloging and retrieval of the power grid regulation and control service data on the semantic level.
In one embodiment, the feature extraction model includes a plurality of different sub-network models for different types of regulatory data.
In one embodiment, for the power grid regulation and control data in the image type mode, in view of the fact that the number of images regulated and controlled by the power grid cannot meet the distribution requirement of independent and same distribution of a deep neural network model, a data-driven deep learning model is difficult to support to extract semantic features, a technical route that sub-network model parameters are pre-trained on an ImageNet large data set and then the sub-network model parameters are fine-tuned on a data set of power grid regulation and control service data is proposed.
In one embodiment, aiming at the power grid regulation and control data in the text type mode, a word2vect method is adopted to project the text data to a semantic space to form a one-dimensional vector.
In one embodiment, the semantic features of the power grid regulation data in the time sequence type mode are extracted through a long-term and short-term memory network.
In one embodiment, when the acquired regulation and control data to be stored is a plurality of data of different types, after semantic features corresponding to the regulation and control data are extracted by using a feature extraction network, the semantic features are fused, and then corresponding hash values are calculated.
It should be noted that the above-mentioned modalities refer to one type of regulation data, the single modality refers to a single type, and the multi-modality refers to having multiple types of regulation data.
In order to support the retrieval and query of multi-type data, after the extracted time sequence, image and text type data characteristics are extracted, the data are linked together to form a one-dimensional vector with fixed length, and then the fused characteristics are subjected to semantic cataloguing by adopting a multi-level hash algorithm.
Aiming at multi-type data, a shared space projection method is adopted, the characteristics extracted from text, images and time sequence type data are uniformly projected to a shared space, then the vector of the space is utilized to carry out semantic cataloguing, namely, the vector of the common space is input to a Hash calculation module, and the Hash value is calculated, and the value can be used as the basis of cataloguing.
The method is similar to a single mode when the power grid regulation and control data are retrieved. In the single mode, the regulation and control data to be retrieved is a single data type such as a text or an image, when retrieval is performed, semantic conversion is performed on the single mode data for retrieval at first (because our catalog is catalogued on a semantic level, semantic conversion is performed on the searched single mode data in advance for searching), once conversion is performed on the semantic level, the semantic feature can be projected, the semantic feature of the shared space is calculated, and finally the hash value is calculated on the feature, so that retrieval can be performed according to the hash value.
The multi-modal data retrieval is similar to a single modality, for example, a user adds an image to a section of text to retrieve the text or the image similar to the section of text or the image, namely, the multi-modal data to be retrieved is projected in a shared space, then the hash value of the multi-modal data is calculated according to the projected vector, the multi-modal data is retrieved in a three-level cache space according to the hash value, and if the retrieval speed is required to be improved, the semantic ratio cataloguing can be carried out by utilizing a multi-level hash method.
In one embodiment, for the problems of complicated kinds of power grid regulation data and various modes, a cross-modal binary code learning technique based on fusion similarity hash is adopted, as shown in fig. 3.
Firstly, the multi-modal power grid regulation data characteristics are mapped to a public Hamming (Hamming) space, so that the similarity between the cross-modal power grid regulation service data characteristics can be represented by Hamming distance. Then, an asymmetric graph model is constructed to represent the fusion similarity between different modes, and finally, the similarity relation between the intra-mode and the inter-mode is learned through learning a binary code.
Fused similarities may be used to better measure similarities across modality data because of the complementary relationship between different similarities. While intra-modal and inter-modal similarity noise may be more robust through binary learning that fuses similarities. Therefore, an efficient objective function is designed to learn the accurate binary code and the corresponding hash function.
Specifically, a similarity matrix is first constructed within each modality, and then they are combined to construct a matrix that can map the fused similarities. According to the similarity matrixes, an asymmetric graph is constructed to learn the hash function. Thereby finding the best fused graph to generate a more identifiable hash code.
When multi-mode data of power grid regulation and control services are stored, similar data can be expected to be stored in similar positions, the multi-mode data are matched firstly, and the data are similar, and the matching and the sequencing are similar.
In one embodiment, aiming at the problem of power grid regulation and control data matching, a template matching measurement technology based on the diversity of feature matching between a target feature window and a template is adopted. The method mainly adopts a feature-based optimal partner similarity parameter-free matching method, and provides a powerful clue for template matching based on two attributes of a 'nearest neighbor' field of a matching item between a point of a target window and a template.
Specifically, to measure the similarity between the target window and the template, its Nearest Neighbors (NN) are first found in appearance for each target patch. The key idea is that the similarity between the target and the template is captured by two properties that imply the NN field. When the target and template correspond, most target patches have a unique NN match in the template. This means that the NN field is highly diverse, pointing to many different patches in the template. Conversely, if the target and template do not correspond, then for any target most patches do not have a good match and the NN converges to a small number of template points that are just similar to the target patch.
Further, arbitrary matching usually means large deformation, so two methods are proposed to quantify the diversity and the deformation of the NN field, that is, the diversity similarity and the deformation diversity similarity, so that the matching accuracy of the deformed power grid regulation image data can be greatly increased.
In one embodiment, the present application further provides a power grid-oriented regulatory data cache system, which includes a trained feature extraction network, a trained frequency prediction network, a hash value calculation module, a first cache space, a second cache space, and a third cache space;
the trained frequency prediction network is used for calculating the reading frequency of the regulation and control data and storing the regulation and control data into the second cache space or the database according to the reading frequency;
the trained feature extraction network is used for extracting semantic features of the regulation and control data and sending the semantic features to the hash value calculation module;
the hash value calculation module is used for calculating a hash value corresponding to the semantic features and storing the hash value into a first cache space;
and the third cache space is used for storing the regulation and control data which is read in advance.
For specific limitations of the regulation data cache system facing the power grid, reference may be made to the above limitations of the regulation data cache method facing the power grid, and details thereof are not repeated here.
In the power grid-oriented regulation and control data caching method, characteristic analysis is performed on power grid regulation and control service data, the rule of data request of regulation and control service users, the requirements of special data on a caching strategy and a replacement strategy and the like. The method comprises the steps of designing a data reading frequency prediction algorithm according to rules and characteristics of power grid regulation and control service user requests, carrying out cache pre-reading, cache strategies and replacement strategies according to predicted reading frequency ranking, and designing a cache architecture model based on a three-level cache space, wherein a first-level cache is used for storing semantic cataloging information of various types of data requested by the power grid regulation and control service, a second-level cache is used for storing data with the highest frequency which is frequently accessed, and a third-level cache is mainly used for storing some pre-read power grid regulation and control data. In order to match with a data request and cache model, uniform data expression and storage of power grid regulation and control data are required, a semantic feature uniform expression deep neural network model is firstly constructed, then hash semantic features of the data are subjected to semantic cataloging, and the unified semantic cataloging combined with a hash table is adopted and is mainly used for statistical data access and query. The data access and query process firstly extracts and analyzes the query data according to semantics, and finds data similar to the query or similar to the query by the proposed semantic features through a hash table. The power grid regulation and control data caching method greatly improves the speed of query and range.
It should be understood that although the various steps in the flow charts of fig. 1-2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 4, there is provided a regulation data cache device facing a power grid, including: a data acquisition module 301, a semantic feature extraction module 302, and a data storage module 303, wherein:
the data acquisition module 301 is configured to acquire regulation and control data to be stored;
a semantic feature extraction module 302, configured to extract a semantic feature of the regulation and control data by using a trained feature extraction network, calculate a corresponding hash value according to the semantic feature, and store the hash value in a first cache space;
the data storage module 303 is configured to calculate a reading frequency of the control data by using the trained frequency prediction network, and store the control data in a second cache space or a database according to the reading frequency.
For specific limitations of the regulation data cache device facing the power grid, reference may be made to the above limitations of the regulation data cache method facing the power grid, and details thereof are not repeated here. All or part of each module in the regulation and control data cache device facing the power grid can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 5. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing part of the regulation data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a grid-oriented regulation data caching method.
Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring regulation and control data to be stored;
extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
the first cache space is used for storing hash values corresponding to the stored regulation and control data and taking the hash values as an index directory;
and the second cache space is used for storing the regulation and control data with the reading frequency within the preset ranking.
In one embodiment, the processor, when executing the computer program, further performs the steps of:
the regulation data caching method further comprises the step of inquiring the stored regulation data, and the inquiring method comprises the following steps:
acquiring regulation and control data to be inquired;
calculating the reading frequency of the regulation and control data by using the frequency prediction network, and if the reading frequency is within the preset ranking, inquiring the corresponding regulation and control data in the second cache;
if the reading frequency is not within the preset ranking, extracting semantic features of the regulation and control data by using the feature extraction network, and calculating corresponding hash values according to the semantic features;
and after finding the corresponding hash value in the index directory in the first cache space according to the hash value, acquiring corresponding regulation and control data from the database or in real time according to the hash value, and reading the regulation and control data into a third cache space.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring regulation and control data to be stored;
extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the first cache space is used for storing hash values corresponding to the stored regulation and control data and taking the hash values as an index directory;
and the second cache space is used for storing the regulation and control data with the reading frequency within the preset ranking.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the regulation data caching method further comprises the step of inquiring the stored regulation data, and the inquiring method comprises the following steps:
acquiring regulation and control data to be inquired;
calculating the reading frequency of the regulation and control data by using the frequency prediction network, and if the reading frequency is within the preset ranking, inquiring the corresponding regulation and control data in the second cache;
if the reading frequency is not within the preset ranking, extracting semantic features of the regulation and control data by using the feature extraction network, and calculating corresponding hash values according to the semantic features;
and after finding the corresponding hash value in the index directory in the first cache space according to the hash value, acquiring corresponding regulation and control data from the database or in real time according to the hash value, and reading the regulation and control data into a third cache space.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. The regulation and control data cache method facing to the power grid is characterized by comprising the following steps:
acquiring regulation and control data to be stored;
extracting semantic features of the regulation and control data by using a trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and calculating the reading frequency of the regulation and control data by using the trained frequency prediction network, and storing the regulation and control data into a second cache space or a database according to the reading frequency.
2. The throttling data caching method according to claim 1,
the first cache space is used for storing hash values corresponding to the regulation and control data, and the hash values are used as index directories;
and the second cache space is used for storing the regulation and control data with the reading frequency within the preset ranking.
3. The throttling data caching method according to claim 2, wherein the throttling data caching method further comprises querying stored throttling data, the querying method comprising:
acquiring regulation and control data to be inquired;
calculating the reading frequency of the regulation and control data by using the frequency prediction network, and if the reading frequency is within the preset ranking, inquiring the corresponding regulation and control data in the second cache;
if the reading frequency is not within the preset ranking, extracting semantic features of the regulation and control data by using the feature extraction network, and calculating corresponding hash values according to the semantic features;
and after finding the corresponding hash value in the index directory in the first cache space according to the hash value, acquiring corresponding regulation and control data from the database or in real time according to the hash value, and reading the regulation and control data into a third cache space.
4. The method of claim 3, wherein when the third cache space already stores the same regulatory data as the newly read data, the newly read regulatory data replaces the existing regulatory data.
5. A regulation data caching method according to claim 1, wherein the types of regulation data comprise text, images and time series data;
the feature extraction network projects the different types of regulation and control data to the same semantic space, so that the extracted semantic features have uniformity.
6. The throttling data caching method according to claim 5,
when the regulation and control data to be stored are a plurality of data of different types, after the semantic features corresponding to the regulation and control data are extracted by using the feature extraction network, the semantic features are fused, and then the corresponding hash value is calculated.
7. The regulatory data caching method of claim 5, wherein the frequency prediction network comprises a plurality of sub-network models and a converged network model, the sub-network models comprising:
the time sequence long-short term memory network model is used for identifying the regulation and control data of the time sequence type;
the image processing and identifying network model is used for identifying the regulation and control data of the image type; and
and the long-short term network model is used for identifying the regulation data of the text type.
8. A power grid-oriented regulation data caching device, comprising:
the data acquisition module is used for acquiring the regulation and control data to be stored;
the semantic feature extraction module is used for extracting semantic features of the regulation and control data by using the trained feature extraction network, calculating corresponding hash values according to the semantic features, and storing the hash values into a first cache space;
and the data storage module is used for calculating the reading frequency of the regulation and control data by utilizing the trained frequency prediction network and storing the regulation and control data into a second cache space or a database according to the reading frequency.
9. A computer arrangement comprising a memory and a processor, the memory storing a computer program, wherein the processor when executing the computer program performs the steps of the grid-oriented regulation data caching method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the grid-oriented regulation data caching method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110391693.1A CN113127515A (en) | 2021-04-12 | 2021-04-12 | Power grid-oriented regulation and control data caching method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110391693.1A CN113127515A (en) | 2021-04-12 | 2021-04-12 | Power grid-oriented regulation and control data caching method and device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113127515A true CN113127515A (en) | 2021-07-16 |
Family
ID=76775864
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110391693.1A Pending CN113127515A (en) | 2021-04-12 | 2021-04-12 | Power grid-oriented regulation and control data caching method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113127515A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114327672A (en) * | 2021-12-14 | 2022-04-12 | 中国平安财产保险股份有限公司 | Data caching time setting method and device, computer equipment and storage medium |
| CN117435144A (en) * | 2023-12-20 | 2024-01-23 | 山东云天安全技术有限公司 | Intelligent data hierarchical security management method and system for data center |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105117180A (en) * | 2015-09-28 | 2015-12-02 | 联想(北京)有限公司 | Data storing method and device and solid state disc |
| US20180349735A1 (en) * | 2015-07-23 | 2018-12-06 | Beijing Jingdong Shangke Information Technology Co Ltd. | Method and Device for Comparing Similarities of High Dimensional Features of Images |
| CN109976905A (en) * | 2019-03-01 | 2019-07-05 | 联想(北京)有限公司 | EMS memory management process, device and electronic equipment |
| CN112287140A (en) * | 2020-10-28 | 2021-01-29 | 汪礼君 | Image retrieval method and system based on big data |
-
2021
- 2021-04-12 CN CN202110391693.1A patent/CN113127515A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180349735A1 (en) * | 2015-07-23 | 2018-12-06 | Beijing Jingdong Shangke Information Technology Co Ltd. | Method and Device for Comparing Similarities of High Dimensional Features of Images |
| CN105117180A (en) * | 2015-09-28 | 2015-12-02 | 联想(北京)有限公司 | Data storing method and device and solid state disc |
| CN109976905A (en) * | 2019-03-01 | 2019-07-05 | 联想(北京)有限公司 | EMS memory management process, device and electronic equipment |
| CN112287140A (en) * | 2020-10-28 | 2021-01-29 | 汪礼君 | Image retrieval method and system based on big data |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114327672A (en) * | 2021-12-14 | 2022-04-12 | 中国平安财产保险股份有限公司 | Data caching time setting method and device, computer equipment and storage medium |
| CN114327672B (en) * | 2021-12-14 | 2024-04-05 | 中国平安财产保险股份有限公司 | Data caching time setting method and device, computer equipment and storage medium |
| CN117435144A (en) * | 2023-12-20 | 2024-01-23 | 山东云天安全技术有限公司 | Intelligent data hierarchical security management method and system for data center |
| CN117435144B (en) * | 2023-12-20 | 2024-03-22 | 山东云天安全技术有限公司 | Intelligent data hierarchical security management method and system for data center |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107909108A (en) | Edge cache system and method based on content popularit prediction | |
| Ali et al. | Intelligent web proxy caching approaches based on machine learning techniques | |
| CN108710639B (en) | Ceph-based access optimization method for mass small files | |
| CN110134714B (en) | Distributed computing framework cache index method suitable for big data iterative computation | |
| Fan et al. | PA-cache: Evolving learning-based popularity-aware content caching in edge networks | |
| CN102236674B (en) | Method and device for updating index page | |
| CN113127515A (en) | Power grid-oriented regulation and control data caching method and device, computer equipment and storage medium | |
| Li et al. | Adaptive priority-based cache replacement and prediction-based cache prefetching in edge computing environment | |
| JP2007317068A (en) | Recommending device and recommending system | |
| CN113778683B (en) | Handle identification system analysis load balancing method based on neural network | |
| Gellert et al. | Web prefetching through efficient prediction by partial matching | |
| CN117056250A (en) | Cache replacement method and device, electronic equipment and storage medium | |
| CN108833933A (en) | A kind of method and system for recommending video flow using support vector machines | |
| Chao | Web cache intelligent replacement strategy combined with GDSF and SVM network re-accessed probability prediction | |
| US9367454B2 (en) | Address index recovery using hash-based exclusive or | |
| Qiu et al. | OA-cache: Oracle approximation-based cache replacement at the network edge | |
| WO2024207834A1 (en) | Multi-level cache adaptive system and strategy based on machine learning | |
| Ugwuanyi et al. | A novel predictive-collaborative-replacement (PCR) intelligent caching scheme for multi-access edge computing | |
| WO2023030227A1 (en) | Data processing method, apparatus and system | |
| Songwattana et al. | A learning-based approach for web cache management | |
| CN113901278A (en) | Data search method and device based on global multi-detection and adaptive termination | |
| Baskaran et al. | Study of combined Web prefetching with Web caching based on machine learning technique | |
| CN113268458A (en) | Caching method and system based on cost-sensitive classification algorithm | |
| Niloy et al. | Intellcache: An intelligent web caching scheme for multimedia contents | |
| CN115017435B (en) | Method and device for determining cache resources, nonvolatile storage medium and processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210716 |