CN111177163A - Data access method and device for time mark data and storage medium - Google Patents
Data access method and device for time mark data and storage medium Download PDFInfo
- Publication number
- CN111177163A CN111177163A CN201911247118.3A CN201911247118A CN111177163A CN 111177163 A CN111177163 A CN 111177163A CN 201911247118 A CN201911247118 A CN 201911247118A CN 111177163 A CN111177163 A CN 111177163A
- Authority
- CN
- China
- Prior art keywords
- data
- time
- section
- memory
- access method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/06—Electricity, gas or water supply
Abstract
The invention discloses a data access method, a storage medium and equipment for time mark data, which comprise a data storage method and a data access method, wherein the data storage method comprises the following steps: storing the data cross section in a memory according to a set time interval, and storing data between data cross section moments in a data change sequence; the data access method comprises the following steps: firstly, finding a data section moment closest to the query moment, and returning the value of data to be queried in the data section if the query moment is the data section moment; if the query time is not equal to the data section time, retrieving whether a data change sequence exists in a memory where the data section is located, and if not, returning the value of the queried data in the data section; if so, returning the value of the data queried at the time closest to the query time. The invention has high efficiency when searching a large number of data points at the same time; the segmented data storage is adopted, so that the pressure of long-time scale data access on a memory can be reduced.
Description
Technical Field
The invention relates to the technical field of automatic data management of power systems, in particular to a data access method and device for time scale data and a storage medium.
Background
With the increasingly complex operation mode of the power grid, the scheduling operation management of the power grid faces new challenges, data of a plurality of monitoring units need to be concentrated for comprehensive processing, meanwhile, the time for accurate analysis of the power grid to need the data can be consistent, and therefore the data which can ensure the consistency of time scales needs to be used in a large scale.
At present, the problem of time scale consistency can be solved by adopting a time sequence database to store data, but the time sequence database data is suitable for accessing specific data points according to multiple time periods, and the efficiency is not high when a large number of data points at the same moment are searched.
Disclosure of Invention
In order to solve the defects in the prior art, the invention provides a data access method, a device and a storage medium facing time scale data, and solves the problem of low efficiency in searching a large number of data points at the same moment.
In order to achieve the above purpose, the invention adopts the following technical scheme:
in a first aspect, the present invention provides a data access method for timestamp data, including:
determining the data section time closest to the query time;
if the query time is the data section time, returning the value of the data to be queried in the data section; if the query time is not equal to the data section time, retrieving whether a data change sequence exists in a memory where the data section is located, and if not, returning the value of the queried data in the data section; if yes, returning the value of the data queried at the time closest to the query time;
the data section is stored in a memory according to a set time interval; and storing data between data section moments in the data change sequence.
Further, the data section is a data set formed by all data at the same time point.
Further, data profiles at different times are stored in different memory cells.
Further, the data change sequence is stored in the memory unit of the previous data section.
Further, the data in the data cross section each include a time scale of the time when the data is generated.
Further, still include: and writing the data section into a file when the storage of the data section in the memory exceeds a set time length.
Further, still include: and after the data section time closest to the query time is found, judging whether the section exists in the memory, if not, finding a file of the section, and loading the file to the memory.
Further, the sequence of changes comprises: a data value and a time stamp.
In a second aspect, the present invention provides a data access apparatus for timestamp data, including:
the determining module is used for determining the data section time closest to the query time;
the processing module is used for returning the value of the data to be inquired in the data section if the inquiry time is the data section time; if the query time is not equal to the data section time, retrieving whether a data change sequence exists in a memory where the data section is located, and if not, returning the value of the queried data in the data section; if yes, returning the value of the data queried at the time closest to the query time;
the data section is stored in a memory according to a set time interval; and storing data between data section moments in the data change sequence.
In a third aspect, the present invention provides a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the timestamp data oriented data storage access method as described above.
The invention achieves the following beneficial effects:
the invention adopts the memory to access the section data, is beneficial to the quick access of the time scale data of the whole section, and has high efficiency when a large number of data points at the same moment are searched; the segmented data storage is adopted, so that the pressure of long-time scale data access on a memory can be reduced.
Drawings
Fig. 1 is a flow chart of a data access method according to an embodiment of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
Example 1:
all data at the same time point in the system form a complete data set as a data section of the system at the moment, and the method manages section data in a mode of storing data sections with fixed time intervals and data change sequences among the data sections, rather than storing one section per second.
A data storage access method facing time mark data comprises the following steps:
a data storage method comprising the steps of:
1. selecting time according to a set time interval, storing all data at the time in the system as data sections in a memory, storing the data sections at different times in different memory units, wherein the data forming the sections all have time marks for generating the time; for example, a section is set every 10 seconds, the set of all data values at the time 2:30:00 in the system is a data section of 2:30:00, and the set of all data values at the time 2:30:10 is a data section of 2:30: 10;
2. data between the time instants of the sections are stored in a change sequence, and if a certain data changes between the time instants of the two data sections, values and time stamps (i.e., the time instants of the changes) are recorded in the order of the change. For example, continuing in the example of step 1, if a data change is 1 at 2:30:01 and 5 at 2:30:05, the system records the change sequence of the data as [1(2:30:01), 5(2:30:05) ], and the data change sequence between sections is stored in the memory unit of the previous section.
3. And setting a time interval, writing the data section in the storage unit in the memory into a file if the difference between the time interval and the current time exceeds the time interval, and releasing the storage unit of the memory to reduce resource consumption.
As shown in fig. 1, the data access method includes the steps of:
1. accessing the moment of inquiring data according to needs, and obtaining the data section moment closest to the moment according to a set time interval; for example, data with a time scale of 2:30:06, the closest cross section is 2:30: 00.
2. If the section does not exist in the memory, finding a file with the section being persistent, and loading the file into the memory;
3. if the query time is the data section time, returning the data value recorded by the data in the data section;
4. if the query time is not equal to the data section time, retrieving whether a change sequence of the data exists in a storage unit of the current data section, if the change sequence does not exist, returning the value of the data recorded in the data section, and if the change sequence exists, searching the change sequence, finding the data at the time closest to the query time and returning. For example, the change sequence is [1(2:30:01), 5(2:30:05) ], the query time is 2:30:06, and data 5 at time 2:30:05 is returned.
Example 2
Based on the same inventive concept, the invention provides a data storage access device facing time scale data, which comprises:
the determining module is used for determining the data section time closest to the query time;
the processing module is used for returning the value of the data to be inquired in the data section if the inquiry time is the data section time; if the query time is not equal to the data section time, retrieving whether a data change sequence exists in a memory where the data section is located, and if not, returning the value of the queried data in the data section; if yes, returning the value of the data queried at the time closest to the query time;
the data section is stored in a memory according to a set time interval; and storing data between data section moments in the data change sequence.
Example 3
Based on the same inventive concept, the present invention provides a computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform a timestamp data oriented data storage access method as described in embodiment 1.
Example 4
Based on the same inventive concept, the present invention provides a computing device comprising one or more processors, memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing a timestamp data oriented data store access method of embodiment 1.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A data access method facing time mark data is characterized in that:
determining the data section time closest to the query time;
if the query time is the data section time, returning the value of the data to be queried in the data section; if the query time is not equal to the data section time, retrieving whether a data change sequence exists in a memory where the data section is located, and if not, returning the value of the queried data in the data section; if yes, returning the value of the data queried at the time closest to the query time;
the data section is stored in a memory according to a set time interval; and storing data between data section moments in the data change sequence.
2. The data access method for time stamp data according to claim 1, wherein: the data section is a data set formed by all data at the same time point.
3. The data access method for time stamp data according to claim 1, wherein: the data profiles at different times are stored in different memory cells.
4. The data access method for time stamp data according to claim 1, wherein: the data change sequence is stored in the memory unit of the previous data section.
5. The data access method for time stamp data according to claim 1, wherein: the data in the data sections each include a time scale of the time at which the data was generated.
6. The data access method for time stamp data according to claim 1, wherein: further comprising: and writing the data section into a file when the storage of the data section in the memory exceeds a set time length.
7. The data access method for time stamp data according to claim 1, wherein: further comprising: and after the data section time closest to the query time is found, judging whether the section exists in the memory, if not, finding a file of the section, and loading the file to the memory.
8. The data access method for time stamp data according to claim 1, wherein: the sequence of changes comprises: a data value and a time stamp.
9. A data access apparatus for time stamp data, characterized in that: the method comprises the following steps:
the determining module is used for determining the data section time closest to the query time;
the processing module is used for returning the value of the data to be inquired in the data section if the inquiry time is the data section time; if the query time is not equal to the data section time, retrieving whether a data change sequence exists in a memory where the data section is located, and if not, returning the value of the queried data in the data section; if yes, returning the value of the data queried at the time closest to the query time;
the data section is stored in a memory according to a set time interval; and storing data between data section moments in the data change sequence.
10. A computer-readable storage medium characterized by: the computer readable storage medium stores one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the timestamp data oriented data storage access method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911247118.3A CN111177163A (en) | 2019-12-09 | 2019-12-09 | Data access method and device for time mark data and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911247118.3A CN111177163A (en) | 2019-12-09 | 2019-12-09 | Data access method and device for time mark data and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111177163A true CN111177163A (en) | 2020-05-19 |
Family
ID=70653819
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911247118.3A Pending CN111177163A (en) | 2019-12-09 | 2019-12-09 | Data access method and device for time mark data and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111177163A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106844666A (en) * | 2017-01-20 | 2017-06-13 | 北京理工大学 | A kind of time series data querying method of self adaptation |
| CN108073586A (en) * | 2016-11-09 | 2018-05-25 | 中国石油天然气股份有限公司 | Crash analysis method and apparatus based on oil-gas pipeline SCADA system |
| CN108228734A (en) * | 2017-12-12 | 2018-06-29 | 上海天旦网络科技发展有限公司 | A kind of system and method for data visualization |
| CN108874859A (en) * | 2018-04-17 | 2018-11-23 | 中国电力科学研究院有限公司 | It is a kind of to dispatch real-time profile data generation method and system |
| CN109388596A (en) * | 2018-09-29 | 2019-02-26 | 上海依图网络科技有限公司 | A kind of data manipulation method and device |
| CN110008249A (en) * | 2019-01-31 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of time-based data query method, device and equipment |
-
2019
- 2019-12-09 CN CN201911247118.3A patent/CN111177163A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108073586A (en) * | 2016-11-09 | 2018-05-25 | 中国石油天然气股份有限公司 | Crash analysis method and apparatus based on oil-gas pipeline SCADA system |
| CN106844666A (en) * | 2017-01-20 | 2017-06-13 | 北京理工大学 | A kind of time series data querying method of self adaptation |
| CN108228734A (en) * | 2017-12-12 | 2018-06-29 | 上海天旦网络科技发展有限公司 | A kind of system and method for data visualization |
| CN108874859A (en) * | 2018-04-17 | 2018-11-23 | 中国电力科学研究院有限公司 | It is a kind of to dispatch real-time profile data generation method and system |
| CN109388596A (en) * | 2018-09-29 | 2019-02-26 | 上海依图网络科技有限公司 | A kind of data manipulation method and device |
| CN110008249A (en) * | 2019-01-31 | 2019-07-12 | 阿里巴巴集团控股有限公司 | A kind of time-based data query method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633277B (en) | Time sequence data storage method, device, computer equipment and storage medium | |
| CN106648446B (en) | Storage method and device for time sequence data and electronic equipment | |
| CN108932257B (en) | Multi-dimensional data query method and device | |
| CN107122126B (en) | Data migration method, device and system | |
| CN104993974A (en) | Detection method for identifying equipment working state, and upper computer | |
| US20150234883A1 (en) | Method and system for retrieving real-time information | |
| CN110941602B (en) | Database configuration method and device, electronic equipment and storage medium | |
| CN108616556B (en) | Data processing method, device and system | |
| WO2021169386A1 (en) | Graph data processing method, apparatus and device, and medium | |
| JP2020057416A (en) | Method and device for processing data blocks in distributed database | |
| Rasheduzzaman et al. | Task shape classification and workload characterization of google cluster trace | |
| CN104270605A (en) | Method and device for processing video monitoring data | |
| CN110413631B (en) | Data query method and device | |
| CN104636401B (en) | A kind of method and device of SCADA system data rewind | |
| WO2021012861A1 (en) | Method and apparatus for evaluating data query time consumption, and computer device and storage medium | |
| CN104216977A (en) | Time series data search method and device | |
| CN104991836A (en) | Database recovery method and apparatus and database server | |
| CN107644033A (en) | A kind of method and apparatus that data query is carried out in non-relational database | |
| CN111177163A (en) | Data access method and device for time mark data and storage medium | |
| CN113094154A (en) | Big data processing method and system based on Aliyun | |
| CN110765074B (en) | Method and system for quickly accessing electric load curve data of acquisition terminal | |
| CN110019357B (en) | Database query script generation method and device | |
| CN110580253A (en) | Time sequence data set loading method and device, storage medium and electronic equipment | |
| CN106570193B (en) | Method for loading time sequence big data | |
| CN105468726A (en) | Local computing and distributed computing based data computing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |