CN113177143A - Time sequence data access method and device, storage medium and electronic equipment - Google Patents
Time sequence data access method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN113177143A CN113177143A CN202110349472.8A CN202110349472A CN113177143A CN 113177143 A CN113177143 A CN 113177143A CN 202110349472 A CN202110349472 A CN 202110349472A CN 113177143 A CN113177143 A CN 113177143A
- Authority
- CN
- China
- Prior art keywords
- data
- time
- accessed
- storage
- source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 230000005236 sound signal Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000000630 rising effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure relates to a time sequence data access method, a time sequence data access device, a storage medium and an electronic device, wherein the method comprises the following steps: acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information; the method comprises the steps of determining a target time zone storage unit corresponding to time sequence data to be accessed according to time attribute information of the time sequence data to be accessed and a time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to data source attribute information and a data source label corresponding to each data source storage subunit in the target time zone storage unit, carrying out access operation on the time sequence data to be accessed in the target data source storage subunit, and reducing the area of the access operation through the time attribute information and the data source attribute information of the time sequence data to be accessed, so that the efficiency of the access operation is improved.
Description
Technical Field
The present disclosure relates to the field of database technologies, and in particular, to a method and an apparatus for accessing time series data, a storage medium, and an electronic device.
Background
In the related art, a conventional two-dimensional storage method is used to store data. The two-dimensional storage method refers to planar storage in a line recording manner. For the data of the management class, the traditional two-dimensional storage method can meet the requirement because the data of the management class does not need much extra processing. However, for the time series data collected from various devices by a collector or a sensor at regular time, because the time series data has the characteristics of time sequence, large data volume, frequent writing and random reading, the storage and query of the time series data need additional processing, and the efficiency of data reading is seriously influenced when the data volume is large due to the planarization of the data storage of the traditional two-dimensional storage method.
Disclosure of Invention
The invention aims to provide a time sequence data access method, a time sequence data access device, a storage medium and electronic equipment.
In order to achieve the above object, in a first aspect, the present disclosure provides a time series data access method applied to an electronic device, where the electronic device includes a storage device, the storage device includes a plurality of time zone storage units, each time zone storage unit includes at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag, the method includes:
acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
and determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
Optionally, the data source storage subunit is a source device type storage subunit, the data source tag is a source device type tag, each source device type storage subunit includes at least one source device storage submodule, and the source device storage submodule is provided with a corresponding source device tag;
determining a target data source storage subunit corresponding to the to-be-accessed time sequence data according to the data source attribute information and the data source tag corresponding to each data source storage subunit in the target time zone storage unit, and performing an access operation on the to-be-accessed time sequence data in the target data source storage subunit, where the determining includes:
determining a target source device type storage subunit corresponding to the time sequence data to be accessed according to source device type information in the data source attribute information and a source device type label corresponding to each source device type storage subunit in the target time zone storage unit;
and determining a target source device storage submodule according to a source device identifier in the data source attribute information and a source device label of each source device storage submodule in the target source device type storage submodule, and performing access operation on the time sequence data to be accessed in the target source device storage submodule.
Optionally, the source device storage submodule further includes at least one data type storage submodule, and the data type storage submodule is provided with a corresponding data type tag;
the accessing operation of the time sequence data to be accessed in the storage submodule of the target source device comprises:
acquiring data type information of the data to be accessed;
determining a target data type storage submodule according to the data type information and a data type label corresponding to each data type storage submodule in the target source equipment storage submodule;
and performing access operation on the time sequence data to be accessed in the target data type storage submodule.
Optionally, the time series data includes an index value, the access operation is a write operation, and the performing an access operation on the time series data to be accessed in the target data type storage submodule includes:
and under the condition that the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data strip by the target data type storage submodule last time, updating the termination time point of the data strip and the statistical frequency information of the data strip according to the time attribute information of the time sequence data to be accessed, wherein the starting time point of the data strip is set for each data strip according to the starting time point of the time sequence data corresponding to the initially written index value, and the statistical frequency information is used for representing the number of the index values recorded by the corresponding data strip.
Optionally, the method further comprises:
under the condition that the index value in the time sequence data to be accessed is different from the index value in the time sequence data written in the data strip last time by the target data type storage submodule, determining a first slope according to the index value of the time sequence data to be accessed and any index value in the data strip;
when the difference between the first slope and a second slope between any two index values stored in the data strip is larger than a preset threshold value, generating a new data strip, setting a starting time point and an ending time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index values in the time sequence data to be accessed into the data strip, and setting the statistical frequency information of the new data strip.
Optionally, the method further comprises:
when the difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to the preset threshold, updating the termination time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence to be accessed into the data strip, and updating the statistical frequency information of the data strip; alternatively, the first and second electrodes may be,
when the difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to the preset threshold, if the last written index value of the data strip is not the first index value written in the data strip, updating the last written index value of the data strip according to the index value in the time sequence to be accessed, updating the termination time point of the data strip according to the time attribute information of the time sequence data to be accessed, and updating the statistical frequency information of the data strip.
In a second aspect, the present disclosure provides a time series data access apparatus applied to an electronic device, where the electronic device includes a storage device, the storage device includes a plurality of time zone storage units, each time zone storage unit includes at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag, the apparatus includes:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring time sequence data to be accessed, the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
and the access module is used for determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
Optionally, the data source storage subunit is a source device type storage subunit, the data source tag is a source device type tag, each source device type storage subunit includes at least one source device storage submodule, and the source device storage submodule is provided with a corresponding source device tag;
the access module includes:
a first access sub-module, configured to determine, according to source device type information in the data source attribute information and a source device type tag corresponding to each source device type storage sub-unit in the target time zone storage unit, a target source device type storage sub-unit corresponding to the to-be-accessed time series data;
and the second accessing submodule is used for determining a target source device storage submodule according to a source device identifier in the data source attribute information and a source device label of each source device storage submodule in the target source device type storage submodule, and performing access operation on the time sequence data to be accessed in the target source device storage submodule.
In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of any of the first aspects described above.
In a fourth aspect, the present disclosure provides an electronic device comprising:
a storage device having a computer program stored thereon;
a processor for executing the computer program in the storage device to implement the steps of the method of any of the first aspect above.
According to the technical scheme, the target time zone storage unit for the access operation is locked according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit; and locking the target data source storage subunit for the access operation according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, namely, reducing the area of the access operation through the time attribute information and the data source attribute information of the time sequence data to be accessed and positioning to the root position corresponding to the access operation, so that the efficiency of the access operation can be improved.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
fig. 1 is a flowchart illustrating a time series data access method according to an exemplary embodiment of the present disclosure.
Fig. 2 is a schematic diagram illustrating a storage structure of a storage device according to an exemplary embodiment of the present disclosure.
FIG. 3 is a schematic diagram illustrating a storage structure of a time zone storage unit according to an exemplary embodiment of the present disclosure.
Fig. 4 is a block diagram illustrating a time series data access apparatus according to an exemplary embodiment of the present disclosure.
Fig. 5 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
In the related art, it is usually necessary to record each index of the equipment at each time point, so as to analyze the operation state of the equipment subsequently according to each index continuously. As background art shows, table 1 is a two-dimensional database table corresponding to a two-dimensional storage method:
TABLE 1
Table 1 lists only a small amount of time-series data, but since the amount of time-series data is large, table 1 stores the time-series data by a large number of line records. When a certain time series data is queried, the row records in table 1 need to be continuously traversed, and the querying efficiency is seriously affected under the condition of large time series data volume.
In view of the above, the present disclosure provides a method, an apparatus, a storage medium, and an electronic device for accessing time series data, which store time series data on a new storage structure to improve efficiency of access operations.
Fig. 1 is a flowchart illustrating a time series data access method according to an exemplary embodiment of the present disclosure. Referring to fig. 1, the time series data access method may include the steps of:
And 102, determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit.
And 104, performing access operation on the time sequence data to be accessed in the target data source storage subunit.
For example, the time series data in the present disclosure may be time series data obtained by monitoring equipment in the field of automation monitoring operation and maintenance. The device may be, for example, a host, a display screen, etc.
The time sequence data access method in the disclosure can be applied to an electronic device, which includes a storage device, the storage device includes a plurality of time zone storage units, each time zone storage unit includes at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag.
The time zone label represents a time range to which the acquisition time of the time sequence data belongs. In a possible manner, the time zone label can characterize time data within 7 days and also time series data within 1 month. For example, the time zone label may indicate a time range of 1 month, 1 day 00:00:00, 2021 year to 1 month, 7 days, 24:00:00, 2021 year. Correspondingly, the data source storage subunit under the time zone storage unit corresponding to the time zone label stores the time sequence data with the acquisition time within the range of 1 month and 1 day 00:00:00 in 2021 year to 1 month and 7 days 24:00:00 in 2021 year.
The data source tag characterizes the source of the time series data. It should be understood that different source tags of timing data represent different sources of timing data. The source of the time series data may be from a certain type of device, or from a certain type of device, and specifically from a certain IP address. In a possible mode, under the condition that the data source label represents the type of the source equipment, the data source label can represent a host type label and can also represent a display screen type label; in the case where the data source tag represents a certain type of device and is a source device of a certain IP address, the data source tag may represent a device tag of an IP address of 192.168.100.10, or may represent a device tag of an IP address of 192.168.100.11.
Illustratively, the time attribute information of the time series data to be accessed is time information of collecting the time series data to be accessed. For example, when the time-series data collected by the host is sorted at 9 on 1/2021, the time attribute information of the time-series data is 9:00:00 on 1/2021.
For example, the data source attribute information of the time series data to be accessed may be a source device type, or may be a source device belonging to a certain device type and being a certain IP address.
Illustratively, the access operation may be a write operation or a query operation.
By the mode, the target time zone storage unit for the access operation of the time sequence data to be accessed is locked according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit by utilizing the storage structure characteristics of the storage equipment; and locking the target data source storage subunit to which the access operation of the time sequence data to be accessed is directed according to the data source attribute information and the data source tag corresponding to each data source storage subunit in the target time zone storage unit, namely, reducing the area of the access operation by the time attribute information and the data source attribute information of the time sequence data to be accessed, and positioning the area to the root position corresponding to the access operation, so that the efficiency of the access operation can be improved.
It should be noted that, in the present disclosure, the storage device may maintain the time zone storage units corresponding to different time zone labels through a two-dimensional table; each time zone storage unit can maintain the data source storage subunits corresponding to different data source labels through a two-dimensional table.
In a possible manner, the data source storage subunit is a source device type storage subunit, the data source tag is a source device type tag, each source device type storage subunit includes at least one source device storage submodule, and the source device storage submodule is provided with a corresponding source device tag.
For example, the source device type tag may be a host type tag, and accordingly, only the timing data of the host itself, such as the CPU of the host, the temperature of the host, and the like, is stored in the source device storage sub-module under the source device type storage sub-unit corresponding to the source device type tag.
In a possible manner, step 103 shown in fig. 1 may include: determining a target source equipment type storage subunit corresponding to the time sequence data to be accessed according to the source equipment type information in the data source attribute information and a source equipment type label corresponding to each source equipment type storage subunit in the target time zone storage unit; and determining the target source equipment storage submodule according to the source equipment identifier in the data source attribute information and the source equipment label of each source equipment storage submodule in the target source equipment type storage submodule. Wherein, the source device identifier may be an IP identifier characterizing the unique device.
Step 104 shown in fig. 1 may include: and performing access operation on the time sequence data to be accessed in the storage submodule of the target source equipment.
Exemplarily, fig. 2 is a schematic diagram illustrating a storage structure of a storage device according to an exemplary embodiment of the present disclosure. Referring to fig. 2, A, B, C and D are time zone labels indicating that different time zone storage units, i.e., A, B, C and D represent different time ranges; a1, a2, a3 and a4 are source device type labels, which indicate that different data source storage subunits, namely, the device types represented by a1, a2, a3 and a4 are different; e. f, g and h are source device tags, indicating different source device storage submodules, i.e. e, f, g and h represent unique devices. For example, if the time attribute information of the time series data to be accessed corresponds to the time range represented by a, the access operation of the time series data to be accessed can be located to the time zone storage unit corresponding to a; further, if the source device type information of the time series data to be accessed corresponds to the device type represented by a2, the access operation of the time series data to be accessed can be further positioned to the data source storage subunit corresponding to a 2; furthermore, if the identifier of the source device of the time series data to be accessed corresponds to the source device characterized by g, the access operation of the time series data to be accessed may be further located in the source device storage sub-module corresponding to g, so that the access operation of the time series data to be accessed is performed in the source device storage sub-module corresponding to g in the data source storage sub-unit corresponding to a2 in the time zone storage unit corresponding to a. If the access operation is a write operation, the data to be accessed can be written into g, and if the access operation is an inquiry operation, the relevant information of the data to be accessed can be inquired in g.
By the above manner, the source device type storage subunit is further divided, after a time zone storage unit (i.e. a target time zone storage unit) to which an access operation of the time sequence data to be accessed is directed is determined, according to the source device type information in the data source attribute information and the source device type tag corresponding to each source device type storage subunit in the target time zone storage unit, a source device type storage subunit to which the access operation of the time sequence data is directed is further determined, and then according to the source device identifier in the data source attribute information and the source device tag of each source device storage submodule in the source device type storage subunit, a source device storage submodule to which the access operation of the time sequence data is directed is further determined, so that the root position of the access operation of the data to be accessed is gradually determined through information of different attributes of the time sequence data, further improving the efficiency of the access operation.
In a possible manner, the data type of the time series data considering the same device is different, and for example, the data may be temperature data, CPU occupancy rate, or the like. Therefore, in order to further improve the efficiency of data access operations in a large amount of data scenarios, the source device storage submodule further includes at least one data type storage submodule, and the data type storage submodule is provided with a corresponding data type tag.
It should be understood that one data type storage submodule stores time series data of the same data type. The data type label represents the data type of the time sequence data stored by the data type storage submodule corresponding to the data type label.
FIG. 3 is a schematic diagram illustrating a storage structure of a time zone storage unit according to an exemplary embodiment of the present disclosure. FIG. 3 is a further explanation of one structure of the A time zone storage unit shown in FIG. 2, with reference to FIG. 3, A denoting the time zone storage unit, a1, a2, a3 and a4 being source device type labels indicating that the different data source storage subunits, i.e., the device types represented by a1, a2, a3 and a4, are different; e. f, g and h are source equipment labels and represent different source equipment storage submodules, namely e, f, g and h represent unique equipment; 1, 2, 3, 4, 5 and 6 under each source device storage submodule are different data type tags, which indicate different data type storage submodules, that is, each data type storage submodule stores time sequence data of the same data type.
Under the structure shown in fig. 3, the performing, in the target source device storage submodule, an access operation on the to-be-accessed time series data may include: acquiring data type information of data to be accessed; determining a target data type storage submodule according to the data type information and a data type label corresponding to each data type storage submodule in the target source equipment storage submodule; and performing access operation on the time sequence data to be accessed in the target data type storage submodule.
Through the method, in the target source equipment storage submodule, the target data type storage submodule of the time sequence data to be accessed is further determined according to the data type information of the data to be accessed and the data type label corresponding to each data type storage submodule in the target source equipment storage submodule, so that the access operation can be conveniently carried out in the data type storage submodule storing the data of the same type, and the efficiency of the access operation is further improved.
It should be noted that, in the present disclosure, each time zone storage list may maintain the source device type storage subunit corresponding to different source device type tags through a two-dimensional table; each source equipment type storage subunit can maintain source equipment storage submodules corresponding to different source equipment labels through a two-dimensional table; each source device storage submodule can maintain the data type storage submodule corresponding to different data type labels through a two-dimensional table.
In a possible manner, when the access operation is a write operation, the performing, in the target data type storage submodule, an access operation on the sequential data to be accessed may include: under the condition that the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data strip by the target data type storage submodule last time, updating the ending time point of the data strip and the counting frequency information of the data strip according to the time attribute information of the time sequence data to be accessed, wherein the starting time point of the data strip is set according to the starting time point of the time sequence data corresponding to the initially written index value of each data strip, and the counting frequency information is used for representing the number of the index values recorded by the corresponding data strip.
In the present disclosure, the time series data includes an index value. For example, when the time series data is the CPU occupancy, the index value may be a numerical size of the occupancy, such as 20%, or the like.
It should be noted that the data type storage submodule can maintain the time series data through a two-dimensional table. The two-dimensional table may be as shown in table 2:
| starting point in time | Point of time of termination | Index value | Statistics of frequency information |
| 2021-1-109:00:00 | 2021-1-109:15:00 | 2 | 4 |
| 2021-1-109:20:00 | 2021-1-109:45:00 | 3 | 6 |
TABLE 2
As shown in table 2, table 2 includes two data bars, and table 2 collects and stores time series data every 5 minutes. In the first data record in table 2, the index values of the time series data acquired four times consecutively in the period from 2021-1-109:00:00 to 2021-1-109:15:00 are all 2.
Taking table 2 as an example, when the index value of the time series data acquired from 2021-1-109:50:00 is 3, since the value is the same as the index value of the time series data written in the data strip last time in the table (i.e. the data strip with the start time point of 2021-1-100009:20: 00), the end time point and the statistical frequency information of the data strip only need to be updated at this time, that is, the end time point is updated to 2021-1-109:50:00, and the statistical frequency information is updated to 7, so that the time series data of 2021-1-109:50:00 can be reflected in the data strip, and if the time series data in the data type storage submodule needs to be analyzed subsequently, the original data of each acquisition time point can be recovered according to the statistical frequency information and the acquisition interval time.
By the mode, continuous and same time sequence data are compressed and stored, and a plurality of continuous and same time sequence data are represented by one data record, so that the number of data strips in the data type storage submodule is reduced, the purpose of compressing and storing the data is achieved, and the memory occupied by the time sequence data on the storage device is further reduced.
In a possible mode, the time sequence data stored in the data type storage submodule at least comprises index values of time sequence of two time points. For example, table 2 above may be of the form of table 3:
| starting point in time | Point of time of | Index value | 1 | |
Statistics of frequency information |
| 2021-1-109:00:00 | 2021-1-109:05:00 | 2 | 2 | 2 | |
| 2021-1-109:15:00 | 2021-1-109:20:00 | 4 | 5 | 2 |
TABLE 3
In table 3, the index value 1 is an index value of time-series data corresponding to the start time point, and the index value 2 is an index value of time-series data corresponding to the end time point. It should be understood that the index values of the time series data corresponding to the acquisition time between the start time point and the end time point may be recorded in the table.
Under the condition that at least two index values are recorded in one data strip, under the condition that the index value in the time sequence data to be accessed is different from the index value in the time sequence data written in the data strip last time by the target data type storage submodule, determining a first slope according to the index value of the time sequence data to be accessed and any index value in the data strip; when the difference between the first slope and the second slope between any two index values stored in the data strip is larger than a preset threshold value, generating a new data strip, setting the starting time point and the ending time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index values in the time sequence data to be accessed into the data strip, and setting the statistical frequency information of the new data strip.
It should be noted that, the larger the difference between the first slope and the second slope, the larger the difference between the trend of the time series data added to the data strip representing the time series data to be accessed and the trend of the time series data in the data strip; conversely, the smaller the difference between the first slope and the second slope, the smaller the difference between the trend of the time series data added to the data strip, which represents the time series data to be accessed, and the trend of the time series data in the data strip.
For example, the preset threshold may be set according to actual conditions.
Taking table 3 as an example, if the index value of the time series data to be accessed in 2021-1-109:25:00 is 8, in the data strip of 2021-1-109:15: 00-2021-109: 20:00, the second slope of the two index values is: (index value 2-index value 1)/acquisition time interval, and calculating to obtain a second slope (5-4)/5-0.2; the first slope may be: (index value 2-index value of time series data to be accessed)/acquisition time interval, calculating to obtain that the first slope is (8-4)/5 is 0.8, and because the difference between the first slope and the second slope is larger (the first slope is characterized by rising in a small trend, and the second slope is characterized by rising in a large trend), a new data strip needs to be generated in table 3 to record the time series data, and the starting time point and the ending time point of the newly generated data strip are 2021-1-109:25:00, then table 3 is transformed as shown in table 4:
| starting point in time | Point of time of | Index value | 1 | |
Statistics of frequency information |
| 2021-1-109:00:00 | 2021-1-109:05:00 | 2 | 2 | 2 | |
| 2021-1-109:15:00 | 2021-1-109:20:00 | 4 | 5 | 2 | |
| 2021-1-109:25:00 | 2021-1-109:25:00 | 8 | 1 |
TABLE 4
When the difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to a preset threshold value, in a possible mode, updating the termination time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence data to be accessed into the data strip, and updating the statistical frequency information of the data strip. For example, if the index value of the time series data to be accessed of 2021-1-109:25:00 is 6, in the data strip of 2021-1-109:15: 00-2021-109: 20:00, the second slope of the two index values is: (index value 2-index value 1)/acquisition time interval, and calculating to obtain a second slope (5-4)/5-0.2; the first slope may be: the first slope of (6-5)/5 is calculated to be 0.2 (index value-index value 2)/collection time interval of the time series data to be accessed), and since the difference between the first slope and the second slope is zero, table 3 is updated in the above manner, and then table 5 can be obtained:
TABLE 5
By the method, under the condition that continuous time sequence data have similar trends, the time sequence data are recorded by using one data strip.
When the difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to a preset threshold value, and if the last written index value of the data strip is not the first index value written in the data strip, the last written index value of the data strip can be updated according to the index values in the time sequence to be accessed, the termination time point of the data strip can be updated according to the time attribute information of the time sequence data to be accessed, and the statistical frequency information of the data strip can be updated. If the index value of the time-series data to be accessed is still 6 in 2021-1-109:25:00, table 3 is updated in the above manner, and table 6 can be obtained:
| starting point in time | Point of time of | Index value | 1 | |
Statistics of frequency information |
| 2021-1-109:00:00 | 2021-1-109:05:00 | 2 | 2 | 2 | |
| 2021-1-109:15:00 | 2021-1-109:25:00 | 4 | 6 | 3 |
TABLE 6
By the method, when the continuous time series data have similar trends, the continuous time series data with the similar trends are reflected by one data strip in a lossy compression mode, namely the recording number of index values is reduced, the memory occupied by the data is further reduced, the recording mode is favorable for recovering all original data, and the subsequent data analysis of the time series data is not influenced.
Based on the same inventive concept, an embodiment of the present disclosure further provides a time series data access apparatus, which is applied to an electronic device, where the electronic device includes a storage device, the storage device includes a plurality of time zone storage units, each time zone storage unit includes at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag. Fig. 4 is a block diagram illustrating a time series data access apparatus according to an exemplary embodiment of the present disclosure, and referring to fig. 4, the time series data access apparatus 400 includes:
an obtaining module 401, configured to obtain time series data to be accessed, where the time series data to be accessed includes attribute information of the time series data to be accessed, and the attribute information includes time attribute information and data source attribute information;
an accessing module 402, configured to determine a target time zone storage unit corresponding to the to-be-accessed time sequence data according to the time attribute information of the to-be-accessed time sequence data and the time zone tag corresponding to each time zone storage unit, determine a target data source storage subunit corresponding to the to-be-accessed time sequence data according to the data source attribute information and the data source tag corresponding to each data source storage subunit in the target time zone storage unit, and perform an accessing operation on the to-be-accessed time sequence data in the target data source storage subunit.
Optionally, the data source storage subunit is a source device type storage subunit, the data source tag is a source device type tag, each source device type storage subunit includes at least one source device storage submodule, and the source device storage submodule is provided with a corresponding source device tag;
the access module 402 comprises:
a first access sub-module, configured to determine, according to source device type information in the data source attribute information and a source device type tag corresponding to each source device type storage sub-unit in the target time zone storage unit, a target source device type storage sub-unit corresponding to the to-be-accessed time series data;
and the second accessing submodule is used for determining a target source device storage submodule according to a source device identifier in the data source attribute information and a source device label of each source device storage submodule in the target source device type storage submodule, and performing access operation on the time sequence data to be accessed in the target source device storage submodule.
Optionally, the source device storage submodule further includes at least one data type storage submodule, and the data type storage submodule is provided with a corresponding data type tag;
the access module 402 further comprises:
the acquisition submodule is used for acquiring the data type information of the data to be accessed;
the third access submodule determines a target data type storage submodule according to the data type information and a data type label corresponding to each data type storage submodule in the target source equipment storage submodule;
and the fourth access submodule is used for performing access operation on the time sequence data to be accessed in the target data type storage submodule.
Optionally, the time series data includes an index value, the access operation is a write operation, and the fourth access sub-module is specifically configured to, when the index value in the time series data to be accessed is the same as the index value in the time series data written in the data bar last time by the target data type storage sub-module, update the end time point of the data bar and the statistical frequency information of the data bar according to the time attribute information of the time series data to be accessed, where each data bar is provided with a start time point of the data bar according to the start time point of the time series data corresponding to the initially written index value, and the statistical frequency information is used to represent the number of index values recorded by the corresponding data bar.
Optionally, the apparatus 400 further includes a slope determining module, configured to determine a first slope according to the index value of the to-be-accessed time series data and any index value in a data strip when the index value in the to-be-accessed time series data is not the same as the index value in the time series data written by the target data type storage submodule last time in the data strip;
and the data adding module is used for generating a new data strip when the difference between the first slope and a second slope between any two index values stored in the data strip is greater than a preset threshold value, setting a starting time point and an ending time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index values in the time sequence data to be accessed into the data strip, and setting the statistical frequency information of the new data strip.
Optionally, the apparatus 400 further includes a first data updating module, configured to update a termination time point of the data strip according to the time attribute information of the time sequence data to be accessed when a difference between the first slope and a second slope between any two index values stored in the data strip is less than or equal to the preset threshold, write the index value in the time sequence to be accessed into the data strip, and update the statistics information of the data strip;
and a second data updating module, configured to, when a difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to the preset threshold, update the last written index value of the data strip according to the index value in the to-be-accessed time sequence if the last written index value of the data strip is not the first index value written in the data strip, update the termination time point of the data strip according to the time attribute information of the to-be-accessed time sequence data, and update the statistical frequency information of the data strip.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Based on the same inventive concept, the embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the time series data access method according to any one of the method embodiments.
Based on the same inventive concept, an embodiment of the present disclosure further provides an electronic device, including:
a storage device having a computer program stored thereon;
a processor for executing the computer program in the memory device to perform the steps of the method of sequential data access of any of the method implementation examples.
Fig. 5 is a block diagram illustrating an electronic device 500 in accordance with an example embodiment. As shown in fig. 5, the electronic device 500 may include: a processor 501 and a storage device 502. The electronic device 500 may also include one or more of a multimedia component 503, an input/output (I/O) interface 504, and a communication component 505.
The processor 501 is configured to control the overall operation of the electronic device 500, so as to complete all or part of the steps in the above-mentioned time-series data access method. The storage device 502 is used to store various types of data to support operation at the electronic device 500, such as instructions for any application or method operating on the electronic device 500 and application-related data, such as contact data, messaging, pictures, audio, video, and so forth. The Memory device 502 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 503 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may further be stored in the storage device 502 or transmitted through the communication component 505. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 504 provides an interface between the processor 501 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so that the corresponding Communication component 505 may include: Wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic Device 500 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described sequential data access method.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described method of sequential data access is also provided. For example, the computer readable storage medium may be the storage device 502 described above that includes program instructions that are executable by the processor 501 of the electronic device 500 to perform the above-described sequential data access method.
The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, various possible combinations will not be separately described in this disclosure.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.
Claims (10)
1. A time sequence data access method is applied to electronic equipment, the electronic equipment comprises storage equipment, the storage equipment comprises a plurality of time zone storage units, each time zone storage unit comprises at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone label, and the data source storage subunit is provided with a corresponding data source label, and the method comprises the following steps:
acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
and determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
2. The method according to claim 1, wherein the data source storage subunit is a source device type storage subunit, the data source tags are source device type tags, each source device type storage subunit includes at least one source device storage submodule, and the source device storage submodule is provided with a corresponding source device tag;
determining a target data source storage subunit corresponding to the to-be-accessed time sequence data according to the data source attribute information and the data source tag corresponding to each data source storage subunit in the target time zone storage unit, and performing an access operation on the to-be-accessed time sequence data in the target data source storage subunit, where the determining includes:
determining a target source device type storage subunit corresponding to the time sequence data to be accessed according to source device type information in the data source attribute information and a source device type label corresponding to each source device type storage subunit in the target time zone storage unit;
and determining a target source device storage submodule according to a source device identifier in the data source attribute information and a source device label of each source device storage submodule in the target source device type storage submodule, and performing access operation on the time sequence data to be accessed in the target source device storage submodule.
3. The method of claim 2, wherein the source device storage submodule further comprises at least one data type storage submodule, the data type storage submodule being provided with a corresponding data type tag;
the accessing operation of the time sequence data to be accessed in the storage submodule of the target source device comprises:
acquiring data type information of the data to be accessed;
determining a target data type storage submodule according to the data type information and a data type label corresponding to each data type storage submodule in the target source equipment storage submodule;
and performing access operation on the time sequence data to be accessed in the target data type storage submodule.
4. The method according to claim 3, wherein the time series data includes an index value, the access operation is a write operation, and the performing the access operation on the time series data to be accessed in the target data type storage submodule includes:
and under the condition that the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data strip by the target data type storage submodule last time, updating the termination time point of the data strip and the statistical frequency information of the data strip according to the time attribute information of the time sequence data to be accessed, wherein the starting time point of the data strip is set for each data strip according to the starting time point of the time sequence data corresponding to the initially written index value, and the statistical frequency information is used for representing the number of the index values recorded by the corresponding data strip.
5. The method of claim 4, further comprising:
under the condition that the index value in the time sequence data to be accessed is different from the index value in the time sequence data written in the data strip last time by the target data type storage submodule, determining a first slope according to the index value of the time sequence data to be accessed and any index value in the data strip;
when the difference between the first slope and a second slope between any two index values stored in the data strip is larger than a preset threshold value, generating a new data strip, setting a starting time point and an ending time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index values in the time sequence data to be accessed into the data strip, and setting the statistical frequency information of the new data strip.
6. The method of claim 5, further comprising:
when the difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to the preset threshold, updating the termination time point of the data strip according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence to be accessed into the data strip, and updating the statistical frequency information of the data strip; alternatively, the first and second electrodes may be,
when the difference between the first slope and a second slope between any two index values stored in the data strip is smaller than or equal to the preset threshold, if the last written index value of the data strip is not the first index value written in the data strip, updating the last written index value of the data strip according to the index value in the time sequence to be accessed, updating the termination time point of the data strip according to the time attribute information of the time sequence data to be accessed, and updating the statistical frequency information of the data strip.
7. A time sequence data access device is applied to electronic equipment, the electronic equipment comprises a storage device, the storage device comprises a plurality of time zone storage units, each time zone storage unit comprises at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone label, and the data source storage subunit is provided with a corresponding data source label, the device comprises:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring time sequence data to be accessed, the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
and the access module is used for determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
8. The apparatus of claim 7, wherein the data source storage subunit is a source device type storage subunit, the data source tag is a source device type tag, each source device type storage subunit includes at least one source device storage submodule, and the source device storage submodule is provided with a corresponding source device tag;
the access module includes:
a first access sub-module, configured to determine, according to source device type information in the data source attribute information and a source device type tag corresponding to each source device type storage sub-unit in the target time zone storage unit, a target source device type storage sub-unit corresponding to the to-be-accessed time series data;
and the second accessing submodule is used for determining a target source device storage submodule according to a source device identifier in the data source attribute information and a source device label of each source device storage submodule in the target source device type storage submodule, and performing access operation on the time sequence data to be accessed in the target source device storage submodule.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
10. An electronic device, comprising:
a storage device having a computer program stored thereon;
a processor for executing the computer program in the storage device to carry out the steps of the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110349472.8A CN113177143B (en) | 2021-03-31 | 2021-03-31 | Time sequence data access method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110349472.8A CN113177143B (en) | 2021-03-31 | 2021-03-31 | Time sequence data access method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113177143A true CN113177143A (en) | 2021-07-27 |
| CN113177143B CN113177143B (en) | 2023-10-27 |
Family
ID=76922900
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110349472.8A Active CN113177143B (en) | 2021-03-31 | 2021-03-31 | Time sequence data access method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113177143B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070300019A1 (en) * | 2006-06-27 | 2007-12-27 | Fujitsu Limited | Memory access apparatus, memory access method and memory manufacturing method |
| US20100199060A1 (en) * | 2009-02-04 | 2010-08-05 | Panasonic Corporation | Memory controller, nonvolatile memory module, access module, and nonvolatile memory system |
| WO2017080454A1 (en) * | 2015-11-12 | 2017-05-18 | 北京国双科技有限公司 | Website access path aggregation method and device |
| WO2018119587A1 (en) * | 2016-12-26 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Data processing method, device, and system, and information acquisition apparatus |
| CN109271277A (en) * | 2017-07-17 | 2019-01-25 | 阿里巴巴集团控股有限公司 | Access method, device and system after database delay machine |
| CN111090879A (en) * | 2019-12-05 | 2020-05-01 | 达闼科技成都有限公司 | Data processing method, device, readable storage medium, electronic equipment and system |
| CN111638968A (en) * | 2019-03-01 | 2020-09-08 | 维塔科技(北京)有限公司 | Shared memory access permission setting method and device, electronic equipment and storage medium |
-
2021
- 2021-03-31 CN CN202110349472.8A patent/CN113177143B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070300019A1 (en) * | 2006-06-27 | 2007-12-27 | Fujitsu Limited | Memory access apparatus, memory access method and memory manufacturing method |
| US20100199060A1 (en) * | 2009-02-04 | 2010-08-05 | Panasonic Corporation | Memory controller, nonvolatile memory module, access module, and nonvolatile memory system |
| WO2017080454A1 (en) * | 2015-11-12 | 2017-05-18 | 北京国双科技有限公司 | Website access path aggregation method and device |
| WO2018119587A1 (en) * | 2016-12-26 | 2018-07-05 | 深圳前海达闼云端智能科技有限公司 | Data processing method, device, and system, and information acquisition apparatus |
| CN109271277A (en) * | 2017-07-17 | 2019-01-25 | 阿里巴巴集团控股有限公司 | Access method, device and system after database delay machine |
| CN111638968A (en) * | 2019-03-01 | 2020-09-08 | 维塔科技(北京)有限公司 | Shared memory access permission setting method and device, electronic equipment and storage medium |
| CN111090879A (en) * | 2019-12-05 | 2020-05-01 | 达闼科技成都有限公司 | Data processing method, device, readable storage medium, electronic equipment and system |
Non-Patent Citations (2)
| Title |
|---|
| 于楠;胡金炎;邹岸;王涛;林霖;邱健盛;: "一种母婴安全监护系统数据库的设计和实现", 计算机应用与软件, no. 05 * |
| 韩艺坤;李富年;陈志丹;: "基于InfluxDB的桥梁监测系统设计与实现", 现代电子技术, no. 16 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113177143B (en) | 2023-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108415952B (en) | User data storage method, label calculation method and calculation equipment | |
| CN111046034B (en) | Method and system for managing memory data and maintaining data in memory | |
| US11636083B2 (en) | Data processing method and apparatus, storage medium and electronic device | |
| CN111061758B (en) | Data storage method, device and storage medium | |
| CN108875077B (en) | Column storage method and device of database, server and storage medium | |
| CN108932286B (en) | Data query method and device | |
| CN110659282B (en) | Data route construction method, device, computer equipment and storage medium | |
| CN113704243A (en) | Data analysis method, data analysis device, computer device, and storage medium | |
| CN111143422B (en) | Data retrieval method, data retrieval device, storage medium and electronic equipment | |
| CN111291037A (en) | Data storage and query method, device, equipment and computer storage medium | |
| US9213759B2 (en) | System, apparatus, and method for executing a query including boolean and conditional expressions | |
| CN111966747A (en) | Data synchronization method, system, terminal device and storage medium | |
| TWI769665B (en) | Target data updating method, electronic equipment and computer readable storage medium | |
| CN108920516B (en) | Real-time analysis method, system, device and computer readable storage medium | |
| CN114548059A (en) | Method and device for managing structured data, storage medium and electronic equipment | |
| CN114020713A (en) | File merging method and device of log structure merging tree, electronic equipment and storage medium | |
| CN113177143B (en) | Time sequence data access method and device, storage medium and electronic equipment | |
| CN111639087A (en) | Data updating method and device in database and electronic equipment | |
| CN115858471A (en) | Service data change recording method, device, computer equipment and medium | |
| CN114356912A (en) | Method for writing data into database and computer equipment | |
| CN110866003B (en) | Index value number estimation method and device and electronic equipment | |
| CN109933580B (en) | Training data generation method and device and server | |
| CN108984720B (en) | Data query method and device based on column storage, server and storage medium | |
| CN108665221B (en) | Performance data processing method and device, computer equipment and readable medium | |
| CN111782588A (en) | File reading method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |