CN113177143B - Time sequence data access method and device, storage medium and electronic equipment - Google Patents

Time sequence data access method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN113177143B
CN113177143B CN202110349472.8A CN202110349472A CN113177143B CN 113177143 B CN113177143 B CN 113177143B CN 202110349472 A CN202110349472 A CN 202110349472A CN 113177143 B CN113177143 B CN 113177143B
Authority
CN
China
Prior art keywords
data
time
source
accessed
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110349472.8A
Other languages
Chinese (zh)
Other versions
CN113177143A (en
Inventor
马云存
闻英友
王晨
窦丽莉
吕昕东
葛东
刘建生
徐赛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Neusoft Cloud Technology Co ltd
Neusoft Corp
Original Assignee
Neusoft Cloud Technology Co ltd
Neusoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Neusoft Cloud Technology Co ltd, Neusoft Corp filed Critical Neusoft Cloud Technology Co ltd
Priority to CN202110349472.8A priority Critical patent/CN113177143B/en
Publication of CN113177143A publication Critical patent/CN113177143A/en
Application granted granted Critical
Publication of CN113177143B publication Critical patent/CN113177143B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/901Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/906Clustering; Classification

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The disclosure relates to a time sequence data access method, a time sequence data access device, a storage medium and electronic equipment, wherein the method comprises the following steps: acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information; determining a target time zone storage unit corresponding to the time sequence data to be accessed according to time attribute information of the time sequence data to be accessed and time zone labels corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to data source attribute information and data source labels corresponding to each data source storage subunit in the target time zone storage unit, performing access operation on the time sequence data to be accessed in the target data source storage subunit, and reducing the area of the access operation according to the time attribute information and the data source attribute information of the time sequence data to be accessed, so that the efficiency of the access operation is improved.

Description

Time sequence data access method and device, storage medium and electronic equipment
Technical Field
The disclosure relates to the technical field of databases, and in particular relates to a time sequence data access method, a time sequence data access device, a storage medium and electronic equipment.
Background
In the related art, a conventional two-dimensional storage method is used to store data. The two-dimensional storage method refers to performing planarization storage in a line record manner. For the data of the management class, the conventional two-dimensional storage method can meet the requirement because the data of the management class does not need too much additional processing. However, for time series data collected from various devices at regular time by a collector or a sensor, because the time series data has the characteristics of time series, large data volume, frequent writing and random reading, the storage and the inquiry of the time series data need additional processing, and the data storage planarization of the traditional two-dimensional storage method has serious influence on the data reading efficiency under the condition of large data volume.
Disclosure of Invention
The invention aims to provide a time sequence data access method, a time sequence data access device, a storage medium and electronic equipment, wherein time attribute information and data source attribute information of time sequence data to be accessed are directly positioned to a target data source storage subunit corresponding to an access operation, so that the efficiency of the access operation is improved.
To achieve the above object, in a first aspect, the present disclosure provides a time-series data access method, applied to an electronic device, where the electronic device includes a storage device, the storage device includes a plurality of time zone storage units, each of the time zone storage units includes at least one data source storage subunit, each of the time zone storage units is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag, and the method includes:
Acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
Optionally, the data source storage subunit is a source equipment type storage subunit, the data source tag is a source equipment type tag, each source equipment type storage subunit includes at least one source equipment storage subunit, and the source equipment storage subunit is provided with a corresponding source equipment tag;
the determining the target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing the access operation on the time sequence data to be accessed in the target data source storage subunit includes:
Determining a target source equipment type storage subunit corresponding to the time sequence data to be accessed according to source equipment type information in the data source attribute information and source equipment type labels corresponding to each source equipment type storage subunit in the target time zone storage unit;
and determining a target source equipment storage sub-module according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the target source equipment type storage sub-unit, and performing access operation on the time sequence data to be accessed in the target source equipment storage sub-module.
Optionally, the source device storage sub-module further includes at least one data type storage sub-module, where the data type storage sub-module is provided with a corresponding data type tag;
the accessing operation of the time sequence data to be accessed in the storage sub-module of the target source equipment comprises the following steps:
acquiring data type information of the data to be accessed;
determining a target data type storage sub-module according to the data type information and the data type label corresponding to each data type storage sub-module in the target source equipment storage sub-module;
And performing access operation on the time sequence data to be accessed in the target data type storage sub-module.
Optionally, the time sequence data includes an index value, the access operation is a write operation, and the accessing operation on the time sequence data to be accessed in the target data type storage sub-module includes:
and under the condition that the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data strip last time by the target data type storage submodule, updating the ending time point of the data strip and the statistics times information of the data strip according to the time attribute information of the time sequence data to be accessed, wherein each data strip is provided with the starting time point of the data strip according to the starting time point of the time sequence data corresponding to the index value which is initially written, and the statistics times information is used for representing the number of the index values recorded by the corresponding data strip.
Optionally, the method further comprises:
determining a first slope according to the index value of the time sequence data to be accessed and any index value in the data strip under the condition that the index value in the time sequence data to be accessed is different from the index value in the time sequence data written in the data strip last time by the target data type storage sub-module;
When the difference between the first slope and the second slope between any two index values stored in the data bar is larger than a preset threshold value, generating a new data bar, setting a starting time point and a stopping time point of the data bar according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence data to be accessed into the data bar, and setting statistical number information of the new data bar.
Optionally, the method further comprises:
when the difference between the first slope and the second slope between any two index values stored in the data bar is smaller than or equal to the preset threshold value, updating the termination time point of the data bar according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence number to be accessed into the data bar, and updating the statistics number information of the data bar; or alternatively, the process may be performed,
when the difference between the first slope and the second slope between any two index values stored in the data bar is smaller than or equal to the preset threshold, if the index value written by the data bar last time is not the index value written by the data bar first time, updating the index value written by the data bar last time according to the index value in the time sequence number to be accessed, updating the termination time point of the data bar according to the time attribute information of the time sequence data to be accessed, and updating the statistics number information of the data bar.
In a second aspect, the present disclosure provides a time-series data access apparatus applied to an electronic device, the electronic device including a storage device, the storage device including a plurality of time zone storage units, each of the time zone storage units including at least one data source storage subunit, each of the time zone storage units being provided with a corresponding time zone tag, the data source storage subunit being provided with a corresponding data source tag, the apparatus comprising:
the acquisition module is used for acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
the access module is used for determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
Optionally, the data source storage subunit is a source equipment type storage subunit, the data source tag is a source equipment type tag, each source equipment type storage subunit includes at least one source equipment storage subunit, and the source equipment storage subunit is provided with a corresponding source equipment tag;
the access module comprises:
the first access sub-module is used for determining a target source equipment type storage sub-unit corresponding to the time sequence data to be accessed according to source equipment type information in the data source attribute information and source equipment type labels corresponding to each source equipment type storage sub-unit in the target time zone storage unit;
and the second access sub-module is used for determining a target source equipment storage sub-module according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the target source equipment type storage sub-unit, and performing access operation on the time sequence data to be accessed in the target source equipment storage sub-module.
In a third aspect, the present disclosure provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of the first aspects above.
In a fourth aspect, the present disclosure provides an electronic device comprising:
a storage device having a computer program stored thereon;
a processor for executing the computer program in the storage device to implement the steps of the method of any of the first aspects above.
According to the technical scheme, the target time zone storage unit for the access operation is locked according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit; according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, the target data source storage subunit aimed by the access operation is locked, that is, the area of the access operation is reduced through the time attribute information and the data source attribute information of the time sequence data to be accessed, and the area is positioned to the root position corresponding to the access operation, so that the efficiency of the access operation can be improved.
Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
Fig. 1 is a flowchart illustrating a method of sequential data access according to an exemplary embodiment of the present disclosure.
Fig. 2 is a schematic diagram of a storage structure of a storage device according to an exemplary embodiment of the present disclosure.
Fig. 3 is a schematic diagram showing a storage structure of a time zone storage unit according to an exemplary embodiment of the present disclosure.
Fig. 4 is a block diagram of a sequential data access device according to an exemplary embodiment of the present disclosure.
Fig. 5 is a block diagram of an electronic device, according to an exemplary embodiment of the present disclosure.
Detailed Description
Specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the disclosure, are not intended to limit the disclosure.
In the related art, it is generally required to record each index of the device at each point in time, so as to analyze the operation state of the device according to each index continuously. As described in the background art, table 1 is a two-dimensional database table corresponding to a two-dimensional storage manner:
TABLE 1
In table 1, only a small amount of time series data is listed, but since the amount of time series data is large, table 1 stores time series data with a large number of line records. When a certain time series data is queried, the row records in the table 1 need to be continuously traversed, and the query efficiency is seriously affected under the condition of large time series data quantity.
In view of the above, the present disclosure provides a method, an apparatus, a storage medium and an electronic device for accessing time series data, which store the time series data on a new storage structure to improve the efficiency of the access operation.
Fig. 1 is a flowchart illustrating a method of sequential data access according to an exemplary embodiment of the present disclosure. Referring to fig. 1, the time series data access method may include the steps of:
step 101, obtaining time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information.
Step 102, determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit.
Step 103, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit.
And 104, performing access operation on the time sequence data to be accessed in the target data source storage subunit.
By way of example, the timing data in the present disclosure may be timing data obtained by monitoring a device in the field of automated monitoring operation and maintenance. The device may be, for example, a host, a display screen, etc.
The time sequence data access method in the disclosure can be applied to electronic equipment, the electronic equipment comprises storage equipment, the storage equipment comprises a plurality of time zone storage units, each time zone storage unit comprises at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone label, and each data source storage subunit is provided with a corresponding data source label.
The time zone tag characterizes a time range to which acquisition time of the time sequence data belongs. In a possible manner, the time zone tag can characterize time data within 7 days, as well as time series data within 1 month. For example, the time zone tag may represent a time range 2021, year 1 month 1 day 00:00:00 to month 1 year 7 day 24:00:00. A corresponding one of the two sets the data source storage subunit under the time zone storage unit corresponding to the time zone label is stored time series data in the range of 2021, 1 month, 1 day, 00:00, 2021, 1 month, 7 days, 24:00:00 are acquired.
The data source tag characterizes the source of the time series data. It should be appreciated that different source tags of time series data represent different sources of time series data. The source of the time series data may be from a certain type of device, or may be from a certain type of device, and in particular, a certain IP address. In a possible manner, in the case that the data source tag characterizes the source device type, the data source tag may characterize a host type tag, or may characterize a display screen type tag; in the case where the data origin tag characterizes a certain type of device and is an origin device with a certain IP address, the data origin tag may characterize a device tag with an IP address of 192.168.100.10, or may characterize a device tag with an IP address of 192.168.100.11.
For example, the time attribute information of the time series data to be accessed is time information of collecting the time series data to be accessed. For example, if the time series data collected by the host is integrated at 9 points of 1.1.1.2021, the time attribute information of the time series data is 9:00:00 of 1.1.1.1.1.
For example, the data source attribute information of the time-series data to be accessed may be a source device type, or may be a source device which belongs to a certain device type and is a certain IP address.
The access operation may be a write operation or a query operation, for example.
By the mode, the target time zone storage unit for the access operation of the time sequence data to be accessed is locked according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit by utilizing the storage structure characteristics of the storage equipment; according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, the target data source storage subunit aimed at by the access operation of the time sequence data to be accessed is locked, that is, the area of the access operation is reduced through the time attribute information and the data source attribute information of the time sequence data to be accessed, and the area is positioned to the root position corresponding to the access operation, so that the efficiency of the access operation can be improved.
It should be noted that in the present disclosure, the time zone storage units corresponding to different time zone tags may be maintained in the storage device through a two-dimensional table; each time zone storage unit can maintain the data source storage sub-units corresponding to different data source labels through a two-dimensional table.
In a possible manner, the data source storage subunit is a source device type storage subunit, the data source tag is a source device type tag, and each source device type storage subunit includes at least one source device storage subunit, and the source device storage subunit is provided with a corresponding source device tag.
For example, the source device type tag may be a host type tag, and accordingly, only the time sequence data of the host itself, such as a host CPU, a host temperature, and the like, is stored in the source device storage sub-module under the source device type storage sub-unit corresponding to the source device type tag.
In a possible manner, step 103 shown in fig. 1 may include: determining a target source equipment type storage subunit corresponding to the time sequence data to be accessed according to the source equipment type information in the data source attribute information and the source equipment type label corresponding to each source equipment type storage subunit in the target time zone storage unit; and determining a target source equipment storage sub-module according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the target source equipment type storage sub-unit. Wherein the source device identification may be an IP identification characterizing the unique device.
Step 104 shown in fig. 1 may include: and performing access operation on the time sequence data to be accessed in the storage submodule of the target source equipment.
Illustratively, fig. 2 is a schematic diagram of a memory structure of a memory device according to an exemplary embodiment of the present disclosure. Referring to fig. 2, a, B, C, and D are time zone tags indicating that the time ranges represented by the different time zone storage units, namely A, B, C and D, are different; a1, a2, a3 and a4 are source device type labels, which represent that different data source storage subunits, i.e. the device types represented by a1, a2, a3 and a4 are different; e. f, g and h are source device tags representing different source device storage sub-modules, i.e., e, f, g and h represent unique devices. For example, if the time attribute information of the time series data to be accessed corresponds to the time range represented by a, the access operation of the time series data to be accessed may be located in the time zone storage unit corresponding to a; further, the source device type information of the time sequence data to be accessed corresponds to the device type represented by a2, and then the access operation of the time sequence data to be accessed can be further positioned to the data source storage subunit corresponding to a 2; further, the source device identifier of the time-series data to be accessed corresponds to the source device represented by g, so that the source device storage sub-module corresponding to g can be further located in the access operation of the time-series data to be accessed, and therefore, the access operation of the time-series data to be accessed is performed in the source device storage sub-module corresponding to g under the data source storage sub-unit corresponding to a2 under the time zone storage unit corresponding to a. If the access operation is a write operation, the data to be accessed can be written into g, and if the query operation is a query operation, the related information of the data to be accessed can be queried in g.
According to the method, the source equipment type storage sub-units are further divided, after the time zone storage unit (namely the target time zone storage unit) aimed by the access operation of the time sequence data to be accessed is determined, the source equipment type storage sub-units aimed by the access operation of the time sequence data are further determined according to the source equipment type information in the data source attribute information and the source equipment type label corresponding to each source equipment type storage sub-unit in the target time zone storage unit, the source equipment type storage sub-units aimed by the access operation of the time sequence data are further determined according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the source equipment type storage sub-units, and the source equipment storage sub-modules aimed by the access operation of the time sequence data are further determined.
In a possible manner, the time series data of the same device may be considered to be different in data type, for example, temperature data, CPU occupancy rate, and the like. Therefore, in order to further improve the efficiency of data access operations in a large number of data scenarios, the source device storage sub-module further comprises at least one data type storage sub-module, which is provided with a corresponding data type tag.
It should be understood that stored under one data type storage sub-module is time series data of the same data type. The data type tag characterizes the data type of the time sequence data stored by the data type storage submodule corresponding to the data type tag.
Fig. 3 is a schematic diagram showing a storage structure of a time zone storage unit according to an exemplary embodiment of the present disclosure. FIG. 3 is a further illustration of one configuration of the time zone storage unit of FIG. 2, with reference to FIG. 3, A showing the time zone storage unit, a1, a2, a3, and a4 being source device type labels, showing the different data source storage subunits, i.e., the device types represented by a1, a2, a3, and a4 being different; e. f, g and h are source equipment labels and represent different source equipment storage sub-modules, namely e, f, g and h represent unique equipment; each source device storage sub-module 1, 2, 3, 4, 5 and 6 is a different data type label, which represents different data type storage sub-modules, that is, each data type storage sub-module stores time sequence data of the same data type.
Under the structure shown in fig. 3, the accessing operation of the time-series data to be accessed in the storage sub-module of the target source device may include: acquiring data type information of data to be accessed; determining a target data type storage sub-module according to the data type information and the data type label corresponding to each data type storage sub-module in the target source equipment storage sub-module; and performing access operation on the time sequence data to be accessed in the target data type storage sub-module.
By the method, the target data type storage sub-module of the time sequence data to be accessed is determined in the target source equipment storage sub-module according to the data type information of the data to be accessed and the data type label corresponding to each data type storage sub-module in the target source equipment storage sub-module, so that the access operation is conveniently performed in the data type storage sub-module of the data with the same type, and the efficiency of the access operation is further improved.
It should be noted that, in the present disclosure, each time zone storage list may maintain source device type storage subunits corresponding to different source device type tags through a two-dimensional table; each source equipment type storage subunit can maintain source equipment storage subunits corresponding to different source equipment labels through a two-dimensional table; each source device storage sub-module can maintain the data type storage sub-modules corresponding to different data type labels through a two-dimensional table.
In a possible manner, when the access operation is a write operation, the performing the access operation on the time-series data to be accessed in the target data type storage sub-module may include: and under the condition that the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data strip last time by the target data type storage submodule, updating the termination time point of the data strip and the statistics number information of the data strip according to the time attribute information of the time sequence data to be accessed, wherein each data strip is provided with the start time point of the data strip according to the start time point of the time sequence data corresponding to the index value of initial writing, and the statistics number information is used for representing the number of the index values of the corresponding data strip record.
In the present disclosure, the time series data includes an index value. For example, when the time series data is the CPU occupancy, the index value may be a numerical value of the occupancy, such as 20%, or the like.
It should be noted that the data type storage sub-module may maintain time-ordered data through a two-dimensional table. The two-dimensional table may be as shown in table 2:
start time point Termination time point Index value Statistics of frequency information
2021-1-109:00:00 2021-1-109:15:00 2 4
2021-1-109:20:00 2021-1-109:45:00 3 6
TABLE 2
As shown in table 2, table 2 includes two data pieces, and time series data are collected and stored every 5 minutes in table 2. In the first data record in Table 2, the index values representing the time series data acquired four times in succession during the period 2021-1-109:00:00 to 2021-1-109:15:00 are all 2.
Taking table 2 as an example, when the index value of the time series data collected at 2021-1-109:50:00 is 3, since the index value is the same as the index value in the time series data written in the data strip last time in the above table (i.e. the data strip with the starting time point of 2021-1-100009:20:00), only the ending time point and the statistics of the data strip need to be updated at this time, that is, the ending time point is 2021-1-109:50:00, and the statistics of the data strip is updated to 7, the time series data of 2021-1-109:50:00 can be reflected in the data strip, and then if the time series data in the data type storage submodule needs to be analyzed, the original data at each collection time point can be recovered according to the statistics and the collection interval time.
By the method, continuous identical time sequence data are compressed and stored, one data record is used for representing a plurality of continuous identical time sequence data, so that the number of data strips in a data type storage submodule is reduced, the aim of compressing and storing the data is achieved, and the occupied memory of the time sequence data to storage equipment is further reduced.
In a possible manner, the time series data stored in the data type storage sub-module includes at least index values of time series at two points in time. For example, table 2 above may be in the form of table 3:
start time point Termination time point Index value 1 Index value 2 Statistics of frequency information
2021-1-109:00:00 2021-1-109:05:00 2 2 2
2021-1-109:15:00 2021-1-109:20:00 4 5 2
TABLE 3 Table 3
In table 3, an index value 1 is an index value of time series data corresponding to a start time point, and an index value 2 is an index value of time series data corresponding to an end time point. It should be understood that the index values of the time series data corresponding to the acquisition time between the start time point and the end time point may also be recorded in the table.
Under the condition that at least two index values are recorded in one data strip, under the condition that the index value in the time sequence data to be accessed is different from the index value in the time sequence data written in the data strip last time by the target data type storage sub-module, determining a first slope according to the index value of the time sequence data to be accessed and any index value in the data strip; when the difference between the first slope and the second slope between any two index values stored in the data bar is larger than a preset threshold value, a new data bar is generated, a starting time point and a stopping time point of the data bar are set according to time attribute information of time sequence data to be accessed, the index values in the time sequence data to be accessed are written into the data bar, and statistical number information of the new data bar is set.
It should be noted that the larger the slope difference between the first slope and the second slope, the larger the trend of the time series data characterizing the time series data to be accessed in joining the data strip is different from the trend of the time series data in the data strip; conversely, the smaller the slope difference between the first slope and the second slope, the smaller the trend of the time series data characterizing the time series data to be accessed in joining the piece of data and the trend of the time series data in the piece of data differ.
For example, the preset threshold may be set according to actual conditions.
Taking table 3 as an example, if the index value of the time-series data to be accessed at 2021-1-109:25:00 is 8, the second slopes of the two index values in the data strips 2021-1-109:15:00-2021-109:20:00 are: (index value 2-index value 1)/acquisition time interval, calculating to obtain a second slope of (5-4)/5=0.2; the first slope may be: (index value 2-index value of time series data to be accessed)/acquisition time interval, the first slope is calculated to be (8-4)/5=0.8, because the first slope is greatly different from the second slope (the first slope is characterized by small trend rising, the second slope is characterized by large trend rising), so that a new data strip needs to be generated in table 3 to record the time series data, and the starting time point and the ending time point of the newly generated data strip are 2021-1-109:25:00, then table 3 is transformed as shown in table 4:
Start time point Termination time point Index value 1 Index value 2 Statistics of frequency information
2021-1-109:00:00 2021-1-109:05:00 2 2 2
2021-1-109:15:00 2021-1-109:20:00 4 5 2
2021-1-109:25:00 2021-1-109:25:00 8 1
TABLE 4 Table 4
When the difference between the first slope and the second slope between any two index values stored in the data bar is smaller than or equal to a preset threshold value, in a possible manner, updating the termination time point of the data bar according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence data to be accessed into the data bar, and updating the statistics number information of the data bar. For example, if the index value of the time-series data to be accessed at 2021-1-109:25:00 is 6, the second slope of the two index values in the data strips 2021-1-109:15:00-2021-109:20:00 is: (index value 2-index value 1)/acquisition time interval, calculating to obtain a second slope of (5-4)/5=0.2; the first slope may be: (index value of time series data to be accessed-index value 2)/acquisition time interval, calculating to obtain a first slope of (6-5)/5=0.2, updating table 3 in the above manner because the difference between the first slope and the second slope is zero, and obtaining table 5:
TABLE 5
In the above manner, in the case where continuous time series data has a similar tendency, the time series data is recorded with one piece of data.
When the difference between the first slope and the second slope between any two index values stored in the data bar is smaller than or equal to a preset threshold, and if the index value written by the data bar last time is not the index value written by the data bar first time, the index value written by the data bar last time can be updated according to the index value in the time sequence number to be accessed, the termination time point of the data bar is updated according to the time attribute information of the time sequence data to be accessed, and the statistics number information of the data bar is updated. When the index value of the time series data to be accessed is still 6 according to 2021-1-109:25:00, table 3 is updated in the above manner, and table 6 may be obtained:
start time point Termination time point Index value 1 Index value 2 Statistics of frequency information
2021-1-109:00:00 2021-1-109:05:00 2 2 2
2021-1-109:15:00 2021-1-109:25:00 4 6 3
TABLE 6
By adopting the mode, under the condition that continuous time sequence data has similar trend, one data strip is used for reflecting the continuous time sequence data with similar trend in a lossy compression mode, namely, the recording quantity of index values is reduced, the memory occupied by the data is further reduced, the recording form is favorable for recovering all original data, and the subsequent data analysis of the time sequence data is not influenced.
Based on the same inventive concept, the embodiment of the disclosure further provides a time sequence data access device, which is applied to electronic equipment, wherein the electronic equipment comprises a storage device, the storage device comprises a plurality of time zone storage units, each time zone storage unit comprises at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone label, and each data source storage subunit is provided with a corresponding data source label. Fig. 4 is a block diagram of a time series data access apparatus according to an exemplary embodiment of the present disclosure, and referring to fig. 4, the time series data access apparatus 400 includes:
An obtaining module 401, configured to obtain time sequence data to be accessed, where the time sequence data to be accessed includes attribute information of the time sequence data to be accessed, and the attribute information includes time attribute information and data source attribute information;
the access module 402 is configured to determine a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and a time zone tag corresponding to each time zone storage unit, determine a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and a data source tag corresponding to each data source storage subunit in the target time zone storage unit, and perform an access operation on the time sequence data to be accessed in the target data source storage subunit.
Optionally, the data source storage subunit is a source equipment type storage subunit, the data source tag is a source equipment type tag, each source equipment type storage subunit includes at least one source equipment storage subunit, and the source equipment storage subunit is provided with a corresponding source equipment tag;
The access module 402 includes:
the first access sub-module is used for determining a target source equipment type storage sub-unit corresponding to the time sequence data to be accessed according to source equipment type information in the data source attribute information and source equipment type labels corresponding to each source equipment type storage sub-unit in the target time zone storage unit;
and the second access sub-module is used for determining a target source equipment storage sub-module according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the target source equipment type storage sub-unit, and performing access operation on the time sequence data to be accessed in the target source equipment storage sub-module.
Optionally, the source device storage sub-module further includes at least one data type storage sub-module, where the data type storage sub-module is provided with a corresponding data type tag;
the access module 402 further includes:
the acquisition sub-module is used for acquiring the data type information of the data to be accessed;
the third access sub-module determines a target data type storage sub-module according to the data type information and the data type label corresponding to each data type storage sub-module in the target source equipment storage sub-module;
And the fourth access sub-module is used for performing access operation on the time sequence data to be accessed in the target data type storage sub-module.
Optionally, the time sequence data includes an index value, the access operation is a write operation, and the fourth access submodule is specifically configured to update, when the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data stripe last time by the target data type storage submodule, an ending time point of the data stripe and statistics number information of the data stripe according to time attribute information of the time sequence data to be accessed, where each data stripe is provided with the starting time point of the data stripe according to a starting time point of the time sequence data corresponding to the index value of initial writing, and the statistics number information is used to characterize the number of index values recorded by the corresponding data stripe.
Optionally, the apparatus 400 further includes a slope determining module, configured to determine, when the index value in the time-series data to be accessed is different from the index value in the time-series data written in the data stripe last time by the target data type storage sub-module, a first slope according to the index value of the time-series data to be accessed and any index value in the data stripe;
The data adding module is used for generating a new data bar when the difference between the first slope and the second slope between any two index values stored in the data bar is larger than a preset threshold value, setting a starting time point and a stopping time point of the data bar according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence data to be accessed into the data bar, and setting statistical number information of the new data bar.
Optionally, the apparatus 400 further includes a first data updating module, configured to update a termination time point of the data strip according to the time attribute information of the time sequence data to be accessed when a difference between the first slope and a second slope between any two index values stored in the data strip is less than or equal to the preset threshold, and write the index value in the time sequence data to be accessed into the data strip, and update statistical number information of the data strip;
and the second data updating module is used for updating the index value written last time of the data strip according to the index value in the time sequence to be accessed, updating the termination time point of the data strip according to the time attribute information of the time sequence data to be accessed and updating the statistics number information of the data strip if the index value written last time of the data strip is not the index value written first time of the data strip when the difference between the first slope and the second slope between any two index values stored in the data strip is smaller than or equal to the preset threshold value.
The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.
Based on the same inventive concept, the embodiments of the present disclosure further provide a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the time-series data access method of any one of the method embodiments.
Based on the same inventive concept, the embodiments of the present disclosure further provide an electronic device, including:
a storage device having a computer program stored thereon;
a processor for executing the computer program in the storage device to implement the steps of the time series data access method of any one of the method implementation examples.
Fig. 5 is a block diagram of an electronic device 500, according to an example embodiment. As shown in fig. 5, the electronic device 500 may include: processor 501, memory device 502. The electronic device 500 may also include one or more of a multimedia component 503, an input/output (I/O) interface 504, and a communication component 505.
The processor 501 is configured to control the overall operation of the electronic device 500 to perform all or part of the steps in the above-described time series data access method. The storage device 502 is used to store various types of data to support operations at the electronic device 500, which may include, for example, instructions for any application or method operating on the electronic device 500, as well as application-related data, such as contact data, transceived messages, pictures, audio, video, and the like. The Memory device 502 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 503 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the storage device 502 or transmitted through the communication component 505. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 504 provides an interface between the processor 501 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 505 is used for wired or wireless communication between the electronic device 500 and other devices. Wireless communication, such as Wi-Fi, bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G or 4G, or a combination of one or more thereof, the corresponding communication component 505 may thus comprise: wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic device 500 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), digital signal processors (Digital Signal Processor, abbreviated as DSP), digital signal processing devices (Digital Signal Processing Device, abbreviated as DSPD), programmable logic devices (Programmable Logic Device, abbreviated as PLD), field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described method of accessing time-sequential data.
In another exemplary embodiment, a computer readable storage medium is also provided, comprising program instructions which, when executed by a processor, implement the steps of the above-described time-series data access method. For example, the computer readable storage medium may be the storage device 502 described above including program instructions executable by the processor 501 of the electronic device 500 to perform the time-series data access method described above.
The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.
In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. The various possible combinations are not described further in this disclosure in order to avoid unnecessary repetition.
Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.

Claims (10)

1. A time-series data access method, characterized in that it is applied to an electronic device, the electronic device includes a storage device, the storage device includes a plurality of time zone storage units, each time zone storage unit includes at least one data source storage subunit, each time zone storage unit is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag, the method includes:
acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
2. The method of claim 1, wherein the data source storage sub-units are source device type storage sub-units, the data source tags are source device type tags, each source device type storage sub-unit comprises at least one source device storage sub-module provided with a corresponding source device tag;
the determining the target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing the access operation on the time sequence data to be accessed in the target data source storage subunit includes:
determining a target source equipment type storage subunit corresponding to the time sequence data to be accessed according to source equipment type information in the data source attribute information and source equipment type labels corresponding to each source equipment type storage subunit in the target time zone storage unit;
and determining a target source equipment storage sub-module according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the target source equipment type storage sub-unit, and performing access operation on the time sequence data to be accessed in the target source equipment storage sub-module.
3. The method of claim 2, wherein the source device storage sub-module further comprises at least one data type storage sub-module provided with a corresponding data type tag;
the accessing operation of the time sequence data to be accessed in the storage sub-module of the target source equipment comprises the following steps:
acquiring data type information of the data to be accessed;
determining a target data type storage sub-module according to the data type information and the data type label corresponding to each data type storage sub-module in the target source equipment storage sub-module;
and performing access operation on the time sequence data to be accessed in the target data type storage sub-module.
4. The method of claim 3, wherein the timing data comprises an indicator value, the access operation is a write operation, the accessing the timing data to be accessed in the target data type storage sub-module comprises:
and under the condition that the index value in the time sequence data to be accessed is the same as the index value in the time sequence data written in the data strip last time by the target data type storage submodule, updating the ending time point of the data strip and the statistics times information of the data strip according to the time attribute information of the time sequence data to be accessed, wherein each data strip is provided with the starting time point of the data strip according to the starting time point of the time sequence data corresponding to the index value which is initially written, and the statistics times information is used for representing the number of the index values recorded by the corresponding data strip.
5. The method according to claim 4, wherein the method further comprises:
determining a first slope according to the index value of the time sequence data to be accessed and any index value in the data strip under the condition that the index value in the time sequence data to be accessed is different from the index value in the time sequence data written in the data strip last time by the target data type storage sub-module;
when the difference between the first slope and the second slope between any two index values stored in the data bar is larger than a preset threshold value, generating a new data bar, setting a starting time point and a stopping time point of the data bar according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence data to be accessed into the data bar, and setting statistical number information of the new data bar.
6. The method of claim 5, wherein the method further comprises:
when the difference between the first slope and the second slope between any two index values stored in the data bar is smaller than or equal to the preset threshold value, updating the termination time point of the data bar according to the time attribute information of the time sequence data to be accessed, writing the index value in the time sequence number to be accessed into the data bar, and updating the statistics number information of the data bar; or alternatively, the process may be performed,
When the difference between the first slope and the second slope between any two index values stored in the data bar is smaller than or equal to the preset threshold, if the index value written by the data bar last time is not the index value written by the data bar first time, updating the index value written by the data bar last time according to the index value in the time sequence number to be accessed, updating the termination time point of the data bar according to the time attribute information of the time sequence data to be accessed, and updating the statistics number information of the data bar.
7. A time-series data access apparatus, characterized in that it is applied to an electronic device, the electronic device includes a storage device, the storage device includes a plurality of time zone storage units, each of the time zone storage units includes at least one data source storage subunit, each of the time zone storage units is provided with a corresponding time zone tag, and the data source storage subunit is provided with a corresponding data source tag, the apparatus includes:
the acquisition module is used for acquiring time sequence data to be accessed, wherein the time sequence data to be accessed comprises attribute information of the time sequence data to be accessed, and the attribute information comprises time attribute information and data source attribute information;
The access module is used for determining a target time zone storage unit corresponding to the time sequence data to be accessed according to the time attribute information of the time sequence data to be accessed and the time zone label corresponding to each time zone storage unit, determining a target data source storage subunit corresponding to the time sequence data to be accessed according to the data source attribute information and the data source label corresponding to each data source storage subunit in the target time zone storage unit, and performing access operation on the time sequence data to be accessed in the target data source storage subunit.
8. The apparatus of claim 7, the data source storage subunit being a source device type storage subunit, the data source tag being a source device type tag, each source device type storage subunit comprising at least one source device storage subunit provided with a corresponding source device tag;
the access module comprises:
the first access sub-module is used for determining a target source equipment type storage sub-unit corresponding to the time sequence data to be accessed according to source equipment type information in the data source attribute information and source equipment type labels corresponding to each source equipment type storage sub-unit in the target time zone storage unit;
And the second access sub-module is used for determining a target source equipment storage sub-module according to the source equipment identification in the data source attribute information and the source equipment label of each source equipment storage sub-module in the target source equipment type storage sub-unit, and performing access operation on the time sequence data to be accessed in the target source equipment storage sub-module.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1-6.
10. An electronic device, comprising:
a storage device having a computer program stored thereon;
a processor for executing the computer program in the storage device to implement the steps of the method of any one of claims 1-6.
CN202110349472.8A 2021-03-31 2021-03-31 Time sequence data access method and device, storage medium and electronic equipment Active CN113177143B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110349472.8A CN113177143B (en) 2021-03-31 2021-03-31 Time sequence data access method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110349472.8A CN113177143B (en) 2021-03-31 2021-03-31 Time sequence data access method and device, storage medium and electronic equipment

Publications (2)

Publication Number Publication Date
CN113177143A CN113177143A (en) 2021-07-27
CN113177143B true CN113177143B (en) 2023-10-27

Family

ID=76922900

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110349472.8A Active CN113177143B (en) 2021-03-31 2021-03-31 Time sequence data access method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN113177143B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017080454A1 (en) * 2015-11-12 2017-05-18 北京国双科技有限公司 Website access path aggregation method and device
WO2018119587A1 (en) * 2016-12-26 2018-07-05 深圳前海达闼云端智能科技有限公司 Data processing method, device, and system, and information acquisition apparatus
CN109271277A (en) * 2017-07-17 2019-01-25 阿里巴巴集团控股有限公司 Access method, device and system after database delay machine
CN111090879A (en) * 2019-12-05 2020-05-01 达闼科技成都有限公司 Data processing method, device, readable storage medium, electronic equipment and system
CN111638968A (en) * 2019-03-01 2020-09-08 维塔科技(北京)有限公司 Shared memory access permission setting method and device, electronic equipment and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008009564A (en) * 2006-06-27 2008-01-17 Fujitsu Ltd Memory access device, memory access method, memory manufacturing method and program
US20100199060A1 (en) * 2009-02-04 2010-08-05 Panasonic Corporation Memory controller, nonvolatile memory module, access module, and nonvolatile memory system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017080454A1 (en) * 2015-11-12 2017-05-18 北京国双科技有限公司 Website access path aggregation method and device
WO2018119587A1 (en) * 2016-12-26 2018-07-05 深圳前海达闼云端智能科技有限公司 Data processing method, device, and system, and information acquisition apparatus
CN109271277A (en) * 2017-07-17 2019-01-25 阿里巴巴集团控股有限公司 Access method, device and system after database delay machine
CN111638968A (en) * 2019-03-01 2020-09-08 维塔科技(北京)有限公司 Shared memory access permission setting method and device, electronic equipment and storage medium
CN111090879A (en) * 2019-12-05 2020-05-01 达闼科技成都有限公司 Data processing method, device, readable storage medium, electronic equipment and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
一种母婴安全监护系统数据库的设计和实现;于楠;胡金炎;邹岸;王涛;林霖;邱健盛;;计算机应用与软件(05);全文 *
基于InfluxDB的桥梁监测系统设计与实现;韩艺坤;李富年;陈志丹;;现代电子技术(16);全文 *

Also Published As

Publication number Publication date
CN113177143A (en) 2021-07-27

Similar Documents

Publication Publication Date Title
US11636083B2 (en) Data processing method and apparatus, storage medium and electronic device
KR102177489B1 (en) Method and device of searching index for sensor tag data
CN111046034B (en) Method and system for managing memory data and maintaining data in memory
EP3287910A1 (en) Data backup method and data processing system
US9081838B2 (en) Methods, apparatuses, and computer program products for database record recovery
CN111061758B (en) Data storage method, device and storage medium
CN110659282B (en) Data route construction method, device, computer equipment and storage medium
CN108875077B (en) Column storage method and device of database, server and storage medium
CN108932286B (en) Data query method and device
CN105550314A (en) Data statistics method, device and system based on combined major key
CN111190792A (en) Log storage method and device, electronic equipment and readable storage medium
CN111966747A (en) Data synchronization method, system, terminal device and storage medium
CN108920516B (en) Real-time analysis method, system, device and computer readable storage medium
TWI769665B (en) Target data updating method, electronic equipment and computer readable storage medium
CN113177143B (en) Time sequence data access method and device, storage medium and electronic equipment
CN111046240B (en) Gateway traffic statistics method, device, computer equipment and storage medium
CN110188160B (en) Data storage method and data reading method
CN109633331B (en) Data storage method and fault recording device
CN111639087A (en) Data updating method and device in database and electronic equipment
CN115858471A (en) Service data change recording method, device, computer equipment and medium
CN114356912A (en) Method for writing data into database and computer equipment
CN114564501A (en) Database data storage and query methods, devices, equipment and medium
CN110866003B (en) Index value number estimation method and device and electronic equipment
US20200159734A1 (en) Data tracking apparatus, method, and non-transitory computer readable storage medium thereof
CN112783711A (en) Method and storage medium for analyzing program memory on NodeJS

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant