CN112000619A - Time sequence data storage method, device, equipment and readable storage medium - Google Patents
Time sequence data storage method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN112000619A CN112000619A CN202010851768.5A CN202010851768A CN112000619A CN 112000619 A CN112000619 A CN 112000619A CN 202010851768 A CN202010851768 A CN 202010851768A CN 112000619 A CN112000619 A CN 112000619A
- Authority
- CN
- China
- Prior art keywords
- target
- data
- file
- time
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000013500 data storage Methods 0.000 title claims abstract description 58
- 238000004590 computer program Methods 0.000 claims description 10
- 238000012163 sequencing technique Methods 0.000 abstract 1
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000004883 computer application Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
Abstract
The invention discloses a time sequence data storage method, a time sequence data storage device, time sequence data storage equipment and a readable storage medium, wherein the method comprises the following steps: acquiring target data which needs to be arranged according to a time sequence, and determining the target data as a target element; determining a target class corresponding to the target element; the target inherits the abstract class with the timestamp members; determining an object file for storing object elements in an NIO mode by using the object class; and putting the target elements into a target cache of the target file, and storing the target elements into a target channel of the target file. Therefore, in the method, the members based on the time stamp can realize the sequencing of the data according to the time axis, the data storage efficiency can be accelerated by the NIO mode, and the influence on other service performances caused by the storage of a large amount of time axis form data can be avoided.
Description
Technical Field
The present invention relates to the field of computer application technologies, and in particular, to a method, an apparatus, a device, and a readable storage medium for storing time sequence data.
Background
"timeline-form data," i.e., data arranged in chronological order, such as log data, some recorded data, etc. If the 'data in the time axis form' is stored in the database, when the amount of the stored data is large, sorting is needed, and the speed is low. Meanwhile, pressure is added to the MySql server, and other normal services are affected.
In summary, how to effectively improve the storage of the "data in the form of time axis" is a technical problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
The invention aims to provide a time sequence data storage method, a time sequence data storage device, time sequence data storage equipment and a readable storage medium, which can accelerate the storage efficiency of time axis form data and avoid influencing other service performances.
In order to solve the technical problems, the invention provides the following technical scheme:
a method of time sequential data storage, comprising:
acquiring target data which needs to be arranged according to a time sequence, and determining the target data as a target element;
determining a target class corresponding to the target element; the target inherits an abstract class with a timestamp member;
determining an object file for storing the object element by adopting an NIO mode by utilizing the object class;
and putting the target element into a target cache of the target file, and storing the target element into a target channel of the target file.
Preferably, the placing the target element into a target cache of the target file, and storing the target element into a target channel of the target file includes:
acquiring a file operation lock of the target file;
acquiring the size of data in the target channel;
comparing the corresponding relation between the data size and the record size;
and according to the corresponding relation, the target element is placed into the target cache and is stored into the target channel.
Preferably, according to the correspondence, the step of placing the target element into the target cache and storing the target element into the target channel includes:
applying a target cache with a corresponding size according to the record size under the condition that the data size is larger than or equal to the record size;
reading the last element in the target channel, and judging whether the time stamp member assignment of the element is greater than or equal to the time stamp member assignment of the target element;
if so, assigning a value by using the timestamp member of the target element, and deleting the element which is later than the target element in the target channel; and after deleting, the target data is put into the target cache and is stored into the target channel.
Preferably, according to the correspondence, the step of placing the target element into the target cache and storing the target element into the target channel includes:
and under the condition that the data size is smaller than the record size, the target element is placed into the target cache of the target file and is stored into the target channel.
Preferably, the method further comprises the following steps:
receiving and analyzing a data reading request to obtain an appointed class and time information corresponding to data to be read;
determining a storage file of data to be read by using the specified class;
and reading the object data set from the storage file by using the time information.
Preferably, the time information includes an incoming start time and an end time, and the reading of the object data set from the storage file using the time information includes:
searching the number of elements corresponding to the starting time and the ending time in a channel of the storage file;
applying for reading a cache according to the number of the elements;
circulating the elements in the channel for storing the file, and reading the elements to the reading cache according to step;
producing the object dataset by reflection using the read cache that is full.
Preferably, the acquiring of the target data to be arranged in time sequence includes:
receiving a data storage request and determining data to be stored;
judging whether the data to be stored need to be arranged according to a time sequence;
and if so, determining the data to be stored as the target data.
A time sequential data storage device comprising:
the target element determining module is used for acquiring target data which needs to be arranged according to a time sequence and determining the target data as target elements;
the target class determining module is used for determining a target class corresponding to the target element; the target inherits an abstract class with a timestamp member;
the target file determining module is used for determining a target file for storing the target elements in an NIO mode by using the target class;
and the storage module is used for placing the target element into a target cache of the target file and storing the target element into a target channel of the target file.
A time sequential data storage device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the above-mentioned time-sequential data storage method when executing the computer program.
A readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned time-sequential data storage method.
By applying the method provided by the embodiment of the invention, the target data which needs to be arranged according to the time sequence is obtained, and the target data is determined as the target element; determining a target class corresponding to the target element; the target inherits the abstract class with the timestamp members; determining an object file for storing object elements in an NIO mode by using the object class; and putting the target elements into a target cache of the target file, and storing the target elements into a target channel of the target file.
Since the NIO (new input/output) differs from the original I/O mainly in the way data is packed and transmitted. Specifically, the original I/O processes data in a streaming manner, while the NIO processes data in a block manner. Processing data in blocks is much faster than processing data in bytes of a stream. In the method, the target data with the time sequence ordering requirement is stored by adopting NIO to accelerate the data storage efficiency. In the method, when target data are stored, time sequence arrangement is considered, and a scheme is proposed: firstly, target data needing to be arranged according to the time sequence is determined as target elements, and then a target class equal to the target elements is determined. In particular, to ensure chronological ordering, the target class also needs to inherit the abstract class with the members of the timestamp, i.e., timestamp the target element. Then, an object file storing the object elements in the NIO mode is determined based on the object class. And finally, accessing the target element to a target cache of the target file, and storing the target element into a target channel of the target file to finish data storage. Therefore, in the method, the members based on the time stamp can sort the data according to the time axis, the data storage efficiency can be improved by storing the data in an NIO mode, and the influence on other service performances caused by storing a large amount of time axis form data can be avoided.
Accordingly, embodiments of the present invention further provide a time sequence data storage apparatus, a device and a readable storage medium corresponding to the time sequence data storage method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating an implementation of a method for storing time-sequential data according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a file topology relationship according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a chronological data storage device according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a chronological data storage device in an embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a chronological data storage device according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating a method for storing time-sequential data according to an embodiment of the present invention, the method including the following steps:
s101, acquiring target data which needs to be arranged according to a time sequence, and determining the target data as a target element.
It should be noted that, in the embodiment of the present invention, efficient storage is mainly performed on target data that needs to be sorted in a time sequence. The target data may be system monitoring information acquired in real time, such as a log, or other data that needs to be sorted in time sequence, such as meteorological recording data.
For convenience of description, after the target data is obtained, the target data is determined as a target element.
Specifically, the process of acquiring the target data includes:
step one, receiving a data storage request and determining data to be stored;
step two, judging whether the data to be stored need to be arranged according to a time sequence;
and step three, if so, determining the data to be stored as target data.
For convenience of description, the above three steps will be described in combination.
After receiving a data storage request, data to be stored is determined first. The method for judging whether the data to be stored is time that needs to be arranged according to the time sequence may specifically be to preset information such as a data type, a data source, and a data tag that need to be arranged according to the time sequence. After the data to be stored are obtained, whether the data to be stored need to be arranged according to the time sequence can be determined through comparison. And under the condition that the data need to be arranged according to the time sequence, determining the data to be stored as target data.
S102, determining a target class corresponding to the target element; the target inherits the abstract class with the timestamp members.
Referring to fig. 2, fig. 2 is a schematic view of a file topology relationship according to an embodiment of the present invention. In the embodiment of the invention, the DataBase is represented by DataBase, a plurality of types of tables are arranged in the DataBase, each type of Table is a specific Java class, the corresponding field of the member is, and a fixed sized buffer, namely a buffer in NIO, can be generated according to the combination because the Java basic types can be freely combined. Record corresponds to a row of records in the table, each Record has a unique identifier stored as a NIO file name, and a plurality of data sets ordered according to time sequence are stored in each file.
The abstract class, i.e. the data format abstract class, corresponds to a table in MySql, where all classes in specific formats inherit the abstract class. Only the timeStamp member in this abstract class takes the current system time in the constructor and assigns it.
Based on this, in the embodiment of the present invention, after the target element is acquired, the target class corresponding to the target element may be determined.
S103, determining an object file for storing the object element in an NIO mode by using the object class.
The database table operation type DataTable is designed, and the database table operation type packages data operations.
After the target class of the target element is determined, the target file for storing the target element in the NIO mode can be determined based on the target class. That is, the target file is a file for storing data corresponding to the target class.
And S104, placing the target element into a target cache of the target file, and storing the target element into a target channel of the target file.
After the target file of the target element is determined to be stored, the target element can be placed into the target cache of the target file, the target element is stored into the target channel of the target file, and finally the tray falling is achieved.
The specific storage process comprises the following steps:
step one, acquiring a file operation lock of a target file;
step two, acquiring the size of data in a target channel;
step three, comparing the corresponding relation between the data size and the record size;
and step four, according to the corresponding relation, the target element is placed into a target cache and is stored into a target channel.
The above four steps are combined for convenience of description.
The method avoids the situation that a plurality of threads operate the same target file at the same time, and data confusion is caused. In the embodiment of the invention, the file operation lock of the target file can be obtained firstly. Then, the data size of the target channel of the target file, and the data size in the current channel of the target file are obtained. By comparing the data size with the record size (recordSize), it is determined at which position in the target file the target element is stored.
Specifically, according to the corresponding relationship between the data size and the record size, the step four may specifically include the following two different processing modes:
mode 1: the processing procedure corresponding to the case that the data size is larger than or equal to the record size comprises the following steps:
step 1.1, under the condition that the size of the data is larger than or equal to the size of the record, applying for a target cache with a corresponding size according to the size of the record;
step 1.2, reading the last element in the target channel, and judging whether the time stamp member assignment of the element is greater than or equal to the time stamp member assignment of the target element;
step 1.3, if yes, assigning a value by using a time stamp member of the target element, and deleting an element which is later than the target element in the target channel; and after deleting, the target data is put into the target cache and is stored into the target channel.
In the mode 1, if the data size in the channel of the target file is larger than or equal to the current recordSize, applying for a buffer with a corresponding size according to the recordSize; and reading the last element in the channel, if the corresponding time of the element is taken out and is more than or equal to the corresponding time of the target element to be stored, calculating the subscript of the element of which the time is greater than the corresponding time of the target element in the channel according to the time of the target element to be stored, and reserving the data which is earlier than the target element (the data can be searched by a dichotomy). Finally, the target element is put into the buffer and stored into the channel. If the corresponding time of the element is taken out to be less than the corresponding time of the target element to be stored, the target element can be directly put in behind the element.
Mode 2: the processing procedure corresponding to the case where the data size is smaller than the record size includes:
and under the condition that the data size is smaller than the record size, the target element is placed into a target cache of the target file and is stored into a target channel.
By way of example, the method can be used: the apenddata (T record) determines the storage location and stores. The apppenddata (t record) can calculate the position of the target element in the nio file according to the specific type of the incoming data, and write the target element into the nio file.
By applying the method provided by the embodiment of the invention, the target data which needs to be arranged according to the time sequence is obtained, and the target data is determined as the target element; determining a target class corresponding to the target element; the target inherits the abstract class with the timestamp members; determining an object file for storing object elements in an NIO mode by using the object class; and putting the target elements into a target cache of the target file, and storing the target elements into a target channel of the target file.
Since the NIO (new input/output) differs from the original I/O mainly in the way data is packed and transmitted. Specifically, the original I/O processes data in a streaming manner, while the NIO processes data in a block manner. Processing data in blocks is much faster than processing data in bytes of a stream. In the method, the target data with the time sequence ordering requirement is stored by adopting NIO to accelerate the data storage efficiency. In the method, when target data are stored, time sequence arrangement is considered, and a scheme is proposed: firstly, target data needing to be arranged according to the time sequence is determined as target elements, and then a target class equal to the target elements is determined. In particular, to ensure chronological ordering, the target class also needs to inherit the abstract class with the members of the timestamp, i.e., timestamp the target element. Then, an object file storing the object elements in the NIO mode is determined based on the object class. And finally, accessing the target element to a target cache of the target file, and storing the target element into a target channel of the target file to finish data storage. Therefore, in the method, the members based on the time stamp can sort the data according to the time axis, the data storage efficiency can be improved by storing the data in an NIO mode, and the influence on other service performances caused by storing a large amount of time axis form data can be avoided.
It should be noted that, based on the above embodiments, the embodiments of the present invention also provide corresponding improvements. In the preferred/improved embodiment, the same steps as those in the above embodiment or corresponding steps may be referred to each other, and corresponding advantageous effects may also be referred to each other, which are not described in detail in the preferred/improved embodiment herein.
Preferably, based on the time-series data storage method provided in the foregoing embodiment, the data reading process can be accelerated. The specific implementation process comprises the following steps:
step one, receiving and analyzing a data reading request to obtain an appointed class and time information corresponding to data to be read;
determining a storage file of data to be read by using the specified class;
and thirdly, reading the object data set from the storage file by using the time information.
For convenience of description, the above three steps will be described in combination.
After receiving the data reading request, the data reading request is analyzed, so that the appointed class and the time information corresponding to the data to be read can be obtained. Based on the specified class, it is then determined from which file to read the stored file. Note that the storage file is a file corresponding to the designated class. At this time, the file operation lock of the storage file may be acquired, and then, using the time information, the object data set may be read from the storage file.
Wherein the time information comprises an incoming start time and an incoming end time, utilizing the time information; reading the object data set from the storage file in the third step specifically includes:
step 3.1, finding out the number of elements corresponding to the starting time and the ending time in a channel of the storage file;
step 3.2, applying for reading the cache according to the number of the elements;
3.3, circularly storing the elements in the channel of the file, and progressively reading the elements into a reading cache according to step;
and 3.4, utilizing the full read cache to produce the object data set through reflection.
Specifically, after the storage file is determined, the file operation lock of the storage file is first acquired, and then the maximum and minimum subscripts of the elements in the channel are found out by a dichotomy according to the transmitted start time and end time, so that the number of the elements can be obtained.
Then, calculating the progressive step value of the elements according to the number of the elements, applying an exclusive cache buffer according to the number of the elements, circulating the elements in the channel, and progressively removing the elements according to the step until the buffer is full. And finally, according to the returned buffer, producing a specific object data set through reflection.
In particular, based on step progression, the time-axis aligned data can be uniformly sampled over time. That is to say, in the embodiment of the present invention, not only the data reading efficiency can be accelerated, but also uniform sampling can be realized.
For example, the following steps are carried out: the method can be adopted: list < T > readRecord (Long startTime, Long endTime, int maxRecord), which is used to read data records. The Long startTime correspondingly and respectively represents the starting time, the Long endTime corresponds to the ending time, the int maxRecord corresponds to the maximum number of returned records, and if the number of the data sets in the incoming time parameter range is larger than the maximum number of the incoming records, the returned data sets are uniformly distributed on the time axis.
Preferably, in other embodiments of the present invention, the design DataBase may be operated by the class DataBase, considering that in practical application, the operations such as modification, deletion, and viewing of the "data in the form of time axis" are also performed. The class encapsulates database table operations, and the main contents comprise the following:
1. the method comprises the following steps: DataBase getInstance (Class < T > clazz), which is used to generate a DataBase object instance.
2. The method comprises the following steps: addData (T Record, int Record), where Record is a subclass type that inherits Record class, and Record is a Record id of data of this class, like a primary key in a database. The function has the main function of calling the data table's apppenddata method to store data.
3. The method comprises the following steps: readRecords (int recordld, long startTime, long endTime, int maxRecordNum). The method is used for looking up data in a record of a certain table. The main function of the function is to call the readRecord method of the DataTable.
4. The method comprises the following steps: delete (int recordid), this method is used to delete a certain record in the table, namely delete the corresponding NIO file.
Corresponding to the above method embodiments, embodiments of the present invention further provide a time-sequential data storage device, and the time-sequential data storage device described below and the time-sequential data storage method described above may be referred to in correspondence with each other.
Referring to fig. 3, the apparatus includes the following modules:
a target element determining module 101, configured to obtain target data that needs to be arranged in a time sequence, and determine the target data as a target element;
a target class determination module 102, configured to determine a target class corresponding to the target element; the target inherits the abstract class with the timestamp members;
the target file determining module 103 is configured to determine, by using the target class, a target file storing the target element in an NIO manner;
and the storage module 104 is configured to place the target element into a target cache of the target file, and store the target element into a target channel of the target file.
By applying the device provided by the embodiment of the invention, target data which needs to be arranged according to a time sequence is obtained, and the target data is determined as a target element; determining a target class corresponding to the target element; the target inherits the abstract class with the timestamp members; determining an object file for storing object elements in an NIO mode by using the object class; and putting the target elements into a target cache of the target file, and storing the target elements into a target channel of the target file.
Since the NIO (new input/output) differs from the original I/O mainly in the way data is packed and transmitted. Specifically, the original I/O processes data in a streaming manner, while the NIO processes data in a block manner. Processing data in blocks is much faster than processing data in bytes of a stream. In the device, the NIO is adopted to store the target data with the time sequence ordering requirement so as to accelerate the data storage efficiency. In the present apparatus, in consideration of time-series arrangement when storing target data, a proposal is proposed: firstly, target data needing to be arranged according to the time sequence is determined as target elements, and then a target class equal to the target elements is determined. In particular, to ensure chronological ordering, the target class also needs to inherit the abstract class with the members of the timestamp, i.e., timestamp the target element. Then, an object file storing the object elements in the NIO mode is determined based on the object class. And finally, accessing the target element to a target cache of the target file, and storing the target element into a target channel of the target file to finish data storage. Therefore, in the device, the members can sort the data according to the time axis based on the time stamp, the data storage efficiency can be accelerated by storing the data in an NIO mode, and the influence on other service performances caused by storing a large amount of time axis form data can be avoided.
In an embodiment of the present invention, the storage module 104 is specifically configured to obtain a file operation lock of a target file; acquiring the size of data in a target channel; comparing the corresponding relation between the data size and the record size; and according to the corresponding relation, putting the target element into a target cache and storing the target element into a target channel.
In a specific embodiment of the present invention, the storage module 104 is specifically configured to, when the size of the data is greater than or equal to the size of the record, apply for a target cache with a corresponding size according to the size of the record; reading the last element in the target channel, and judging whether the time stamp member assignment of the element is greater than or equal to the time stamp member assignment of the target element; if so, assigning a value by using the timestamp member of the target element, and deleting the element which is later than the target element in the target channel; and after deleting, the target data is put into the target cache and is stored into the target channel.
In an embodiment of the present invention, the storage module 104 is specifically configured to, in a case that the data size is smaller than the record size, put the target element into a target cache of the target file, and store the target element into the target channel.
In one embodiment of the present invention, the method further comprises:
the data reading module is used for receiving and analyzing the data reading request to obtain the appointed class and the time information corresponding to the data to be read; determining a storage file of data to be read by using the specified class; and reading the object data set from the storage file by using the time information.
In a specific embodiment of the present invention, the time information includes an incoming start time and an incoming end time, and the data reading module is specifically configured to find out the number of elements corresponding to the start time and the end time in a channel of the storage file; applying for reading the cache according to the number of the elements; circularly storing elements in a channel of the file, and progressively reading the elements to a reading cache according to step; with a full read cache, the object dataset is produced by reflection.
In a specific embodiment of the present invention, the target element determining module 101 is specifically configured to receive a data storage request and determine data to be stored; judging whether the data to be stored need to be arranged according to a time sequence; and if so, determining the data to be stored as the target data.
Corresponding to the above method embodiment, an embodiment of the present invention further provides a time-sequential data storage device, and a time-sequential data storage device described below and a time-sequential data storage method described above may be referred to in correspondence with each other.
Referring to fig. 4, the time-sequential data storage device includes:
a memory 332 for storing a computer program;
a processor 322 for implementing the steps of the time sequential data storage method of the above-described method embodiments when executing the computer program.
Specifically, referring to fig. 5, fig. 5 is a schematic diagram of a specific structure of a time sequence data storage device provided in this embodiment, the time sequence data storage device may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 322 (e.g., one or more processors) and a memory 332, where the memory 332 stores one or more computer applications 342 or data 344. Memory 332 may be, among other things, transient or persistent storage. The program stored in memory 332 may include one or more modules (not shown), each of which may include a sequence of instructions operating on a time-sequential data storage device. Still further, the central processor 322 may be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the time sequential data storage device 301.
The chronological data storage device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341.
The steps in the above-described method of time sequential data storage may be implemented by the structure of a time sequential data storage device.
Corresponding to the above method embodiment, the embodiment of the present invention further provides a readable storage medium, and a readable storage medium described below and a time-series data storage method described above may be referred to in correspondence with each other.
A readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method of time sequential data storage of the above-mentioned method embodiments.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (10)
1. A method of time sequential data storage, comprising:
acquiring target data which needs to be arranged according to a time sequence, and determining the target data as a target element;
determining a target class corresponding to the target element; the target inherits an abstract class with a timestamp member;
determining an object file for storing the object element by adopting an NIO mode by utilizing the object class;
and putting the target element into a target cache of the target file, and storing the target element into a target channel of the target file.
2. The method of claim 1, wherein placing the target element into a target cache of the target file and storing the target element into a target channel of the target file comprises:
acquiring a file operation lock of the target file;
acquiring the size of data in the target channel;
comparing the corresponding relation between the data size and the record size;
and according to the corresponding relation, the target element is placed into the target cache and is stored into the target channel.
3. The method according to claim 2, wherein the step of storing the target element in the target cache and in the target channel according to the correspondence comprises:
applying a target cache with a corresponding size according to the record size under the condition that the data size is larger than or equal to the record size;
reading the last element in the target channel, and judging whether the time stamp member assignment of the element is greater than or equal to the time stamp member assignment of the target element;
if so, assigning a value by using the timestamp member of the target element, and deleting the element which is later than the target element in the target channel; and after deleting, the target data is put into the target cache and is stored into the target channel.
4. The method according to claim 2, wherein the step of storing the target element in the target cache and in the target channel according to the correspondence comprises:
and under the condition that the data size is smaller than the record size, the target element is placed into the target cache of the target file and is stored into the target channel.
5. The method of chronological data storage according to claim 1, further comprising:
receiving and analyzing a data reading request to obtain an appointed class and time information corresponding to data to be read;
determining a storage file of data to be read by using the specified class;
and reading the object data set from the storage file by using the time information.
6. The method of chronological data storage according to claim 5, wherein the time information comprises an incoming start time and end time, with which time information the object data set is read from the storage file, comprising:
searching the number of elements corresponding to the starting time and the ending time in a channel of the storage file;
applying for reading a cache according to the number of the elements;
circulating the elements in the channel for storing the file, and reading the elements to the reading cache according to step;
producing the object dataset by reflection using the read cache that is full.
7. The time-series data storage method according to claim 1, wherein acquiring target data to be arranged in time series comprises:
receiving a data storage request and determining data to be stored;
judging whether the data to be stored need to be arranged according to a time sequence;
and if so, determining the data to be stored as the target data.
8. A time sequential data storage device, comprising:
the target element determining module is used for acquiring target data which needs to be arranged according to a time sequence and determining the target data as target elements;
the target class determining module is used for determining a target class corresponding to the target element; the target inherits an abstract class with a timestamp member;
the target file determining module is used for determining a target file for storing the target elements in an NIO mode by using the target class;
and the storage module is used for placing the target element into a target cache of the target file and storing the target element into a target channel of the target file.
9. A time sequential data storage device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method of storing chronological data according to any of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps of the chronological data storage method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010851768.5A CN112000619A (en) | 2020-08-21 | 2020-08-21 | Time sequence data storage method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010851768.5A CN112000619A (en) | 2020-08-21 | 2020-08-21 | Time sequence data storage method, device, equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112000619A true CN112000619A (en) | 2020-11-27 |
Family
ID=73472349
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010851768.5A Withdrawn CN112000619A (en) | 2020-08-21 | 2020-08-21 | Time sequence data storage method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112000619A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109445955A (en) * | 2018-09-13 | 2019-03-08 | 武汉斗鱼网络科技有限公司 | A kind of method of counting and number system |
| CN111125171A (en) * | 2019-12-22 | 2020-05-08 | 浪潮(北京)电子信息产业有限公司 | Monitoring data access method, device, equipment and readable storage medium |
| CN111125089A (en) * | 2019-11-05 | 2020-05-08 | 远景智能国际私人投资有限公司 | Time sequence data storage method, device, server and storage medium |
| CN111177077A (en) * | 2018-11-09 | 2020-05-19 | 杭州海康威视数字技术股份有限公司 | Data storage and data query method, device, equipment and medium |
| CN111309720A (en) * | 2018-12-11 | 2020-06-19 | 北京京东尚科信息技术有限公司 | Time sequence data storage method, time sequence data reading method, time sequence data storage device, time sequence data reading device, electronic equipment and storage medium |
-
2020
- 2020-08-21 CN CN202010851768.5A patent/CN112000619A/en not_active Withdrawn
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109445955A (en) * | 2018-09-13 | 2019-03-08 | 武汉斗鱼网络科技有限公司 | A kind of method of counting and number system |
| CN111177077A (en) * | 2018-11-09 | 2020-05-19 | 杭州海康威视数字技术股份有限公司 | Data storage and data query method, device, equipment and medium |
| CN111309720A (en) * | 2018-12-11 | 2020-06-19 | 北京京东尚科信息技术有限公司 | Time sequence data storage method, time sequence data reading method, time sequence data storage device, time sequence data reading device, electronic equipment and storage medium |
| CN111125089A (en) * | 2019-11-05 | 2020-05-08 | 远景智能国际私人投资有限公司 | Time sequence data storage method, device, server and storage medium |
| CN111125171A (en) * | 2019-12-22 | 2020-05-08 | 浪潮(北京)电子信息产业有限公司 | Monitoring data access method, device, equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112346829B (en) | Method and equipment for task scheduling | |
| CN109542889B (en) | Stream data column storage method, device, equipment and storage medium | |
| WO2020238066A1 (en) | Software pressure test method and apparatus, and storage medium and server | |
| CN101405728B (en) | Relational database architecture with dynamic load capability | |
| CN110825694A (en) | Data processing method, device, equipment and storage medium | |
| CN112559475A (en) | Data real-time capturing and transmitting method and system | |
| CN113485999A (en) | Data cleaning method and device and server | |
| CN110706148B (en) | Face image processing method, device, equipment and storage medium | |
| CN111767258A (en) | File compression method, device, equipment and storage medium applied to mass files | |
| CN110795026B (en) | Hot spot data identification method, device, equipment and storage medium | |
| CN114356521A (en) | Task scheduling method and device, electronic equipment and storage medium | |
| US20160203032A1 (en) | Series data parallel analysis infrastructure and parallel distributed processing method therefor | |
| WO2023124135A1 (en) | Feature retrieval method and apparatus, electronic device, computer storage medium and program | |
| US7127446B1 (en) | File system based task queue management | |
| CN112000619A (en) | Time sequence data storage method, device, equipment and readable storage medium | |
| CN111147226A (en) | Data storage method, device and storage medium | |
| US20130218851A1 (en) | Storage system, data management device, method and program | |
| CN109947613B (en) | File reading test method and device | |
| CN114328747A (en) | Data synchronization method, data synchronization device, computer equipment and medium | |
| CN111782588A (en) | File reading method, device, equipment and medium | |
| CN114546793A (en) | Log generation method and device and computer readable storage medium | |
| CN111563123A (en) | Live warehouse metadata real-time synchronization method | |
| CN113297245A (en) | Method and device for acquiring execution information | |
| CN110275863A (en) | File moving method, device and storage medium | |
| CN113778777A (en) | Log playback method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201127 |