CN113434557A - Method, device, equipment and storage medium for querying range of label data - Google Patents
Method, device, equipment and storage medium for querying range of label data Download PDFInfo
- Publication number
- CN113434557A CN113434557A CN202110984987.5A CN202110984987A CN113434557A CN 113434557 A CN113434557 A CN 113434557A CN 202110984987 A CN202110984987 A CN 202110984987A CN 113434557 A CN113434557 A CN 113434557A
- Authority
- CN
- China
- Prior art keywords
- data
- query
- range
- tag data
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 78
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013138 pruning Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 abstract description 12
- 238000010586 diagram Methods 0.000 description 22
- 230000008569 process Effects 0.000 description 11
- 238000012545 processing Methods 0.000 description 9
- 230000008030 elimination Effects 0.000 description 5
- 238000003379 elimination reaction Methods 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000009966 trimming Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 235000019633 pungent taste Nutrition 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for querying the range of label data, by establishing a range query data structure in advance according to the attribute dereferencing of the tag data and establishing the corresponding relation between the attribute dereferencing of the tag data and the storage address of the tag data in the time series index file of the time series database, so that after the attribute dereferencing of the target tag data is obtained by querying in the range query data structure according to the query range of the attribute dereferencing in the input tag data range query request, determining the storage address of the target tag data in the time series index file according to the corresponding relation, and then target label data is obtained and output, so that the blank of the function of performing range query on the label data in some time sequence databases is filled, the user can conveniently perform range query on the label data in the time sequence databases, and the functions of the time sequence databases are enriched.
Description
Technical Field
The present application relates to the field of database technologies, and in particular, to a method, an apparatus, a device, and a storage medium for querying a range of tag data.
Background
The time sequence database is called as a time sequence database. The time series database is mainly used for processing data with time tags (which are changed in time sequence, i.e., time-sequenced), and the data with time tags is also called time series data. Tag (tag) data is used to identify the inherent attributes of a data object, and in some time series databases, tag data is stored in string format, which does not support range queries.
Two data range query modes commonly used in the time sequence database at present are provided, one is to use a fuzzy condition method to carry out matching query, and the other is to use an or instruction to splice query conditions. However, the first query method has strict requirements on naming of data field values stored in a time-series database, which not only increases the difficulty for a user to input a range query command, but also easily overlooks field values that do not conform to naming rules. The second query method is only suitable for querying scenes with a small number of field values, and is difficult to encode and implement for querying the number of field values in a large scale and even querying continuously changing field values. Many time series databases do not perform well for range query functions on tag data.
The technical problem to be solved by the technical personnel in the field is to make up the blank of the function of carrying out range query on the tag data in the time sequence database.
Disclosure of Invention
The application aims to provide a method, a device, equipment and a storage medium for querying the range of tag data, which are used for realizing the range query of the tag data in a time sequence database.
In order to solve the above technical problem, the present application provides a method for querying a range of tag data, including:
the method comprises the steps of determining a corresponding relation between attribute values of tag data and storage addresses of the tag data in a time series index file of a time series database in advance, and establishing a range query data structure according to the attribute values of the tag data;
inquiring in the range inquiry data structure according to the inquiry range of the attribute value in the input label data range inquiry request to obtain the attribute value of the target label data;
determining the storage address of the target tag data in the time sequence index file according to the corresponding relation;
acquiring the target label data according to the storage address of the target label data;
and outputting the target label data.
Optionally, the querying the query range of the attribute value in the query request according to the input tag data range to obtain the attribute value of the target tag data by querying in the query data structure of the range specifically includes:
judging whether the attribute value of the target tag data exists in the range query data structure or not according to the query range in the tag data range query request;
if yes, recording the query time of the target tag data, executing the query range of the attribute dereferencing in the input tag data range query request, and querying in the range query data structure to obtain the attribute dereferencing of the target tag data;
if not, recording the query time of the target label data, adding the attribute dereferencing of the target label data into the range query data structure, executing the query range of the attribute dereferencing in the input label data range query request, and querying the range query data structure to obtain the attribute dereferencing of the target label data.
Optionally, the recording the query time of the target tag data specifically includes:
and recording the latest query time of the target tag data.
Optionally, the recording the latest query time for the target tag data specifically includes:
and storing the attribute value of the target label data and the latest query time of the target label data through a Hash table structure.
Optionally, the determining whether the range query data structure has the attribute value of the target tag data specifically includes:
judging whether the latest query time of the target label data is recorded;
if so, determining that the attribute value of the target tag data exists in the range query data structure;
and if not, determining that the attribute value of the target tag data is not contained in the range query data structure.
Optionally, the method further includes:
judging whether the range query data structure reaches a pruning condition;
and if so, pruning the range query data structure according to the latest query time of each tag data in the range query data structure.
Optionally, the range query data structure is specifically a B + tree structure.
In order to solve the above technical problem, the present application further provides a device for querying a range of tag data, including:
the system comprises a creating unit, a searching unit and a searching unit, wherein the creating unit is used for predetermining the corresponding relation between the attribute value of the tag data and the storage address of the tag data in the time sequence index file of the time sequence database and creating a range query data structure according to the attribute value of each tag data;
the query unit is used for querying in the range query data structure to obtain the attribute value of the target tag data according to the query range of the attribute value in the input tag data range query request;
a determining unit, configured to determine, according to the correspondence, a storage address of the target tag data in the time series index file;
the acquisition unit is used for acquiring the target label data according to the storage address of the target label data;
an output unit for outputting the target tag data.
In order to solve the above technical problem, the present application further provides a device for querying a range of tag data, including:
a memory for storing instructions, said instructions comprising the steps of any of the above described methods of range query of tag data;
a processor to execute the instructions.
To solve the above technical problem, the present application further provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for querying the range of tag data according to any one of the above items.
According to the method for querying the range of the label data, a range query data structure is established in advance according to the attribute value of the label data, the corresponding relation between the attribute value of the label data and the storage address of the label data in the time sequence index file of the time sequence database is established, so that the storage address of the target label data in the time sequence index file is determined according to the corresponding relation after the attribute value of the target label data is obtained by querying in the range query data structure according to the query range of the attribute value in an input label data range query request, the target label data is further obtained and output, the blank of performing the range query function on the label data in some time sequence databases is filled, the user can conveniently perform the range query on the label data in the time sequence database, and the functions of the time sequence database are enriched.
The application also provides a device, equipment and storage medium for querying the range of the label data, which have the beneficial effects and are not repeated herein.
Drawings
For a clearer explanation of the embodiments or technical solutions of the prior art of the present application, the drawings needed for the description of the embodiments or prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a method for querying a range of tag data according to an embodiment of the present application
FIG. 2 is a schematic diagram of a data reading relationship in an InfluxDB database;
FIG. 3 is a schematic diagram of a time series index file;
fig. 4 is a schematic diagram of a B + tree structure according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of a query method of a B + tree according to an embodiment of the present application;
fig. 6 is a schematic diagram of a HashSet structure according to an embodiment of the present application;
FIG. 7 is a schematic diagram of the calculation of the HashSet structure;
fig. 8 is a functional topology diagram of a query of a tagged data range in a time series database according to an embodiment of the present application;
fig. 9 is a schematic diagram illustrating a process of eliminating non-hotspot tag data according to an embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of a tag data range query method according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of a device for querying a range of tag data according to an embodiment of the present application;
fig. 12 is a schematic structural diagram of a device for querying a range of tag data according to an embodiment of the present application.
Detailed Description
The core of the application is to provide a method, a device, equipment and a storage medium for querying the range of the tag data, which are used for realizing the range query of the tag data in a time sequence database.
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a flowchart of a method for querying a range of tag data according to an embodiment of the present disclosure; FIG. 2 is a schematic diagram of a data reading relationship in an InfluxDB database; fig. 3 is a schematic diagram of a time series index file.
As shown in fig. 1, a flowchart of a method for querying a range of tag data provided in an embodiment of the present application includes:
s101: the method comprises the steps of determining the corresponding relation between the attribute values of the tag data and the storage addresses of the tag data in a time sequence index file of a time sequence database in advance, and establishing a range query data structure according to the attribute values of the tag data.
S102: and according to the query range of the attribute value in the input tag data range query request, querying in a range query data structure to obtain the attribute value of the target tag data.
S103: and determining the storage address of the target tag data in the time sequence index file according to the corresponding relation.
S104: and acquiring the target label data according to the storage address of the target label data.
S105: and outputting the target label data.
The reference data of the infixdb database is shown in table 1:
table 1 reference data table for infiluxdb database
A range query is a common database query for querying tuples whose attributes take values within a specified range.
As shown in fig. 2, the infixdb database is a time-series database that does not support tag data range queries. In the infiluxdb database, searching for data requires finding a corresponding value (value) in a data area through a data source key (series key) in an index area, where the related basic concepts are as follows:
and (2) Series: the data source is a mapping formed by corresponding data source key values (SerisKey) and values (value), and the query process is to search corresponding values (value) by the data source key values (SerisKey).
Measurement: the name of the table.
Tag: the tag data and the inherent attribute of the data object can only be stored in string format, and the range query is not supported.
field: the dynamic property of the data supports string, float, integer or bootean types, is associated with a timestamp, and supports range query.
timing and map: a timestamp for marking the time of field generation.
In order to take both query efficiency and memory constraints into consideration, the Time Series database generally stores the memory Index by using a Time Series Index (TSI) file, so that data source key values (Series keys) of all data sources meeting query conditions can be found out in a short Time, and the problems that the memory Index is limited by the size of a memory and the recovery Time is long are solved.
The structure of the time-series index file may be as shown in fig. 3. The data source Block (Series Block), the Tag Block (Tag Block), the table name Block (measure Block), the Index File (Index File), and the like are stored in different memory blocks (blocks). When data is inquired, firstly, a corresponding table name (measure) in a command statement is found in a table name Block (measure Block), a Tag Block (Tag Block) corresponding to Tag data (Tag) of a table is obtained, all data source identifications (Series IDs) meeting the requirements of inquiry conditions are found according to the Tag data (Tag) and values in the inquiry conditions, and a corresponding data source Block (Series Block) is found according to the data source identifications (Series IDs).
The range query method for tag data provided in the embodiment of the present application constructs a range query data structure for implementing range query on attribute values of tag data on the basis of the time series index file, where the range query data structure may be a data structure supporting range query on data therein, for example, a balanced multi-way tree structure.
The attribute value of the tag data used for constructing the range query data structure may be a value of all attributes of the tag data, or a value of a part of attributes of the tag data, and may be specifically set according to the type of the tag data and the query habit of the user. It can be understood that, for the convenience of query, the attribute values of the tag data used to construct the range query data structure are not all the data contents of the tag data, and the tag data stored in the time series index file of the time series database is all the contents of the tag data.
Meanwhile, since the storage locations of the tag data in the time-series index file may not be ordered according to the content of the tag data, and the tag data can be obtained from the time-series index file only after the storage address of the tag data is determined, the range query method of the tag data provided in the embodiment of the present application determines in advance the correspondence between the attribute value of the tag data and the storage address of the tag data in the time-series index file, and establishes the range query data structure according to the attribute value of each tag data, so that after the range query of the tag data based on the attribute value is implemented on the range query data structure, the tag data can be further mapped to the time-series index file to obtain the target tag data.
Based on the method for querying the range of the tag data provided by the embodiment of the application, a user inputs the query range of the attribute value of the target tag data, so that the attribute value in the query range can be found in the range query data structure, and then the storage address of the target tag data in the time series index file of the time series database can be determined according to the pre-established corresponding relation, so that the target tag data is taken out of the time series index file of the time series database.
The method for querying the range of the tag data provided by the embodiment of the application establishes the range query data structure according to the attribute value of the tag data in advance, and establishes the corresponding relation between the attribute value of the tag data and the storage address of the tag data in the time series index file of the time series database, so that the query scope of the attribute value in the input tag data scope query request can be obtained, after the attribute value of the target tag data is obtained by query in the range query data structure, determining the storage address of the target tag data in the time series index file according to the corresponding relation, and then target label data is obtained and output, so that the blank of the function of performing range query on the label data in some time sequence databases is filled, the user can conveniently perform range query on the label data in the time sequence databases, and the functions of the time sequence databases are enriched.
Example two
In the above embodiment, the addition and deletion of the tag data in the range query data structure may be performed along with the creation and deletion of the tag data in the time-series index file, or may be performed along with the "hotness" of the tag data.
On the basis of the foregoing embodiment, in the method for querying a range of tag data provided in the embodiment of the present application, step S102: according to the query range of the attribute value in the input tag data range query request, the attribute value of the target tag data is obtained by querying in a range query data structure, and the method specifically comprises the following steps:
judging whether the attribute value of the target tag data exists in the range query data structure or not according to the query range in the tag data range query request;
if yes, recording the query time of the target tag data, executing a query range of attribute values in the input tag data range query request, and querying in a range query data structure to obtain the target tag data;
if not, recording the query time of the target tag data, adding the attribute dereferencing of the target tag data into the range query data structure, executing the query range of the attribute dereferencing according to the input tag data range query request, and querying in the range query data structure to obtain the target tag data.
By recording the query time of the target tag data to the query time of the target tag data, the query data structure management based on the query time range can be further realized. Since the range query is usually a query of a continuous piece of tag data, for example, query of tag data with reference numbers 1-18, if a certain piece of tag data cannot be queried from the temporary data structure, attribute values of the tag data need to be added to the range query data structure for the user to perform the next query. In addition, target tag data that is not described in the range query data structure but is separately queried by the user is also added to the range query data structure.
In order to avoid that the query time data occupies more storage space and is convenient to check, the query time of the target tag data is recorded, which specifically comprises the following steps: and recording the latest query time of the target tag data. And covering the latest query time with the last query time after the tag data is queried once. For the first queried tag data, its query time is created.
Specifically, all tag data in the range query data structure and the corresponding latest query time thereof can be recorded by another data structure, and member update is performed along with update and pruning of the range query data structure.
Then, it is determined whether the range query data structure has the attribute value of the target tag data, which may specifically be:
judging whether the latest query time of the target tag data is recorded;
if yes, determining an attribute value with target tag data in the range query data structure;
and if not, determining that the attribute value of the target tag data is not contained in the range query data structure.
Based on the query time of the tag data, the 'hot degree' of the tag data can be further determined, so that the elimination of non-hot data and the creation of new hot data are performed on the range query data structure, the range query data structure is simplified, and the efficiency of each query is improved.
The method for querying the range of the tag data provided by the embodiment of the application further includes:
judging whether the range query data structure reaches a trimming condition;
and if so, pruning the range query data structure according to the latest query time of each label data in the range query data structure.
Wherein, the trimming condition can be: when the time interval between the latest query time and the current time of the tag data exceeds a first threshold value, deleting the tag data from the range query data structure; and deleting the tag data with the latest query time after the number of members in the range query data structure exceeds a second threshold value, and the like. And a pruning rule of the range query data structure can be established according to the actual application scene.
And for the label data which is not recorded in the range query data structure and is to be queried by the user, adding the label data into the range query data structure according to the creation rule of the range query data structure.
EXAMPLE III
Fig. 4 is a schematic diagram of a B + tree structure according to an embodiment of the present disclosure; fig. 5 is a schematic diagram of a query method of a B + tree according to an embodiment of the present application; fig. 6 is a schematic diagram of a HashSet structure according to an embodiment of the present application; FIG. 7 is a schematic diagram of the calculation of the HashSet structure; fig. 8 is a functional topology diagram of a query of a tagged data range in a time series database according to an embodiment of the present application; fig. 9 is a schematic diagram illustrating a process of eliminating non-hotspot tag data according to an embodiment of the present disclosure; fig. 10 is a schematic structural diagram of a tag data range query method according to an embodiment of the present application.
On the basis of the foregoing embodiment, in the range query method for tag data provided in the embodiment of the present application, the range query data structure may specifically adopt a B tree structure, and preferably adopts a B + tree structure. The B + tree structure can greatly improve the efficiency of carrying out range query on the label field because leaf nodes of the B + tree structure have a linked list structure.
As shown in fig. 4, a 3 rd order B + tree is created using the tag data. Assuming that Tag1-Tag18 are Tag fields of Tag data which are increased in the dictionary order, but actually the corresponding Tag blocks (Tag blocks) in the time series index file may be unordered or discontinuous, the storage address data of the Tag data is stored in the leaf nodes and physically corresponds to the Tag blocks (Tag blocks) in the time series index file, and on the basis of the above, the process of building the tree has completed the dictionary order sorting of the Tag data.
Fig. 5 shows a query process based on a B + tree structure, which specifically includes:
and (3) tree searching: starting from the root node of the B + tree, the range lower limit value of the label data is searched.
Searching a linked list: and traversing the linked list along the position of the lower limit value in the linked list, and inquiring leaf nodes in all inquiry ranges.
And modifying the query time: and updating the query time of the target tag data.
On the basis of the foregoing embodiment, in the range query method for tag data provided in the embodiment of the present application, the query time of the target tag data may be stored specifically by using a HashSet structure. Recording the latest query time of the target tag data, specifically: and storing the attribute value of the target tag data and the latest query time of the target tag data through a HashSet structure.
On the basis of the foregoing embodiment, in the method for querying a range of tag data provided in the embodiment of the present application, a manner of recording a latest query time for target tag data may specifically be: and storing the attribute value of the target label data and the latest query time of the target label data through a Hash table structure. The Hash table structure has the characteristic that repeated key values (keys) are not allowed to be inserted into the Hash table, and the newly inserted key values (keys) will overwrite the old ones.
For example, one of the Hash table structures may be employed: the Hashset structure stores the attribute value of the target label data and the latest query time of the target label data. As shown in fig. 6, the tag data (tags) stored in the B + tree and the latest query time (time) are recorded through the HashSet structure. The bloom filter can be used for easily and efficiently judging whether a tag data is stored in a HashSet structure, and the core of the method is to realize a super-large bit array and a plurality of hash functions. As shown in fig. 7, assuming that the length of the bit array is m and the number of hash functions is k, the bit array is initialized first, and each bit therein is set to 0. For each element in the set, the element is mapped sequentially by k hash functions, each mapping generates a hash value corresponding to a point on the bit array, and then the position corresponding to the bit array is marked as 1. When inquiring whether W element exists in the set, the same method maps W to k points on the bit array through Hash mapping. If one of the k points is not 1, it can be determined that the element must not exist in the set.
The query statement is analyzed to obtain a query range and related tag fields, a HashSet structure is used for marking whether tag data are inserted into a B + tree or not, and query time is recorded, and the method mainly comprises the following steps:
tag member record: recording whether the tag data is inserted into a temporary index of a B + tree structure, adopting a set structure, simultaneously recording a timestamp of the tag data which is inquired recently, if the query is carried out for the first time, recording the insertion time as the timestamp in a set, wherein the set can be used for checking whether a tag data node is in the B + tree or not, and can also be used for scanning whether the node belongs to non-hot data or not in a elimination mechanism.
Temporarily querying the index by the tag data: the range query data structure is composed of a B + tree, and can realize range query, deletion and sequencing of the tag data fields according to the field sequence, and the constructed B + tree structure supports range query.
Based on the structures shown in fig. 2, 4, and 5, as shown in fig. 8, the method for querying the range of tag data provided in the embodiment of the present application specifically includes:
and loading the time series index file into a memory from a disk, traversing the time series index file, storing the value of the tag data and the time for inserting the tag data into the B + tree by using a HashSet structure, and simultaneously inserting the tag data into the B + tree. It should be noted that, at this time, the time series index file is not changed, and only the storage address of the Tag Block (Tag Block) in the time series index file is stored in the B + tree. And meanwhile, monitoring the storage time corresponding to each label data in the HashSet structure by using a timing task, and deleting the node in the B + tree if the time is long. It should also be noted that only nodes in the B + tree are deleted, not the Tag Block (Tag Block) in the time series index file.
In the query process of the tag data based on the B + tree, the query time for updating the target tag data is specifically to add or update the position corresponding to the tag: time key value pair in the HashSet structure, and due to the characteristics of the HashSet structure, the new query time covers the last query time.
As shown in fig. 9, the elimination mechanism based on the HashSet structure scans tag data by defining hot tag data and using a timing task, deletes tag data nodes that do not belong to the hot data in the B + tree, and maintains the query efficiency of the tree; by defining culling thresholdsT thd System time to time sequence databaseTTime of last query with all tag dataT 0 Making a difference to obtain a deltaT=T-T 0 If ΔT >T thd Then the tag data is considered as non-hot data, and the rest of the tag data is defined as hot data.
The timing task can be realized by using a background thread, the scanning object is a timestamp mark of a tag data object recorded in a HashSet structure, if the definition of non-hot data is met, the leaf node in the B + tree structure and the member in the HashSet structure are deleted, and the query efficiency of the B + tree is maintained.
In addition, the user may be provided with the ability to make parameter adjustments. E.g. in the deselection mechanism, related parameters such as deselection thresholdT thd The configurable state should be maintained, i.e. exposed to user-modifiable by means of terminal or instruction.
On the basis of the contents shown in fig. 4 to 9, as shown in fig. 10, the method for querying the range of tag data provided in the embodiment of the present application specifically includes:
s201: and (5) inquiring the instruction.
S202: the tag field/query scope is parsed.
S303: judging whether the index is an empty tree; if yes, go to step S204; if not, the process proceeds to step S206.
S204: and (4) creating a record table (HashSet structure), inserting label data and recording query time.
S205: a temporary index (B + tree) is created.
S206: judging whether the tag data is in a HashSet structure; if yes, go to step S207; if not, the process proceeds to step S208.
S207: b + tree range query. And then proceeds to step S210.
S208: and inserting the label data into a record table, and recording the query time.
S209: prune the temporary index (B + tree). And then proceeds to step S207.
S210: and updating the latest query time of the target tag data. And then proceeds to step S216.
S211: and (5) timing the task.
S212: the latest query time of the tag data is detected.
S213: judging whether the query interval time is greater than a culling threshold value; if yes, go to step S214; if not, step S213 is repeated.
S214: the record of the tag data is deleted in the record table.
S215: the tag data node is deleted in the temporary index (B + tree). And then proceeds to step S216.
S216: the temporary index (B + tree) is in-memory.
In a specific implementation, the method for querying the scope of the tag data provided by the embodiment of the present application can be implemented by predefining a creation/indexing component, a data querying component, and a elimination mechanism executing component.
Steps S202 to S206 and S208 to S209 are implementation blocks of the creation/indexing component, steps S207 and S210 are implementation blocks of the data query component, and steps S211 to S215 are implementation blocks of the elimination mechanism execution component.
On the basis of the above detailed description of the embodiments corresponding to the method for querying the range of the tag data, the application also discloses a device, an apparatus and a storage medium for querying the range of the tag data corresponding to the method.
Example four
Fig. 11 is a schematic structural diagram of an apparatus for querying a range of tag data according to an embodiment of the present application.
As shown in fig. 11, the apparatus for querying a range of tag data according to the embodiment of the present application includes:
a creating unit 301, configured to determine in advance a correspondence between an attribute value of tag data and a storage address of the tag data in a time series index file of a time series database, and create a range query data structure according to the attribute value of each tag data;
a query unit 302, configured to query in a range query data structure to obtain an attribute value of target tag data according to a query range of the attribute value in an input tag data range query request;
a determining unit 303, configured to determine, according to the corresponding relationship, a storage address of the target tag data in the time-series index file;
an obtaining unit 304, configured to obtain target tag data according to a storage address of the target tag data;
an output unit 305 for outputting the target tag data.
Further, the querying unit 302 specifically includes:
the first judgment subunit is used for judging whether the attribute dereferencing of the target tag data exists in the range inquiry data structure or not according to the inquiry range in the tag data range inquiry request; if yes, entering a first recording subunit; if not, entering a second recording subunit;
the first recording subunit is used for recording the query time of the target tag data, then executing the query range of attribute values in the input tag data range query request, and querying in the range query data structure to obtain the target tag data;
and the second recording subunit is used for recording the query time of the target tag data, adding the attribute dereferencing of the target tag data into the range query data structure, executing the query range of the attribute dereferencing according to the input tag data range query request, and querying the range query data structure to obtain the target tag data.
Further, the creating unit 301 further includes:
the second judging subunit is used for judging whether the range query data structure meets the trimming condition; if yes, entering a trimming subunit;
and the pruning subunit is used for pruning the range query data structure according to the latest query time of each label data in the range query data structure.
Since the embodiments of the apparatus portion and the method portion correspond to each other, please refer to the description of the embodiments of the method portion for the embodiments of the apparatus portion, which is not repeated here.
Fig. 12 is a schematic structural diagram of a device for querying a range of tag data according to an embodiment of the present application.
As shown in fig. 12, the apparatus for querying a range of tag data provided in an embodiment of the present application includes:
a memory 410 for storing instructions, the instructions comprising the steps of the tag data range query method according to any one of the above embodiments;
a processor 420 for executing the instructions.
Among other things, processor 420 may include one or more processing cores, such as a 3-core processor, an 8-core processor, and so forth. The processor 420 may be implemented in at least one hardware form of a Digital Signal Processing (DSP), a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), or a Programmable Logic Array (PLA). Processor 420 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a central Processing unit (cpu); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 420 may be integrated with an image processor GPU (graphics Processing unit) that is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, processor 420 may also include an Artificial Intelligence (AI) (artificial intelligence) processor for processing computational operations related to machine learning.
Memory 410 may include one or more storage media, which may be non-transitory. Memory 410 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 410 is at least used for storing the following computer program 411, wherein after the computer program 411 is loaded and executed by the processor 420, the relevant steps in the method for querying the range of tag data disclosed in any of the foregoing embodiments can be implemented. In addition, the resources stored by the memory 410 may also include an operating system 412, data 413, and the like, and the storage may be transient storage or permanent storage. Operating system 412 may be Windows, among others. The data 413 may include, but is not limited to, data involved in the above-described methods.
In some embodiments, the tag data range query device may further include a display 430, a power source 440, a communication interface 450, an input output interface 460, a sensor 470, and a communication bus 480.
Those skilled in the art will appreciate that the structure shown in FIG. 4 does not constitute a limitation of the scope query device for tag data and may include more or fewer components than those shown.
The device for querying the range of the tag data provided by the embodiment of the application comprises the memory and the processor, and when the processor executes the program stored in the memory, the method for querying the range of the tag data can be realized, and the effect is the same as that of the method for querying the range of the tag data.
It should be noted that the above-described embodiments of the apparatus and device are merely illustrative, for example, the division of modules is only one division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of modules or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or modules, and may be in an electrical, mechanical or other form. Modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may be stored in a storage medium. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods described in the embodiments of the present application, or all or part of the technical solutions.
To this end, the present application further provides a storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the range query method such as tag data.
The storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory ROM (Read-Only Memory), a random Access Memory ram (random Access Memory), a magnetic disk, or an optical disk.
The storage medium provided in this embodiment includes a computer program capable of implementing the steps of the method for querying a range of tag data as described above when executed by a processor, and the same effect is achieved.
The above description details a method, an apparatus, a device and a storage medium for querying a range of tag data provided by the present application. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device, the equipment and the storage medium disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A method for querying a range of tag data, comprising:
the method comprises the steps of determining a corresponding relation between attribute values of tag data and storage addresses of the tag data in a time series index file of a time series database in advance, and establishing a range query data structure according to the attribute values of the tag data;
inquiring in the range inquiry data structure according to the inquiry range of the attribute value in the input label data range inquiry request to obtain the attribute value of the target label data;
determining the storage address of the target tag data in the time sequence index file according to the corresponding relation;
acquiring the target label data according to the storage address of the target label data;
and outputting the target label data.
2. The range query method according to claim 1, wherein the querying for the attribute value of the target tag data in the range query data structure according to the query range of the attribute value in the input tag data range query request specifically includes:
judging whether the attribute dereferencing of the target tag data exists in the range inquiry data structure or not according to the inquiry range of the attribute dereferencing in the tag data range inquiry request;
if yes, recording the query time of the target tag data, executing the query range of attribute values in the query request according to the input tag data range, and querying in the range query data structure to obtain the target tag data;
if not, recording the query time of the target label data, adding the attribute dereferencing of the target label data into the range query data structure, executing the query range of the attribute dereferencing in the input label data range query request, and querying in the range query data structure to obtain the target label data.
3. The range query method of claim 2, wherein the recording of the query time for the target tag data specifically comprises:
and recording the latest query time of the target tag data.
4. The range query method of claim 3, wherein the recording the latest query time for the target tag data specifically comprises:
and storing the attribute value of the target label data and the latest query time of the target label data through a Hash table structure.
5. The range query method according to claim 2, wherein the determining whether the range query data structure has the attribute value of the target tag data specifically includes:
judging whether the latest query time of the target label data is recorded;
if so, determining that the attribute value of the target tag data exists in the range query data structure;
and if not, determining that the attribute value of the target tag data is not contained in the range query data structure.
6. The scope query method of claim 2, further comprising:
judging whether the range query data structure reaches a pruning condition;
and if so, pruning the range query data structure according to the latest query time of each tag data in the range query data structure.
7. The range query method of any one of claims 1 to 6, wherein the range query data structure is specifically a B + tree structure.
8. An apparatus for querying a range of tag data, comprising:
the system comprises a creating unit, a searching unit and a searching unit, wherein the creating unit is used for predetermining the corresponding relation between the attribute value of the tag data and the storage address of the tag data in the time sequence index file of the time sequence database and creating a range query data structure according to the attribute value of each tag data;
the query unit is used for querying in the range query data structure to obtain the attribute value of the target tag data according to the query range of the attribute value in the input tag data range query request;
a determining unit, configured to determine, according to the correspondence, a storage address of the target tag data in the time series index file;
the acquisition unit is used for acquiring the target label data according to the storage address of the target label data;
an output unit for outputting the target tag data.
9. A range query device for tag data, comprising:
a memory for storing instructions comprising the steps of the method of range query of tag data according to any one of claims 1 to 7;
a processor to execute the instructions.
10. A storage medium having stored thereon a computer program, characterized in that the computer program, when being executed by a processor, carries out the steps of the method for range query of tag data according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110984987.5A CN113434557B (en) | 2021-08-26 | 2021-08-26 | Method, device, equipment and storage medium for querying range of label data |
| PCT/CN2021/127321 WO2023024247A1 (en) | 2021-08-26 | 2021-10-29 | Range query method, apparatus and device for tag data, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110984987.5A CN113434557B (en) | 2021-08-26 | 2021-08-26 | Method, device, equipment and storage medium for querying range of label data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113434557A true CN113434557A (en) | 2021-09-24 |
| CN113434557B CN113434557B (en) | 2021-12-17 |
Family
ID=77797928
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110984987.5A Active CN113434557B (en) | 2021-08-26 | 2021-08-26 | Method, device, equipment and storage medium for querying range of label data |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113434557B (en) |
| WO (1) | WO2023024247A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023024247A1 (en) * | 2021-08-26 | 2023-03-02 | 苏州浪潮智能科技有限公司 | Range query method, apparatus and device for tag data, and storage medium |
| CN117573944A (en) * | 2024-01-17 | 2024-02-20 | 深圳十沣科技有限公司 | Data retrieval method, device, equipment and storage medium |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117235120A (en) * | 2023-11-09 | 2023-12-15 | 支付宝(杭州)信息技术有限公司 | Hypergraph data storage and query method and device with time sequence characteristics |
| CN117560228B (en) * | 2024-01-10 | 2024-03-19 | 西安电子科技大学杭州研究院 | Real-time attack detection method and system for flow tracing graph based on label and graph alignment |
| CN117591577B (en) * | 2024-01-18 | 2024-05-03 | 中核武汉核电运行技术股份有限公司 | Nuclear power historical data comparison method and system based on file storage |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160092484A1 (en) * | 2014-09-26 | 2016-03-31 | International Business Machines Corporation | Data ingestion stager for time series database |
| US20210034587A1 (en) * | 2019-08-02 | 2021-02-04 | Timescale, Inc. | Type-specific compression in database systems |
| CN112818013A (en) * | 2021-01-27 | 2021-05-18 | 北京百度网讯科技有限公司 | Time sequence database query optimization method, device, equipment and storage medium |
| CN112861022A (en) * | 2021-02-01 | 2021-05-28 | 杭州全拓科技有限公司 | Artificial intelligence-based personnel activity big data record query method |
| CN113111098A (en) * | 2021-06-11 | 2021-07-13 | 阿里云计算有限公司 | Method and device for detecting query of time sequence data and time sequence database system |
| CN113297135A (en) * | 2021-02-10 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103853752A (en) * | 2012-11-30 | 2014-06-11 | 国际商业机器公司 | Method and device for managing time series database |
| WO2020047584A1 (en) * | 2018-09-04 | 2020-03-12 | Future Grid Pty Ltd | Method and system for indexing of time-series data |
| US20200117763A1 (en) * | 2018-10-15 | 2020-04-16 | Ca, Inc. | Relational interval tree with distinct borders |
| WO2020159397A1 (en) * | 2019-01-30 | 2020-08-06 | Siemens Aktiengesellschaft | Method and computerized device for processing numeric time series data |
| CN109902088A (en) * | 2019-02-13 | 2019-06-18 | 北京航空航天大学 | A kind of data index method towards streaming time series data |
| CN113254451B (en) * | 2021-06-01 | 2022-04-19 | 北京城市网邻信息技术有限公司 | Data index construction method and device, electronic equipment and storage medium |
| CN113434557B (en) * | 2021-08-26 | 2021-12-17 | 苏州浪潮智能科技有限公司 | Method, device, equipment and storage medium for querying range of label data |
-
2021
- 2021-08-26 CN CN202110984987.5A patent/CN113434557B/en active Active
- 2021-10-29 WO PCT/CN2021/127321 patent/WO2023024247A1/en unknown
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160092484A1 (en) * | 2014-09-26 | 2016-03-31 | International Business Machines Corporation | Data ingestion stager for time series database |
| US20210034587A1 (en) * | 2019-08-02 | 2021-02-04 | Timescale, Inc. | Type-specific compression in database systems |
| CN112818013A (en) * | 2021-01-27 | 2021-05-18 | 北京百度网讯科技有限公司 | Time sequence database query optimization method, device, equipment and storage medium |
| CN112861022A (en) * | 2021-02-01 | 2021-05-28 | 杭州全拓科技有限公司 | Artificial intelligence-based personnel activity big data record query method |
| CN113297135A (en) * | 2021-02-10 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN113111098A (en) * | 2021-06-11 | 2021-07-13 | 阿里云计算有限公司 | Method and device for detecting query of time sequence data and time sequence database system |
Non-Patent Citations (1)
| Title |
|---|
| 范欣欣: "《时序数据库技术体系——InfluxDB 多维查询之倒排索引》", 《HTTP://HBASEFLY.COM/2018/02/09/TIMESERIES-DATABASE-5/》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023024247A1 (en) * | 2021-08-26 | 2023-03-02 | 苏州浪潮智能科技有限公司 | Range query method, apparatus and device for tag data, and storage medium |
| CN117573944A (en) * | 2024-01-17 | 2024-02-20 | 深圳十沣科技有限公司 | Data retrieval method, device, equipment and storage medium |
| CN117573944B (en) * | 2024-01-17 | 2024-04-02 | 深圳十沣科技有限公司 | Data retrieval method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113434557B (en) | 2021-12-17 |
| WO2023024247A1 (en) | 2023-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113434557B (en) | Method, device, equipment and storage medium for querying range of label data | |
| CN105893349B (en) | Classification tag match mapping method and device | |
| CN108388559B (en) | Named entity identification method and system under geographic space application and computer program | |
| CN109582831B (en) | Graph database management system supporting unstructured data storage and query | |
| CN105373541A (en) | Processing method and system for data operation request of database | |
| CN110109910A (en) | Data processing method and system, electronic equipment and computer readable storage medium | |
| CN113918733B (en) | Data processing system for acquiring target knowledge graph | |
| CN109815240A (en) | For managing method, apparatus, equipment and the storage medium of index | |
| CN109144633A (en) | Data sharing method, device, equipment and the storage medium of active window | |
| CN111984649A (en) | Data index searching method and device and related equipment | |
| CN108376054B (en) | Processing method and device for indexing identification data | |
| CN108959584B (en) | Community structure-based method and device for processing graph data | |
| CN111488341A (en) | Database index management method and device and electronic equipment | |
| CN115017161A (en) | Method, device and application for updating tree data structure by combining virtual DOM | |
| CN113609433B (en) | Method and device for determining arithmetic layout, electronic equipment and storage medium | |
| US20140025691A1 (en) | Method and apparatus for dynamic filtering of an object graph in a content repository | |
| CN110825846B (en) | Data processing method and device | |
| CN114048219A (en) | Graph database updating method and device | |
| CN107169128A (en) | A kind of method that geographical location information is bound to generic data model | |
| CN105095276B (en) | Method and device for mining maximum repetitive sequence | |
| JP5430436B2 (en) | Information storage search method and information storage search program | |
| CN106599076B (en) | Forum guide map generation method and device | |
| CN111506588A (en) | Method and device for extracting key information of electronic document | |
| Fan et al. | Web data extraction based on visual information and partial tree alignment | |
| CN117909301B (en) | Index-based object query method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |