CN111046081B - Industrial time sequence data access method and system - Google Patents
Industrial time sequence data access method and system Download PDFInfo
- Publication number
- CN111046081B CN111046081B CN201911241194.3A CN201911241194A CN111046081B CN 111046081 B CN111046081 B CN 111046081B CN 201911241194 A CN201911241194 A CN 201911241194A CN 111046081 B CN111046081 B CN 111046081B
- Authority
- CN
- China
- Prior art keywords
- time sequence
- time
- database
- data
- industrial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/04—Manufacturing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The application provides an access method and system of industrial time sequence data, wherein the method comprises the following steps: when an industrial time sequence data query request is received, analyzing a data identifier and a query time range from the industrial time sequence data query request; selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the inquiry time range; if the time sequence database to be accessed is a real-time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the real-time sequence database; and if the time sequence database to be accessed is a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the historical time sequence database. In the application, the performance of the query function can be improved in the above manner, and the history time sequence database can save the storage space by storing the compressed data.
Description
Technical Field
The present application relates to the field of timing databases, and in particular, to a method and system for accessing industrial timing data.
Background
Along with the coming of industrial big data age, the rapidly growing internet of things equipment and sensors generate a large amount of time sequence data, the traditional relational database cannot meet the storage query requirement of the time sequence data, and the time sequence database gradually becomes the first choice of industrial big data storage and becomes a core database for supporting various businesses.
As a main function of the time series database, it is necessary to optimize the performance thereof, but how to optimize the performance of the query function becomes a problem.
Disclosure of Invention
In order to solve the above technical problems, an embodiment of the present application provides a method and a system for accessing industrial time series data, so as to achieve the purpose of improving the performance of a query function, and the technical scheme is as follows:
an access method for industrial time series data, comprising:
when an industrial time sequence data query request is received, analyzing a data identifier and a query time range from the industrial time sequence data query request;
selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the inquiry time range; the plurality of different timing databases includes: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with reporting time in a first set time length before the current time, the historical time sequence database stores industrial time sequence data with reporting time in a second set time length before the current time, and the second set time length is longer than the first set time length;
if the time sequence database to be accessed is a real-time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the real-time sequence database;
and if the time sequence database to be accessed is a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the historical time sequence database.
Preferably, the real-time sequence database includes: the first real-time database sub-database and the second real-time database sub-database;
the first real-time database stores industrial time sequence data with reporting time being in a first set duration before the current time, and the second real-time database stores industrial time sequence data with reporting time being in a time before the current time.
Preferably, the process of storing industrial time series data in the plurality of different time series databases respectively includes:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
if yes, storing the target industrial time sequence data into the time sequence database.
Preferably, the method further comprises:
and modifying the data types supported by the time sequence database according to modification instructions aiming at the data types supported by the time sequence database.
Preferably, in the case that the time series database is a historical time series database, the storing the target industrial time series data to the time series database includes:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that each field of the industrial time sequence data of the same kind is respectively stored in different rows.
Preferably, in the case that the time series database is a history time series database based on HBase, the storing the target industrial time series data to the time series database includes:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same kind of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data in a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
An access system for industrial time series data, comprising: a plurality of different timing databases, the plurality of different timing databases comprising: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with reporting time in a first set time length before the current time, the historical time sequence database stores industrial time sequence data with reporting time in a second set time length before the current time, and the second set time length is longer than the first set time length; an analysis module; a selection module; the system comprises a first query module and a second query module;
the analysis module is used for analyzing the data identification and the query time range from the industrial time sequence data query request when the industrial time sequence data query request is received;
the selection module is used for selecting one time sequence database to be accessed from the plurality of different time sequence databases according to the inquiry time range;
the first query module is configured to query industrial time sequence data matched with the data identifier and the query time range from the real-time sequence database if the time sequence database to be accessed is a real-time sequence database;
and the second query module is used for querying industrial time sequence data matched with the data identification and the query time range from the historical time sequence database if the time sequence database to be accessed is the historical time sequence database.
Preferably, the real-time sequence database includes: the first real-time database sub-database and the second real-time database sub-database;
the first real-time database stores industrial time sequence data with reporting time being in a first set duration before the current time, and the second real-time database stores industrial time sequence data with reporting time being in a time before the current time.
Preferably, the method further comprises:
a storage module for:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
if yes, storing the target industrial time sequence data into the time sequence database.
Preferably, the method further comprises:
and the modification module is used for modifying the data types supported by the time sequence database according to modification instructions aiming at the data types supported by the time sequence database.
Preferably, in the case that the time sequence database is a historical time sequence database, the storage module is specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that each field of the industrial time sequence data of the same kind is respectively stored in different rows.
Preferably, in the case that the timing database is an HBase-based historical timing database, the storage module is specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same kind of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data in a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
Compared with the prior art, the application has the beneficial effects that:
in the application, the time sequence database is divided into a real-time sequence database and a historical time sequence database, the real-time sequence database is less and newer than the historical time sequence database in comparison with the data stored in the historical time sequence database, in the case that the time sequence database to be accessed is the real-time sequence database and the historical time sequence database, the query is preferentially performed from the real-time sequence database, and the data stored in the real-time sequence database is smaller, so that the workload of the query can be reduced, the query efficiency is improved, and the performance of the query function of the time sequence database is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a flowchart of an embodiment 1 of an industrial time series data access method provided by the application;
FIG. 2 is a flow chart of storing industrial time series data in a plurality of different time series databases according to the present application;
FIG. 3 is a flow chart of storing industrial time series data in a plurality of different time series databases according to the present application;
FIG. 4 is a flow chart of storing industrial time series data in a plurality of different time series databases according to the present application;
fig. 5 is a schematic logic structure diagram of an industrial time series data access system according to the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The embodiment of the application discloses an access method of industrial time sequence data, which comprises the following steps: when an industrial time sequence data query request is received, analyzing a data identifier and a query time range from the industrial time sequence data query request; determining a time sequence database to be accessed according to the inquiry time range, wherein the time sequence database to be accessed comprises: the system comprises a real-time sequence database or a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with reporting time in a first set time length before the current time, the historical time sequence database stores industrial time sequence data with reporting time in a second set time length before the current time, and the second set time length is longer than the first set time length; if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the real-time sequence database; and if the time sequence database to be accessed is a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the historical time sequence database. In the application, the performance of the query function can be improved.
Next, an industrial time series data access method disclosed in the embodiment of the present application is described, where the industrial time series data access method is applied to a time series database, as shown in fig. 1, and a flowchart of an industrial time series data access method embodiment 1 provided in the present application may include the following steps:
and S11, when an industrial time sequence data query request is received, analyzing a data identifier and a query time range from the industrial time sequence data query request.
The data identification can be understood as: unique identification of a class of industrial time series data. Each type of industrial time sequence data has unique identification, and is convenient to store and search. For example, industrial timing data of the motor includes: the power consumption and the power consumption, the identification of the power consumption is 1, and the identification of the power consumption is 2. The power consumption and the power consumption are distinguished by different identifiers, and are convenient to search.
Industrial time series data of corresponding time periods can be queried from the time series database according to the query time range. If the inquiry time range is from the start time to the end time, the industrial time sequence data with the time period from the start time to the end time can be inquired from the time sequence database.
Step S12, selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the inquiry time range.
In this embodiment, a plurality of different time series databases may be provided. The different time sequence databases are respectively provided with query interfaces for query. The plurality of different timing databases may include: a real-time timing database and a historical timing database. The real-time sequence database and the historical time sequence database comprise industrial time sequence data in different time ranges. For example, the real-time sequence database stores industrial time sequence data with reporting time within a first set time length before the current time, the history time sequence database stores industrial time sequence data with reporting time within a second set time length before the current time, and the second set time length is longer than the first set time length.
The reporting time can be understood as: time of industrial time series data reporting to time series database.
Because each time sequence database comprises industrial time sequence data in different time ranges, each time sequence database corresponds to different time ranges, one time sequence database can be selected from a plurality of different time sequence databases to be accessed according to the inquiry time range.
The first set duration can be flexibly set according to actual needs, for example, the first set duration can be set to be 1 hour.
Of course, the second set duration may also be flexibly set according to actual needs, for example, the second set duration may be set as a difference between the current time and the starting time.
And step S13, if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the real-time sequence database.
If the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, industrial time sequence data matched with the data identification and the query time range are queried from the real-time sequence database, and compared with the industrial time sequence data matched with the data identification and the query time range are queried from the historical time sequence database, the workload of query is small, and the query efficiency is high.
In this embodiment, the real-time timing database may include, but is not limited to: a first real-time database sub-database and a second real-time database sub-database.
The first real-time database stores industrial time sequence data with reporting time being in a first set duration before the current time, and the second real-time database stores industrial time sequence data with reporting time being in a time before the current time.
It can be understood that the second real-time database stores the industrial time sequence data with reporting time being the time before the current time, i.e. the second real-time database stores the latest reported industrial time sequence data.
In the case that the time sequence database to be accessed is the second real-time database sub-database, the first real-time database sub-database and the history time sequence database, the time sequence database to be accessed is preferentially selected from the second real-time database sub-database
Step S14, if the time sequence database to be accessed is a historical time sequence database, industrial time sequence data matched with the data identification and the query time range is queried from the historical time sequence database.
Because the industrial time sequence data in the real-time sequence database is less than the data in the historical time sequence database, the industrial time sequence data matched with the data identification and the query time range needs to be queried from the historical time sequence database under the condition that the time sequence database to be accessed is determined to be the historical time sequence database, and the reliability of query is ensured.
In embodiment 2 of the present application, a process of storing industrial time series data in a plurality of different time series databases in embodiment 1 is described, and as shown in fig. 2, the method may include the following steps:
step S21, when a storage request for target industrial time series data is received, analyzing the data type from the storage request.
Data types may include, but are not limited to: string, boolean, long or Float type.
And S22, judging whether the data type is the data type supported by the time sequence database.
It will be appreciated that the types of data supported by the timing database may be preset, e.g., the timing database support String and Boolean may be preset.
If not, executing step S23; if yes, go to step S24.
In this embodiment, the timing database provides a modification interface for receiving modification instructions for the data types supported by the timing database.
The modification procedure of the data type supported by the time sequence database can comprise the following steps: and modifying the data types supported by the time sequence database according to modification instructions aiming at the data types supported by the time sequence database.
Step S23, discarding the target industrial time sequence data.
In the case where the parsed data type is not a data type supported by the time series database, the time series database cannot parse the data normally and store it, and thus it is necessary to discard the target industrial time series data.
Step S24, storing the target industrial time sequence data into the time sequence database.
The process of storing the target industrial time series data to the time series database may include:
s241, the target industrial time sequence data is sent to a message middleware, and the message middleware stores the target industrial time sequence data within a set storage duration.
The message middleware stores the target industrial time sequence data in a set storage duration, so that external access can be facilitated.
S242, the time sequence database reads the target industrial time sequence data from the message middleware and stores the target industrial time sequence data.
As embodiment 3 of the present application, mainly, a refinement of the process of storing industrial time series data in a plurality of different time series databases described in embodiment 2 above is provided, and as shown in fig. 3, the method may include, but is not limited to, the following steps:
step S31, when a storage request for target industrial time series data is received, the data type is analyzed from the storage request.
And step S32, judging whether the data type is the data type supported by the time sequence database.
If not, executing step S33; if not, step S34 is performed.
And step S33, discarding the target industrial time sequence data.
The detailed procedure of steps S31-S33 can be referred to in the related description of steps S21-S23 in embodiment 2, and will not be described herein.
And step S34, compressing the target industrial time sequence data to obtain compressed data under the condition that the time sequence database is a historical time sequence database.
Step S35, storing the compressed data in the history time sequence database according to the storage mode that each field of the same kind of industrial time sequence data is respectively stored in different rows.
According to the storage mode that each field of the industrial time sequence data of the same kind is respectively stored in different rows, the compressed data is stored in the historical time sequence database, so that each field of the industrial time sequence data can be respectively stored, and the query efficiency is improved when a certain field of the industrial time sequence data is queried.
And, store the said compressed data in the said historical time sequence database, can save the memory space of the historical time sequence database.
Steps S34-S35 are a specific embodiment of step S24 in example 2.
As embodiment 4 of the present application, mainly, a refinement of the process of storing industrial time series data in a plurality of different time series databases described in embodiment 2 above is provided, and as shown in fig. 4, the method may include, but is not limited to, the following steps:
step S41, when a storage request for target industrial time series data is received, analyzing the data type from the storage request.
Step S42, judging whether the data type is the data type supported by the time sequence database.
If not, executing step S43; if yes, go to step S44.
Step S43, discarding the target industrial time sequence data.
And S44, compressing the target industrial time sequence data to obtain compressed data when the time sequence database is a historical time sequence database based on HBase.
In this embodiment, the history time sequence database based on HBase may be set to be automatically created monthly for the data table, the data of the month is stored in the same data table, and the data table may be composed of a row key internal ID, a field name, a field type, and a time stamp (hour). The keys of the time series data of the same kind and the same hour may be set to be the same.
Step S45, storing the compressed data in a first column cluster in the history time sequence database based on HBase according to a storage mode that data of each field of the industrial time sequence data of the same kind are respectively stored in different columns.
And step S46, performing aggregation calculation on the target industrial time sequence data in a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
Aggregate calculations for the target industrial time series data within a third set period of time may include, but are not limited to: and calculating the maximum value, the minimum value, the average value, the total point number, the first value or the last value of the target industrial time sequence data in the third set time length, and respectively returning a calculation result.
The third set period of time may be, but is not limited to: and 1 hour.
In this embodiment, the method of individually storing the aggregate calculation result may improve query efficiency of the aggregate result.
Next, the system for accessing industrial time series data provided by the present application will be described, and the system for accessing industrial time series data described below and the method for accessing industrial time series data described above can be referred to correspondingly.
Referring to fig. 5, the access system for industrial time series data includes: a plurality of different time series databases DB, a parsing module 11, a selection module 12, a first query module 13 and a second query module 14.
The plurality of different time series databases DB are denoted DB1, DB2, …, DBn, respectively. n is an integer greater than 1.
The plurality of different timing database DBs includes: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with reporting time in a first set time length before the current time, the historical time sequence database stores industrial time sequence data with reporting time in a second set time length before the current time, and the second set time length is longer than the first set time length; an analysis module; a selection module; the system comprises a first query module and a second query module;
the parsing module 11 is configured to parse a data identifier and a query time range from the industrial time-series data query request when the industrial time-series data query request is received;
the selecting module 12 is configured to select one of the plurality of different time sequence databases as a time sequence database to be accessed according to the query time range;
the first query module 13 is configured to query industrial time sequence data matching the data identifier and the query time range from the real-time sequence database if the time sequence database to be accessed is a real-time sequence database;
the second query module 14 is configured to query the historical time sequence database for industrial time sequence data matching the data identifier and the query time range if the time sequence database to be accessed is the historical time sequence database.
In this embodiment, the real-time sequence database may include: the first real-time database sub-database and the second real-time database sub-database;
the first real-time database stores industrial time sequence data with reporting time being in a first set duration before the current time, and the second real-time database stores industrial time sequence data with reporting time being in a time before the current time.
In this embodiment, the access system for industrial time series data may further include:
a storage module for:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
if yes, the target industrial time sequence data is stored in the time sequence database DB.
In this embodiment, the access system for industrial time series data may further include:
and the modification module is used for modifying the data types supported by the time sequence database DB according to modification instructions aiming at the data types supported by the time sequence database DB.
In this embodiment, in the case where the time-series database DB is a history time-series database, the storage module may be specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that each field of the industrial time sequence data of the same kind is respectively stored in different rows.
In this embodiment, in the case where the timing database DB is an HBase-based history timing database, the storage module may be specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same kind of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data in a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
It should be noted that, in each embodiment, the differences from the other embodiments are emphasized, and the same similar parts between the embodiments are referred to each other. For the apparatus class embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference is made to the description of the method embodiments for relevant points.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in the same piece or pieces of software and/or hardware when implementing the present application.
From the above description of embodiments, it will be apparent to those skilled in the art that the present application may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present application.
The above describes in detail an industrial time series data access method and system provided by the present application, and specific examples are applied to illustrate the principles and embodiments of the present application, and the above examples are only used to help understand the method and core ideas of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (7)
1. An access method for industrial time series data, comprising:
when an industrial time sequence data query request is received, analyzing a data identifier and a query time range from the industrial time sequence data query request;
selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the inquiry time range; the plurality of different timing databases includes: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with reporting time in a first set time length before the current time, the historical time sequence database stores industrial time sequence data with reporting time in a second set time length before the current time, and the second set time length is longer than the first set time length;
if the time sequence database to be accessed is a real-time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the real-time sequence database;
if the time sequence database to be accessed is a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the historical time sequence database;
if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the real-time sequence database;
the process of storing industrial time sequence data by the plurality of different time sequence databases respectively comprises the following steps: when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request; judging whether the data type is the data type supported by the time sequence database; if not, discarding the target industrial time sequence data; if yes, storing the target industrial time sequence data into the time sequence database;
in the case where the timing database is an HBase-based historical timing database, the storing the target industrial timing data to the timing database comprises: compressing the target industrial time sequence data to obtain compressed data; storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same kind of industrial time sequence data are respectively stored in different columns; and carrying out aggregation calculation on the target industrial time sequence data in the third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database, wherein the aggregation calculation comprises calculation of a maximum value, a minimum value, an average value, a total point number, a first value or a last value.
2. The method of claim 1, wherein the real-time timing database comprises: the first real-time database sub-database and the second real-time database sub-database;
the first real-time database stores industrial time sequence data with reporting time being in a first set duration before the current time, and the second real-time database stores industrial time sequence data with reporting time being in a time before the current time;
and under the condition that the time sequence database to be accessed is a second real-time database sub-database, a first real-time database sub-database and a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiring time range from the second real-time database sub-database.
3. The method of claim 1, wherein the process of modifying the data types supported by the timing database comprises:
and modifying the data types supported by the time sequence database according to modification instructions aiming at the data types supported by the time sequence database.
4. The method of claim 1, wherein, in the event that the timing database is a historical timing database, the storing the target industrial timing data to the timing database comprises:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that each field of the industrial time sequence data of the same kind is respectively stored in different rows.
5. An industrial time series data access system, comprising: a plurality of different timing databases, the plurality of different timing databases comprising: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with reporting time in a first set time length before the current time, the historical time sequence database stores industrial time sequence data with reporting time in a second set time length before the current time, and the second set time length is longer than the first set time length; an analysis module; a selection module; the system comprises a first query module, a second query module and a third query module;
the analysis module is used for analyzing the data identification and the query time range from the industrial time sequence data query request when the industrial time sequence data query request is received;
the selection module is used for selecting one time sequence database to be accessed from the plurality of different time sequence databases according to the inquiry time range;
the first query module is configured to query industrial time sequence data matched with the data identifier and the query time range from the real-time sequence database if the time sequence database to be accessed is a real-time sequence database;
the second query module is configured to query industrial time sequence data matching the data identifier and the query time range from the historical time sequence database if the time sequence database to be accessed is the historical time sequence database;
the third query module is configured to query industrial time sequence data matched with the data identifier and the query time range from the real-time sequence database if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database;
a storage module for: when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request; judging whether the data type is the data type supported by the time sequence database; if not, discarding the target industrial time sequence data; if yes, storing the target industrial time sequence data into the time sequence database;
in the case that the timing database DB is an HBase-based historical timing database, the storage module may be specifically configured to: compressing the target industrial time sequence data to obtain compressed data; storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same kind of industrial time sequence data are respectively stored in different columns; and carrying out aggregation calculation on the target industrial time sequence data in the third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database, wherein the aggregation calculation comprises calculation of a maximum value, a minimum value, an average value, a total point number, a first value or a last value.
6. The system of claim 5, wherein the system further comprises: a fourth query module;
the real-time timing database comprises: the first real-time database sub-database and the second real-time database sub-database;
the first real-time database stores industrial time sequence data with reporting time being in a first set duration before the current time, and the second real-time database stores industrial time sequence data with reporting time being in a time before the current time
The fourth query module is configured to query industrial time sequence data matching the data identifier and the query time range from the second real-time database if the time sequence database to be accessed is the second real-time database sub-database, the first real-time database sub-database and the historical time sequence database.
7. The system of claim 5, further comprising:
and the modification module is used for modifying the data types supported by the time sequence database according to modification instructions aiming at the data types supported by the time sequence database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911241194.3A CN111046081B (en) | 2019-12-06 | 2019-12-06 | Industrial time sequence data access method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911241194.3A CN111046081B (en) | 2019-12-06 | 2019-12-06 | Industrial time sequence data access method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111046081A CN111046081A (en) | 2020-04-21 |
| CN111046081B true CN111046081B (en) | 2023-09-12 |
Family
ID=70234971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911241194.3A Active CN111046081B (en) | 2019-12-06 | 2019-12-06 | Industrial time sequence data access method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111046081B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111752245B (en) * | 2020-06-30 | 2021-06-25 | 北京和利时智能技术有限公司 | Data access and processing method, device, equipment and readable storage medium |
| CN112231531A (en) * | 2020-09-15 | 2021-01-15 | 山东浪潮通软信息科技有限公司 | Data display method, equipment and medium based on openstb |
| CN112487047A (en) * | 2020-11-02 | 2021-03-12 | 广州明珞装备股份有限公司 | Spare part consumption data query method, system and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101984421A (en) * | 2010-09-03 | 2011-03-09 | 中国科学院软件研究所 | Real-time database based meteorological element data organization type-in and querying method |
| CN103605805A (en) * | 2013-12-09 | 2014-02-26 | 冶金自动化研究设计院 | Storage method of massive time series data |
| CN106383864A (en) * | 2016-09-02 | 2017-02-08 | 北京百度网讯科技有限公司 | Query request processing method and apparatus for time series database |
| CN106815318A (en) * | 2016-12-24 | 2017-06-09 | 上海七牛信息技术有限公司 | A kind of clustering method and system of time series database |
| CN108268217A (en) * | 2018-01-10 | 2018-07-10 | 北京航天云路有限公司 | A kind of bedding storage method based on the cold and hot classification of time series data |
| CN108446329A (en) * | 2018-02-13 | 2018-08-24 | 北京工业大数据创新中心有限公司 | Adaptive databases partition method and system towards industrial time series database |
| CN109766394A (en) * | 2018-12-19 | 2019-05-17 | 上海前隆信息科技有限公司 | Metrology platform data query method and device, readable storage medium storing program for executing and terminal |
| CN110209651A (en) * | 2019-05-16 | 2019-09-06 | 南京华盾电力信息安全测评有限公司 | Time series database system based on MongoDB |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN110445671A (en) * | 2019-06-27 | 2019-11-12 | 浪潮思科网络科技有限公司 | A kind of network flow monitoring method based on SDN |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6029951B2 (en) * | 2012-11-27 | 2016-11-24 | 株式会社日立製作所 | Time series database setting automatic generation method, setting automatic generation system and monitoring server |
| CN104516894B (en) * | 2013-09-27 | 2018-08-17 | 国际商业机器公司 | Method and apparatus for managing time series databases |
| US10642851B2 (en) * | 2016-05-26 | 2020-05-05 | Salesforce.Com, Inc. | Caching time series data |
| US10817524B2 (en) * | 2017-04-10 | 2020-10-27 | Servicenow, Inc. | Systems and methods for querying time series data |
-
2019
- 2019-12-06 CN CN201911241194.3A patent/CN111046081B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101984421A (en) * | 2010-09-03 | 2011-03-09 | 中国科学院软件研究所 | Real-time database based meteorological element data organization type-in and querying method |
| CN103605805A (en) * | 2013-12-09 | 2014-02-26 | 冶金自动化研究设计院 | Storage method of massive time series data |
| CN106383864A (en) * | 2016-09-02 | 2017-02-08 | 北京百度网讯科技有限公司 | Query request processing method and apparatus for time series database |
| CN106815318A (en) * | 2016-12-24 | 2017-06-09 | 上海七牛信息技术有限公司 | A kind of clustering method and system of time series database |
| CN108268217A (en) * | 2018-01-10 | 2018-07-10 | 北京航天云路有限公司 | A kind of bedding storage method based on the cold and hot classification of time series data |
| CN108446329A (en) * | 2018-02-13 | 2018-08-24 | 北京工业大数据创新中心有限公司 | Adaptive databases partition method and system towards industrial time series database |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN109766394A (en) * | 2018-12-19 | 2019-05-17 | 上海前隆信息科技有限公司 | Metrology platform data query method and device, readable storage medium storing program for executing and terminal |
| CN110209651A (en) * | 2019-05-16 | 2019-09-06 | 南京华盾电力信息安全测评有限公司 | Time series database system based on MongoDB |
| CN110445671A (en) * | 2019-06-27 | 2019-11-12 | 浪潮思科网络科技有限公司 | A kind of network flow monitoring method based on SDN |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111046081A (en) | 2020-04-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111046081B (en) | Industrial time sequence data access method and system | |
| US7672935B2 (en) | Automatic index creation based on unindexed search evaluation | |
| CN106407360B (en) | Data processing method and device | |
| JP2004518226A (en) | Database system and query optimizer | |
| US8620946B2 (en) | Storage and searching of temporal entity information | |
| CN111813782A (en) | Time sequence data storage method and device | |
| CN111008200A (en) | Data query method and device and server | |
| CN110245145A (en) | Structure synchronization method and apparatus of the relevant database to Hadoop database | |
| CN112256715A (en) | Index updating method and device, electronic equipment and storage medium | |
| CN111752945B (en) | Time sequence database data interaction method and system based on container and hierarchical model | |
| CN102236707A (en) | Method, device and system for obtaining update data of database | |
| CN112364033B (en) | Data retrieval system | |
| CN109299089A (en) | The calculating and storage method and calculating of a kind of label data of drawing a portrait and storage system | |
| CN108694213A (en) | Generation method, search method and the device of journal file | |
| CN112181994B (en) | Method, device and medium for refreshing distributed memory database of operation and maintenance big data | |
| US20190303467A1 (en) | Data aggregation data structure | |
| CN109800233A (en) | A kind of big data fusion searching method | |
| CN115391424A (en) | Database query processing method, storage medium and computer equipment | |
| CN112765200A (en) | Data query method and device based on Elasticissearch | |
| CN112434063A (en) | Monitoring data processing method based on time sequence database | |
| CN108289031B (en) | Home broadband network fault diagnosis method and device | |
| CN112199396A (en) | Industrial Internet identification query method and system facing MES | |
| CN105447137A (en) | Algorithm for retrieving data with same master-slave relation from big data on the basis of relational database | |
| CN105335489B (en) | A kind of accident alarming enquiring and optimizing method | |
| CN111327721B (en) | IP address positioning method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210401 Address after: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant after: NINGBO HELISHI INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant before: NINGBO HELISHI INTELLIGENT TECHNOLOGY Co.,Ltd. Applicant before: BEIJING HOLLYSYS AUTOMATION & DRIVE Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| CB02 | Change of applicant information |
Address after: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant after: Hollysys Kauber Technology Co.,Ltd. Address before: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant before: NINGBO HELISHI INTELLIGENT TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |