CN111046081A - Access method and system for industrial time sequence data - Google Patents
Access method and system for industrial time sequence data Download PDFInfo
- Publication number
- CN111046081A CN111046081A CN201911241194.3A CN201911241194A CN111046081A CN 111046081 A CN111046081 A CN 111046081A CN 201911241194 A CN201911241194 A CN 201911241194A CN 111046081 A CN111046081 A CN 111046081A
- Authority
- CN
- China
- Prior art keywords
- time
- data
- time sequence
- industrial
- sequence database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000002776 aggregation Effects 0.000 claims description 18
- 238000004220 aggregation Methods 0.000 claims description 18
- 230000004048 modification Effects 0.000 claims description 11
- 238000012986 modification Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24554—Unary operations; Data partitioning operations
- G06F16/24556—Aggregation; Duplicate elimination
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2477—Temporal data queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/04—Manufacturing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- Probability & Statistics with Applications (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Manufacturing & Machinery (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides an access method and system of industrial time sequence data, wherein the method comprises the following steps: when an industrial time series data query request is received, analyzing a data identifier and a query time range from the industrial time series data query request; selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the query time range; if the time sequence database to be accessed is a real-time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiry time range from the real-time sequence database; and if the time sequence database to be accessed is a historical time sequence database, inquiring the industrial time sequence data matched with the data identification and the inquiry time range from the historical time sequence database. In the application, the performance of the query function can be improved through the above mode, and the storage space can be saved through the storage of the compressed data in the historical time sequence database.
Description
Technical Field
The application relates to the technical field of time sequence databases, in particular to an access method and system of industrial time sequence data.
Background
With the advent of the industrial big data era, a great amount of time sequence data are generated by the rapidly-increased internet of things equipment and sensors, the traditional relational database cannot meet the storage and query requirements of the time sequence data, and the time sequence database gradually becomes the first choice for industrial big data storage and becomes a core database for supporting various services.
The query function is a main function of the time-series database, and it is necessary to optimize the performance of the time-series database, but how to optimize the performance of the query function becomes a problem.
Disclosure of Invention
In order to solve the foregoing technical problems, embodiments of the present application provide an access method and system for industrial time series data, so as to achieve the purpose of improving the performance of a query function, and the technical solution is as follows:
an access method of industrial time series data comprises the following steps:
when an industrial time series data query request is received, analyzing a data identifier and a query time range from the industrial time series data query request;
selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the query time range; the plurality of different timing databases comprises: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, and the second set time length is greater than the first set time length;
if the time sequence database to be accessed is a real-time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiry time range from the real-time sequence database;
and if the time sequence database to be accessed is a historical time sequence database, inquiring the industrial time sequence data matched with the data identification and the inquiry time range from the historical time sequence database.
Preferably, the real-time series database includes: a first real-time data sub-base and a second real-time data sub-base;
the first real-time data sub-base stores industrial time sequence data with the reporting time within a first set time length before the current time, and the second real-time data sub-base stores industrial time sequence data with the reporting time before the current time.
Preferably, the plurality of different time series databases respectively store processes of industrial time series data, and the processes include:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
and if so, storing the target industrial time sequence data into the time sequence database.
Preferably, the method further comprises:
and modifying the data types supported by the time sequence database according to a modification instruction aiming at the data types supported by the time sequence database.
Preferably, in a case that the time-series database is a historical time-series database, the storing the target industrial time-series data into the time-series database includes:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that all fields of the same type of industrial time sequence data are respectively stored in different rows.
Preferably, in the case that the time-series database is an HBase-based historical time-series database, the storing the target industrial time-series data into the time-series database includes:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same type of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data within a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
An access system for industrial time series data, comprising: a plurality of different time series databases, the plurality of different time series databases comprising: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, and the second set time length is greater than the first set time length; an analysis module; a selection module; a first query module and a second query module;
the analysis module is used for analyzing a data identifier and an inquiry time range from the industrial time series data inquiry request when the industrial time series data inquiry request is received;
the selection module is used for selecting one of the plurality of different time sequence databases as a time sequence database to be accessed according to the query time range;
the first query module is used for querying industrial time sequence data matched with the data identifier and the query time range from the real-time sequence database if the time sequence database to be accessed is a real-time sequence database;
and the second query module is used for querying the industrial time sequence data matched with the data identifier and the query time range from the historical time sequence database if the time sequence database to be accessed is the historical time sequence database.
Preferably, the real-time series database includes: a first real-time data sub-base and a second real-time data sub-base;
the first real-time data sub-base stores industrial time sequence data with the reporting time within a first set time length before the current time, and the second real-time data sub-base stores industrial time sequence data with the reporting time before the current time.
Preferably, the method further comprises the following steps:
a storage module to:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
and if so, storing the target industrial time sequence data into the time sequence database.
Preferably, the method further comprises the following steps:
and the modification module is used for modifying the data types supported by the time sequence database according to the modification instruction aiming at the data types supported by the time sequence database.
Preferably, in a case that the time-series database is a historical time-series database, the storage module is specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that all fields of the same type of industrial time sequence data are respectively stored in different rows.
Preferably, in the case that the time-series database is an HBase-based historical time-series database, the storage module is specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same type of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data within a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
Compared with the prior art, the beneficial effect of this application is:
in the application, the time sequence database is divided into a real-time sequence database and a historical time sequence database, the real-time sequence database stores less and new data compared with the historical time sequence database, the query is preferentially carried out from the real-time sequence database under the condition that the time sequence database to be accessed is the real-time sequence database and the historical time sequence database, and the data stored in the real-time sequence database is small, so that the query workload can be reduced, the query efficiency is improved, and the performance of the query function of the time sequence database is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive labor.
FIG. 1 is a flow chart of an embodiment 1 of an access method for industrial time series data provided by the present application;
FIG. 2 is a flow chart of a plurality of different time series databases respectively storing industrial time series data provided by the present application;
FIG. 3 is a flow chart of another method provided herein for separately storing industrial time series data in a plurality of different time series databases;
FIG. 4 is a flow chart of another method for storing industrial time series data in a plurality of different time series databases;
FIG. 5 is a schematic diagram of a logic structure of an access system for industrial time series data provided by the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application discloses an access method of industrial time sequence data, which comprises the following steps: when an industrial time series data query request is received, analyzing a data identifier and a query time range from the industrial time series data query request; determining a time sequence database to be accessed according to the query time range, wherein the time sequence database to be accessed comprises: the system comprises a real-time sequence database or a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, and the second set time length is greater than the first set time length; if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiry time range from the real-time sequence database; and if the time sequence database to be accessed is a historical time sequence database, inquiring the industrial time sequence data matched with the data identification and the inquiry time range from the historical time sequence database. In the present application, the performance of the query function can be improved.
Next, a method for accessing industrial time series data disclosed in an embodiment of the present application is described, where the method for accessing industrial time series data is applied to a time series database, and as shown in fig. 1, a flowchart of an embodiment 1 of the method for accessing industrial time series data provided by the present application may include the following steps:
and step S11, when an industrial time series data query request is received, analyzing a data identifier and a query time range from the industrial time series data query request.
The data identification can be understood as: unique identification of a certain type of industrial time series data. Each type of industrial time sequence data has unique identification, so that the industrial time sequence data is convenient to store and search. For example, industrial timing data for an electric machine includes: power usage and power consumption, the power usage being identified as 1 and the power consumption being identified as 2. The electricity consumption and the power consumption are distinguished by different identifiers and are convenient to search.
And inquiring the industrial time sequence data of the corresponding time period from the time sequence database according to the inquiry time range. If the query time range is from the start time to the end time, the industrial time series data with the time period from the start time to the end time can be queried from the time series database.
And step S12, selecting one of the time sequence databases as a time sequence database to be accessed according to the query time range.
In this embodiment, a plurality of different time series databases may be provided. And different time sequence databases are respectively provided with an inquiry interface for inquiry. The plurality of different time series databases may include: a real-time timing database and a historical timing database. The real-time sequence database and the historical time sequence database comprise industrial time sequence data in different time ranges. If so, the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, and the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, wherein the second set time length is greater than the first set time length.
The reporting time can be understood as: the industrial time series data is reported to the time of the time series database.
Because each time sequence database comprises the industrial time sequence data in different time ranges, and each time sequence database respectively corresponds to different time ranges, one time sequence database can be selected from a plurality of different time sequence databases to be used as the time sequence database to be accessed according to the query time range.
The first set time period can be flexibly set according to actual needs, for example, the first set time period can be set to 1 hour.
Of course, the second set time duration may also be flexibly set according to actual needs, for example, the second set time duration may be set as a difference between the current time and the starting time.
And step S13, if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiry time range from the real-time sequence database.
And if the time sequence database to be accessed is a real-time sequence database and a historical time sequence database, inquiring the industrial time sequence data matched with the data identification and the inquiry time range from the real-time sequence database, wherein the inquiring workload is small and the inquiring efficiency is high compared with the inquiring of the industrial time sequence data matched with the data identification and the inquiry time range from the historical time sequence database.
In this embodiment, the real-time sequence database may include, but is not limited to: a first real-time data sub-base and a second real-time data sub-base.
The first real-time data sub-base stores industrial time sequence data with the reporting time within a first set time length before the current time, and the second real-time data sub-base stores industrial time sequence data with the reporting time before the current time.
It can be understood that the second real-time database stores the industrial time sequence data of the previous time when the reporting time is the current time, that is, the second real-time database stores the latest reported industrial time sequence data.
When the time sequence database to be accessed is the second real-time database, the first real-time database and the historical time sequence database, the time sequence database to be accessed can be preferentially selected from the second real-time database
And step S14, if the time sequence database to be accessed is a historical time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiry time range from the historical time sequence database.
Because the industrial time sequence data in the real-time sequence database is less than the data in the historical time sequence database, under the condition that the time sequence database to be accessed is determined to be the historical time sequence database, the industrial time sequence data matched with the data identifier and the query time range needs to be queried from the historical time sequence database, and the query reliability is ensured.
In embodiment 2 of the present application, a process of storing industrial time series data in a plurality of different time series databases in embodiment 1 is described, as shown in fig. 2, the process may include the following steps:
and step S21, when a storage request aiming at the target industrial time series data is received, analyzing the data type from the storage request.
The data types may include, but are not limited to: string, Boolean, Long or Float types.
And step S22, judging whether the data type is the data type supported by the time sequence database.
It is understood that the types of data supported by the sequential database may be predetermined, e.g., the sequential database may be predetermined to support String and Boolean.
If not, go to step S23; if yes, go to step S24.
In this embodiment, the time sequence database provides a modification interface for receiving a modification instruction for a data type supported by the time sequence database.
The process of modifying the data types supported by the time series database can comprise the following steps: and modifying the data types supported by the time sequence database according to a modification instruction aiming at the data types supported by the time sequence database.
And step S23, discarding the target industrial time series data.
When the analyzed data type is not the data type supported by the time sequence database, the time sequence database cannot normally analyze and store the data, and therefore the target industrial time sequence data needs to be discarded.
And step S24, storing the target industrial time sequence data into the time sequence database.
The process of storing the target industrial timing data to the timing database may include:
and S241, sending the target industrial time sequence data to a message middleware, and storing the target industrial time sequence data in a set storage time length by the message middleware.
The message middleware stores the target industrial time sequence data within the set storage duration, and can facilitate external access.
And S242, reading the target industrial time sequence data from the message middleware by the time sequence database and storing the target industrial time sequence data.
As embodiment 3 of the present application, it is mainly a refinement of the process of storing industrial time series data in a plurality of different time series databases described in the above embodiment 2, as shown in fig. 3, the method may include, but is not limited to, the following steps:
and step S31, when a storage request aiming at the target industrial time series data is received, analyzing the data type from the storage request.
And step S32, judging whether the data type is the data type supported by the time sequence database.
If not, go to step S33; if not, step S34 is executed.
And step S33, discarding the target industrial time series data.
The detailed procedures of steps S31-S33 can be referred to the related descriptions of steps S21-S23 in embodiment 2, and are not described herein again.
And step S34, compressing the target industrial time sequence data to obtain compressed data under the condition that the time sequence database is a historical time sequence database.
And step S35, storing the compressed data in the historical time sequence database according to the storage mode that each field of the same type of industrial time sequence data is stored in different rows.
According to the storage mode that all fields of the same type of industrial time sequence data are stored in different rows, the compressed data are stored in the historical time sequence database, and all fields of the industrial time sequence data can be stored respectively, so that the query efficiency is improved when a certain field of the industrial time sequence data is queried.
And the compressed data is stored in the historical time sequence database, so that the storage space of the historical time sequence database can be saved.
Steps S34-S35 are a specific implementation of step S24 in example 2.
As embodiment 4 of the present application, it is mainly a refinement of the process of storing industrial time series data in a plurality of different time series databases described in the above embodiment 2, as shown in fig. 4, the method may include, but is not limited to, the following steps:
and step S41, when a storage request aiming at the target industrial time series data is received, analyzing the data type from the storage request.
And step S42, judging whether the data type is the data type supported by the time sequence database.
If not, go to step S43; if yes, go to step S44.
And step S43, discarding the target industrial time series data.
And step S44, compressing the target industrial time sequence data to obtain compressed data under the condition that the time sequence database is a historical time sequence database based on HBase.
In this embodiment, the HBase-based historical timing database may be configured such that a data table is automatically created every month, the data of the month is stored in the same data table, and the data table may be composed of a row key internal ID, a field name, a field type, and a time stamp (hour). Row keys of time series data of the same kind and the same hour can be set to be the same.
And step S45, storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same type of industrial time sequence data are respectively stored in different columns.
And step S46, performing aggregation calculation on the target industrial time sequence data within a third set time length to obtain an aggregation result, and storing the aggregation result in a second cluster in the HBase-based historical time sequence database.
Performing an aggregation calculation on the target industrial time series data within a third set time length may include, but is not limited to: and calculating the maximum value, the minimum value, the average value, the total point number, the first value or the last value of the target industrial time sequence data within the third set time length, and respectively returning a calculation result.
The third set period of time may be, but is not limited to: for 1 hour.
In this embodiment, the aggregation calculation result is stored separately, so that the query efficiency of the aggregation result can be improved.
Next, the access system of the industrial time series data provided by the present application is described, and the access system of the industrial time series data described below and the access method of the industrial time series data described above are referred to correspondingly.
Referring to fig. 5, the system for accessing industrial time series data includes: a plurality of different time series databases DB, a parsing module 11, a selection module 12, a first query module 13 and a second query module 14.
The plurality of different time series databases DB are denoted as DB1, DB2, …, DBn, respectively. n is an integer greater than 1.
The plurality of different time-series database DBs include: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, and the second set time length is greater than the first set time length; an analysis module; a selection module; a first query module and a second query module;
the analysis module 11 is configured to, when receiving an industrial time series data query request, analyze a data identifier and a query time range from the industrial time series data query request;
the selection module 12 is configured to select one of the multiple different time sequence databases as a time sequence database to be accessed according to the query time range;
the first query module 13 is configured to query, if the time sequence database to be accessed is a real-time sequence database, industrial time sequence data matched with the data identifier and the query time range from the real-time sequence database;
the second query module 14 is configured to query, if the time sequence database to be accessed is a historical time sequence database, industrial time sequence data matched with the data identifier and the query time range from the historical time sequence database.
In this embodiment, the real-time sequence database may include: a first real-time data sub-base and a second real-time data sub-base;
the first real-time data sub-base stores industrial time sequence data with the reporting time within a first set time length before the current time, and the second real-time data sub-base stores industrial time sequence data with the reporting time before the current time.
In this embodiment, the access system for the industrial time series data may further include:
a storage module to:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
and if so, storing the target industrial time sequence data to the time sequence database DB.
In this embodiment, the access system for the industrial time series data may further include:
and the modification module is used for modifying the data types supported by the time sequence database DB according to the modification instruction aiming at the data types supported by the time sequence database DB.
In this embodiment, in a case that the time sequence database DB is a historical time sequence database, the storage module may be specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that all fields of the same type of industrial time sequence data are respectively stored in different rows.
In this embodiment, in a case that the time sequence database DB is an HBase-based historical time sequence database, the storage module may be specifically configured to:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same type of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data within a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
It should be noted that each embodiment is mainly described as a difference from the other embodiments, and the same and similar parts between the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
The above detailed description is provided for the method and system for accessing industrial time series data, and specific examples are applied herein to explain the principles and embodiments of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. An access method for industrial time series data, comprising:
when an industrial time series data query request is received, analyzing a data identifier and a query time range from the industrial time series data query request;
selecting one time sequence database from a plurality of different time sequence databases as a time sequence database to be accessed according to the query time range; the plurality of different timing databases comprises: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, and the second set time length is greater than the first set time length;
if the time sequence database to be accessed is a real-time sequence database, inquiring industrial time sequence data matched with the data identification and the inquiry time range from the real-time sequence database;
and if the time sequence database to be accessed is a historical time sequence database, inquiring the industrial time sequence data matched with the data identification and the inquiry time range from the historical time sequence database.
2. The method of claim 1, wherein the real-time series database comprises: a first real-time data sub-base and a second real-time data sub-base;
the first real-time data sub-base stores industrial time sequence data with the reporting time within a first set time length before the current time, and the second real-time data sub-base stores industrial time sequence data with the reporting time before the current time.
3. The method of claim 1, wherein the plurality of different time series databases each store a process of industrial time series data, comprising:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
and if so, storing the target industrial time sequence data into the time sequence database.
4. The method of claim 3, wherein the process of modifying the data types supported by the time series database comprises:
and modifying the data types supported by the time sequence database according to a modification instruction aiming at the data types supported by the time sequence database.
5. The method of claim 3, wherein in the case that the time series database is a historical time series database, the storing the target industrial time series data to the time series database comprises:
compressing the target industrial time sequence data to obtain compressed data;
and storing the compressed data in the historical time sequence database according to a storage mode that all fields of the same type of industrial time sequence data are respectively stored in different rows.
6. The method according to claim 3, wherein in the case that the time sequence database is an HBase-based historical time sequence database, the storing the target industrial time sequence data to the time sequence database comprises:
compressing the target industrial time sequence data to obtain compressed data;
storing the compressed data in a first column cluster in the HBase-based historical time sequence database according to a storage mode that data of each field of the same type of industrial time sequence data are respectively stored in different columns;
and performing aggregation calculation on the target industrial time sequence data within a third set time length to obtain an aggregation result, and storing the aggregation result in a second column cluster in the HBase-based historical time sequence database.
7. An access system for industrial time series data, comprising: a plurality of different time series databases, the plurality of different time series databases comprising: the system comprises a real-time sequence database and a historical time sequence database, wherein the real-time sequence database stores industrial time sequence data with the reporting time within a first set time length before the current time, the historical time sequence database stores industrial time sequence data with the reporting time within a second set time length before the current time, and the second set time length is greater than the first set time length; an analysis module; a selection module; a first query module and a second query module;
the analysis module is used for analyzing a data identifier and an inquiry time range from the industrial time series data inquiry request when the industrial time series data inquiry request is received;
the selection module is used for selecting one of the plurality of different time sequence databases as a time sequence database to be accessed according to the query time range;
the first query module is used for querying industrial time sequence data matched with the data identifier and the query time range from the real-time sequence database if the time sequence database to be accessed is a real-time sequence database;
and the second query module is used for querying the industrial time sequence data matched with the data identifier and the query time range from the historical time sequence database if the time sequence database to be accessed is the historical time sequence database.
8. The system of claim 7, wherein the real-time series database comprises: a first real-time data sub-base and a second real-time data sub-base;
the first real-time data sub-base stores industrial time sequence data with the reporting time within a first set time length before the current time, and the second real-time data sub-base stores industrial time sequence data with the reporting time before the current time.
9. The system of claim 7, further comprising:
a storage module to:
when a storage request aiming at target industrial time sequence data is received, analyzing a data type from the storage request;
judging whether the data type is the data type supported by the time sequence database;
if not, discarding the target industrial time sequence data;
and if so, storing the target industrial time sequence data into the time sequence database.
10. The system of claim 9, further comprising:
and the modification module is used for modifying the data types supported by the time sequence database according to the modification instruction aiming at the data types supported by the time sequence database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911241194.3A CN111046081B (en) | 2019-12-06 | 2019-12-06 | Industrial time sequence data access method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911241194.3A CN111046081B (en) | 2019-12-06 | 2019-12-06 | Industrial time sequence data access method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111046081A true CN111046081A (en) | 2020-04-21 |
| CN111046081B CN111046081B (en) | 2023-09-12 |
Family
ID=70234971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911241194.3A Active CN111046081B (en) | 2019-12-06 | 2019-12-06 | Industrial time sequence data access method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111046081B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111752245A (en) * | 2020-06-30 | 2020-10-09 | 北京和利时智能技术有限公司 | Data access and processing method, device, equipment and readable storage medium |
| CN112231531A (en) * | 2020-09-15 | 2021-01-15 | 山东浪潮通软信息科技有限公司 | Data display method, equipment and medium based on openstb |
| CN112487047A (en) * | 2020-11-02 | 2021-03-12 | 广州明珞装备股份有限公司 | Spare part consumption data query method, system and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101984421A (en) * | 2010-09-03 | 2011-03-09 | 中国科学院软件研究所 | Real-time database based meteorological element data organization type-in and querying method |
| CN103605805A (en) * | 2013-12-09 | 2014-02-26 | 冶金自动化研究设计院 | Storage method of massive time series data |
| US20140149466A1 (en) * | 2012-11-27 | 2014-05-29 | Hitachi, Ltd. | Time-series database setup automatic generation method, setup automatic generation system and monitoring server |
| US20150095381A1 (en) * | 2013-09-27 | 2015-04-02 | International Business Machines Corporation | Method and apparatus for managing time series database |
| CN106383864A (en) * | 2016-09-02 | 2017-02-08 | 北京百度网讯科技有限公司 | Query request processing method and apparatus for time series database |
| CN106815318A (en) * | 2016-12-24 | 2017-06-09 | 上海七牛信息技术有限公司 | A kind of clustering method and system of time series database |
| US20170344614A1 (en) * | 2016-05-26 | 2017-11-30 | Salesforce.Com, Inc. | Caching time series data |
| CN108268217A (en) * | 2018-01-10 | 2018-07-10 | 北京航天云路有限公司 | A kind of bedding storage method based on the cold and hot classification of time series data |
| CN108446329A (en) * | 2018-02-13 | 2018-08-24 | 北京工业大数据创新中心有限公司 | Adaptive databases partition method and system towards industrial time series database |
| US20180293284A1 (en) * | 2017-04-10 | 2018-10-11 | Servicenow, Inc. | Systems and methods for querying time series data |
| CN109766394A (en) * | 2018-12-19 | 2019-05-17 | 上海前隆信息科技有限公司 | Metrology platform data query method and device, readable storage medium storing program for executing and terminal |
| CN110209651A (en) * | 2019-05-16 | 2019-09-06 | 南京华盾电力信息安全测评有限公司 | Time series database system based on MongoDB |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN110445671A (en) * | 2019-06-27 | 2019-11-12 | 浪潮思科网络科技有限公司 | A kind of network flow monitoring method based on SDN |
-
2019
- 2019-12-06 CN CN201911241194.3A patent/CN111046081B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101984421A (en) * | 2010-09-03 | 2011-03-09 | 中国科学院软件研究所 | Real-time database based meteorological element data organization type-in and querying method |
| US20140149466A1 (en) * | 2012-11-27 | 2014-05-29 | Hitachi, Ltd. | Time-series database setup automatic generation method, setup automatic generation system and monitoring server |
| US20150095381A1 (en) * | 2013-09-27 | 2015-04-02 | International Business Machines Corporation | Method and apparatus for managing time series database |
| CN103605805A (en) * | 2013-12-09 | 2014-02-26 | 冶金自动化研究设计院 | Storage method of massive time series data |
| US20170344614A1 (en) * | 2016-05-26 | 2017-11-30 | Salesforce.Com, Inc. | Caching time series data |
| CN106383864A (en) * | 2016-09-02 | 2017-02-08 | 北京百度网讯科技有限公司 | Query request processing method and apparatus for time series database |
| CN106815318A (en) * | 2016-12-24 | 2017-06-09 | 上海七牛信息技术有限公司 | A kind of clustering method and system of time series database |
| US20180293284A1 (en) * | 2017-04-10 | 2018-10-11 | Servicenow, Inc. | Systems and methods for querying time series data |
| CN108268217A (en) * | 2018-01-10 | 2018-07-10 | 北京航天云路有限公司 | A kind of bedding storage method based on the cold and hot classification of time series data |
| CN108446329A (en) * | 2018-02-13 | 2018-08-24 | 北京工业大数据创新中心有限公司 | Adaptive databases partition method and system towards industrial time series database |
| WO2019178979A1 (en) * | 2018-03-21 | 2019-09-26 | 平安科技（深圳）有限公司 | Method for querying report data, apparatus, storage medium and server |
| CN109766394A (en) * | 2018-12-19 | 2019-05-17 | 上海前隆信息科技有限公司 | Metrology platform data query method and device, readable storage medium storing program for executing and terminal |
| CN110209651A (en) * | 2019-05-16 | 2019-09-06 | 南京华盾电力信息安全测评有限公司 | Time series database system based on MongoDB |
| CN110445671A (en) * | 2019-06-27 | 2019-11-12 | 浪潮思科网络科技有限公司 | A kind of network flow monitoring method based on SDN |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111752245A (en) * | 2020-06-30 | 2020-10-09 | 北京和利时智能技术有限公司 | Data access and processing method, device, equipment and readable storage medium |
| CN111752245B (en) * | 2020-06-30 | 2021-06-25 | 北京和利时智能技术有限公司 | Data access and processing method, device, equipment and readable storage medium |
| CN112231531A (en) * | 2020-09-15 | 2021-01-15 | 山东浪潮通软信息科技有限公司 | Data display method, equipment and medium based on openstb |
| CN112487047A (en) * | 2020-11-02 | 2021-03-12 | 广州明珞装备股份有限公司 | Spare part consumption data query method, system and storage medium |
| WO2022088466A1 (en) * | 2020-11-02 | 2022-05-05 | 广州明珞装备股份有限公司 | Query method and system for spare parts consumption data, and storage medium |
| CN112487047B (en) * | 2020-11-02 | 2024-08-13 | 广州明珞装备股份有限公司 | Spare part consumption data query method, system and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111046081B (en) | 2023-09-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107784044B (en) | Table data query method and device | |
| CN111046081A (en) | Access method and system for industrial time sequence data | |
| CN110427368A (en) | Data processing method, device, electronic equipment and storage medium | |
| CN110245145A (en) | Structure synchronization method and apparatus of the relevant database to Hadoop database | |
| CN106407360B (en) | Data processing method and device | |
| CN112364033B (en) | Data retrieval system | |
| CN108415964A (en) | Tables of data querying method, device, terminal device and storage medium | |
| CN111859132A (en) | Data processing method and device, intelligent equipment and storage medium | |
| CN110647531A (en) | Data synchronization method, device, equipment and computer readable storage medium | |
| CN111382182A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN106547805A (en) | The method and apparatus of optimization database index | |
| CN102236707A (en) | Method, device and system for obtaining update data of database | |
| CN114661823A (en) | Data synchronization method and device, electronic equipment and readable storage medium | |
| CN114398520A (en) | Data retrieval method, system, device, electronic equipment and storage medium | |
| CN116595106A (en) | User grouping method, device and storage medium | |
| CN116450607A (en) | Data processing method, device and storage medium | |
| CN116303628A (en) | Alarm data query method, system and equipment based on elastic search | |
| Jiadi et al. | Research on Data Center Operation and Maintenance Management Based on Big Data | |
| CN112434063B (en) | Method for processing monitoring data based on time sequence database | |
| CN115114289A (en) | Data query method and device and electronic equipment | |
| CN110704397A (en) | Data query method and device based on elastic search | |
| CN112181994A (en) | Method, device and medium for refreshing distributed memory database of operation and maintenance big data | |
| CN114996306B (en) | Data management method and system based on multiple dimensions | |
| CN113127549B (en) | Incremental data synchronization method, device, computer equipment and storage medium | |
| CN115422183A (en) | Advertisement retrieval method, device and medium combining multi-index strategy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210401 Address after: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant after: NINGBO HELISHI INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant before: NINGBO HELISHI INTELLIGENT TECHNOLOGY Co.,Ltd. Applicant before: BEIJING HOLLYSYS AUTOMATION & DRIVE Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| CB02 | Change of applicant information |
Address after: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant after: Hollysys Kauber Technology Co.,Ltd. Address before: 5-7, No.38, building 7, East District, Ningbo new material innovation center, Ningbo hi tech Zone, 315000, Zhejiang Province Applicant before: NINGBO HELISHI INTELLIGENT TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |