CN112035563A - Real-time database system based on shared storage - Google Patents
Real-time database system based on shared storage Download PDFInfo
- Publication number
- CN112035563A CN112035563A CN202010885299.9A CN202010885299A CN112035563A CN 112035563 A CN112035563 A CN 112035563A CN 202010885299 A CN202010885299 A CN 202010885299A CN 112035563 A CN112035563 A CN 112035563A
- Authority
- CN
- China
- Prior art keywords
- real
- time
- data
- database
- measuring point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims description 23
- 238000004364 calculation method Methods 0.000 claims abstract description 40
- 238000013500 data storage Methods 0.000 claims abstract description 9
- 238000012544 monitoring process Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 4
- 230000001419 dependent effect Effects 0.000 claims description 4
- 238000007726 management method Methods 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 230000004048 modification Effects 0.000 claims description 2
- 238000012986 modification Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000000034 method Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 238000007405 data analysis Methods 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 210000001503 joint Anatomy 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 241000718543 Ormosia krugii Species 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005272 metallurgy Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 239000012466 permeate Substances 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a database system, comprising: the external unified interface module is used for providing an access interface of real-time data externally in the form of a Restful API (application program interface), and is also used for load balancing of interface access; the database main service module is used for forwarding the service request; the real-time data storage module is used for storing real-time and historical data of time sequence data in a distributed mode and providing a uniform Ceph Client access interface; the message middleware module is used for subscribing and publishing messages, and the messages are used for triggering memory loading, starting computing service and caching real-time data; the calculation service module is used for calculating measuring points of the time sequence database; and the unloading service module is used for unloading the real-time data of the real-time database to the historical database.
Description
Technical Field
The invention relates to the technical field of databases, in particular to a real-time database system based on shared storage.
Background
In the traditional application field, the traditional database has great success with the advantages of strict mathematical basis, simple and clear concept and easy operation. However, in the field of industrial production of processes such as electric power, chemical industry, energy and the like (the most important characteristic is that production is continuously carried out without interruption), a large amount of time sequence data can be generated, and with the digitalization and intellectualization of production equipment and the development of the technology of the internet of things, real-time data acquired in actual production shows a explosive growth trend. Conventional databases have difficulty meeting real-time constraints and high traffic demands for data. The problems that exist are mainly expressed as: traditional database systems aim at handling permanent, stable data, emphasizing maintaining data integrity, consistency, with the goal of high system throughput and low cost, regardless of timing constraints on the data and its processing. In contrast, the time series data management system aims at data with simple structural relationship and stable change amplitude, and can well meet the challenges of high real-time performance, maximum data capacity and large access quantity. The occurrence of the time sequence database realizes the high-efficiency compression storage, query retrieval and statistical analysis of mass data by using the super-large-scale data processing capacity and the high-proportion compression capacity of the time sequence database, provides a data base for an application system to carry out data mining and analysis calculation, and improves the informatization degree of enterprises in the aspects of real-time monitoring, data processing and storage, production information integration and sharing and the like.
At present, the time sequence data storage of the power enterprise is mainly based on a centralized deployment mode. The use and learning costs of the mainstream time sequence database in the market are too high, the maintenance is difficult, the system dependence control is too much, the expansibility is insufficient, the installation, deployment and uninstallation process is complex, and the like. Some time sequence databases have insufficient concurrency numbers of clients, the number of supported points is limited, and when the data capacity is increased, the reading and writing performance is remarkably reduced.
Disclosure of Invention
The invention aims to provide a real-time database system based on shared storage, which aims to solve the problems that the existing mainstream time sequence database has overhigh use and learning cost, difficult maintenance, excessive system dependence controls, insufficient expansibility, insufficient concurrency number and limited supported point quantity, and the reading and writing performance is obviously reduced when the data capacity is increased.
In order to achieve the purpose, the invention adopts the following technical scheme:
a database system, the system comprising:
the external unified interface module provides an access interface of real-time data externally in the form of a Restful API (application program interface); the external unified interface module is also used for load balancing of interface access;
the database main service module is used for forwarding the service request;
the real-time data storage module is used for storing real-time and historical data of time sequence data in a distributed mode and provides a uniform Ceph Client access interface;
the message middleware module is used for subscribing and publishing messages, and the messages are used for triggering memory loading, starting computing service and caching real-time data;
the calculation service module is used for calculating measuring points of a time sequence database;
and the unloading service module is used for unloading the real-time data of the real-time database to the historical database.
Preferably, the load balancing of the interface access is implemented based on Nginx and keepalive.
Preferably, the real-time data is stored in a memory database Redis, the historical data is stored in a document database MongoDB, and the memory database Redis and the document database MongoDB are deployed on the Ceph cluster.
Preferably, the message middleware module is based on kafka message middleware.
Preferably, the database system further comprises,
and the measuring point management module is used for recording basic information of the common real-time measuring point and recording the basic information of the calculating measuring point.
Specifically, the basic information of the common real-time measuring point comprises a measuring point name, a measuring point number, measuring point description, a dead zone and history storage;
the basic information of the measuring points comprises measuring point names, measuring point numbers, measuring point description, common measuring points on which the measuring points depend and a calculation formula.
Specifically, basic information of common real-time measuring points and calculating measuring points is stored in a MongoDB database deployed in a Ceph cluster, adding, modifying and deleting operations of the basic information are sent to a data main service interface through an external unified interface module, modified measuring point information on different devices is synchronized through a message middleware module, a Ceph Client modifies a measuring point information table stored in the MongoDB database by monitoring corresponding measuring point information themes, and the measuring point information table is loaded into a message middleware module cache queue when a time sequence storage system service is started.
Preferably, the measuring point calculating mode comprises a polling mode and a triggering mode.
Specifically, the polling mode includes that periodic calculation is carried out according to a calculation period of a calculation point, a calculation result of the measurement point is calculated according to a preset calculation model by reading real-time data of a real-time measurement point value, and the calculation result of the measurement point is written back to a corresponding calculation measurement point value through a unified access interface.
Specifically, the triggering mode comprises the steps that the calculation measuring point triggers calculation service according to the change situation of the real-time value of the dependent measuring point, the real-time value of the relevant calculation measuring point is calculated, the triggering is based on the real-time measuring point information which is subscribed by the message middleware module and depended by the data measuring point, and the calculation result is written back to the corresponding calculation measuring point value through the unified access interface.
Compared with the prior art, the database system provided by the invention combines a load balancing, a distributed file system and a high-throughput message middleware to establish a real-time data storage architecture, the database has high concurrent access amount, good expandability, high reliability, low learning, using and maintaining cost, rich and flexible data access interfaces, supports almost all languages to access the database, supports butt joint with a big data ecosphere, can effectively solve the problem that the reading and writing efficiency of the traditional real-time database is slow in mass data, supports second-level response of historical data, and can meet the application requirements of real-time and historical data under the condition of large data volume.
Drawings
Fig. 1 is a block diagram of a database system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of data transmission of a database system according to an embodiment of the present invention.
Detailed Description
The invention is further described with reference to specific examples. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
Generally, the time-series database is mainly applied to the fields with high requirements on both database capacity and real-time processing technology, such as industrial control systems and information management systems in power, petrochemical industry, metallurgy and the like. At present, many mature time sequence databases exist at home and abroad. According to the difference of the used kernels, the real-time database capable of processing the time sequence data can be divided into a database adopting a special kernel, a real-time database adopting a relational database kernel and a real-time database realized based on a big data platform open source component. Typical representatives of the special kernel are InfoPlus21 of AspentTech and PI of OSI, and the real-time library adopting the special kernel has the advantages of high response speed, high reliability and large capacity, but has the problems of high purchase cost, insufficient cross-platform aspect (for example, PI is mainly operated on a Windows platform), poor access to a large data platform and the like. A typical representation of the use of a relational database kernel is fabtorysait 2000 from Wonderware. The real-time database realized by the large data platform open source component represents openTSDB, HBase cluster storage is adopted at the bottom layer of the openTSDB, data are compressed according to the characteristics of time sequence, storage space is saved, meanwhile, the operations of inquiry, data aggregation, filtering and the like commonly used for the time sequence data are supported for packaging, and the expandability is good. But the problems of more dependent components, complex installation and high deployment and operation and maintenance costs exist.
Moreover, as the internet of things technology gradually permeates the industry, the ever-increasing sensors, the soaring data volume and the higher large data analysis demand provide further challenges for the original traditional time series database technology, and the traditional time series database has the following technical problems: 1) scalability encounters a bottleneck. Although the traditional technical architecture can ensure that a single machine has extremely high performance, the performance can be linearly expanded by adding machines, dynamic and flexible capacity expansion and capacity reduction can not be realized like a distributed system, and planning needs to be performed in advance. When service upgrading requires system capacity expansion, the expansibility of the old framework is difficult to meet the requirements. 2) And ecological connection with big data cannot be realized. A mature solution for storage and analysis of mass data in the existing big data industry exists, and the existing real-time database has to be upgraded and modified to realize the butt joint with a big data platform. 3) The real-time database has the problem of high price, and the real-time database products on the market have high cost and superior performance.
To this end, the present invention provides a database system, and fig. 1 is a structural diagram of a database system according to an embodiment of the present invention, as shown in fig. 1, the system 100 at least includes:
the external unified interface module 11 is configured to provide an access interface for real-time data externally in the form of a Restful API interface, and the external unified interface module 11 is further configured to balance load of interface access.
The external unified interface of the database adopts an REST software architecture style, is provided externally through a Restful API interface form, and fully exerts the advantages of a stateless protocol HTTP of Restful, strong expansion capability, JSON message serialization, light weight, simplicity, human-computer friendliness, language independence and the like.
Specifically, a unified access interface is provided for the outside through a Restful API interface form, so that a database system is separated from the outside, the upgrading and modification of the database cannot generate the influence of a code level on a user, the Restful language independence and JSON data format are utilized to enable the Restful data interface to cope with most development environments and development languages, and the Restful language independence and JSON data format have inherent advantages in expansibility and applicable environments, and can also be butted with a big data ecosphere Hadoop and spark to provide support for data analysis work.
According to one embodiment, in order to improve the security of the database, the security of the database access can be ensured by using an HTTP (HTTPS) stateless connection state attribute and adopting a mode of combining user login authentication, JWT encryption authentication and a security authentication mechanism of the Ceph in data access and database component use.
In one embodiment, load balancing of interface accesses is based on Nginx and keepalive implementations. The Nginx and keepalive have high stability, rich function sets, low system resource consumption and strong concurrency capability, and can be used for realizing load balance of interface access and improving concurrency and reliability of the interface access.
And the database main service module 12 is used for forwarding the service request.
And the real-time data storage module 13 is used for storing real-time and historical data of the time sequence data in a distributed mode and providing a uniform Ceph Client access interface.
In one embodiment, the real-time data is stored in the memory database Redis, the historical data is stored in the document database MongoDB, and the memory database Redis and the document database MongoDB are deployed on the Ceph cluster.
According to a specific implementation mode, real-time data storage is installed on a Ceph cluster by using memory database Redis and document database MongoDB deployment, and quick access to real-time data is realized by utilizing the memory access advantage of Redis, a key-value storage format and a batch reading interface; the MongoDB database has flexible nosql storage characteristics, a loose data structure, and abundant, powerful and rapid query languages, and meets the requirements of a time sequence database for large-scale, complex and rapid requests of historical data; the distributed storage of real-time data and historical data is realized by utilizing the high performance, high availability and high expansion characteristics of Ceph.
The message middleware module 14 is configured to subscribe and publish a message, where the message is used to trigger memory loading, start computing services, and cache real-time data.
In one embodiment, the message middleware module is based on kafka message middleware.
The kafka message bus mainly utilizes the data subscribing/publishing function of kafka, and aims at the characteristics of large quantity of sensors, high acquisition frequency and high data writing concurrency in the power industry, the kafka message bus plays a role in eliminating peak buffer data when writing real-time data and historical data into a real-time library, plays a role in message notification in calculating point triggering calculation service, and is used for synchronizing the point cache information on different devices when the information of calculating points and real-time points is changed, so that the consistency of the point information is ensured.
And the calculation service module 15 is used for calculating measuring points of the time sequence database.
The essential of the calculation measuring point is a special real-time measuring point, the value of the real-time measuring point is obtained by a series of logic operations of the real-time values of one or more real-time measuring points, namely the calculation measuring point depends on one or more real-time measuring points, the calculation measuring point service is mainly designed aiming at reading and writing of some specific calculation statistical indexes of the power plant, and a client can be configured with a calculation measuring point calculation mode and a reading and writing calculation measuring point value.
In one embodiment, the manner in which the stations are calculated includes a polling manner and a triggering manner.
According to a specific implementation manner, the polling manner may include performing periodic calculation according to a calculation cycle of the calculation station, calculating a calculation result of the measurement station according to a predetermined calculation model by reading real-time data of the real-time measurement point value, and writing the calculation result of the measurement station back to the corresponding calculation measurement point value through the unified access interface.
According to another specific implementation manner, the triggering manner may include that the computing measurement point triggers the computing service according to a change situation of a real-time value of a dependent measurement point, and a real-time value of a relevant computing measurement point is computed, the triggering is based on subscribing real-time measurement point information of a data measurement point dependence through the message middleware module, and a computing result is written back to a corresponding computing measurement point value through the unified access interface.
And the unloading service module 16 is used for unloading the real-time data of the real-time database to the historical database.
In one embodiment, the data system further comprises,
and the measuring point management module 17 is used for recording basic information of common real-time measuring points and recording basic information of calculating measuring points.
According to a specific implementation mode, basic information of a common real-time measuring point comprises a measuring point name, a measuring point number, measuring point description, a dead zone and history of storage or not; the basic information of the measuring points comprises measuring point names, measuring point numbers, measuring point description, common measuring points on which the measuring points depend and a calculation formula.
According to another specific implementation mode, basic information storage of common real-time measuring points and calculating measuring points is deployed in a MongoDB database of a Ceph cluster, adding, modifying and deleting operations of the basic information are sent to a data main service interface through an external unified interface module, modified measuring point information on different devices is synchronized through a message middleware module, a Ceph Client modifies a measuring point information table stored in the MongoDB database by monitoring corresponding measuring point information themes, and the measuring point information table is loaded into a message middleware module cache queue when a time sequence storage system service is started, so that data reading and writing efficiency is improved.
The operation process of the database system provided by the invention is further explained by an embodiment, in the embodiment, the database system is used for storing the real-time measuring point data of the power plant and storing and calculating the measuring point data, fig. 2 shows a data transmission schematic diagram of the database system in the embodiment, and the operation process is as follows:
real-time data writing: the Client sends a write request to a Restful API unified data interface, a load balancing module realized by Nginx and keepalive distributes the request to data main service interfaces deployed on a plurality of devices, a service interface module writes data into a real-time data theme corresponding to kafka according to request content, a Ceph Client writes the data into a redis real-time library by subscribing the theme so as to avoid access pressure on the real-time library caused by overlarge transient data amount, the real-time data can also be directly written into the redis real-time library through the Ceph Client interface, and finally, a real-time data write operation result is fed back to the Client through the Restful API interface.
Real-time data reading: the Client sends a read request to the Restful API unified data interface, a load balancing module realized by Nginx and keepalive distributes the request to data main service interfaces deployed on a plurality of devices, the service interface module reads data through a Ceph Client according to the request content, and the data operation result during real-time writing is fed back to the Client through the Restful API interface.
Writing historical data: the Client sends a writing request to a Restful API unified data interface, a load balancing module realized by Nginx and keepalive distributes the request to a data main service interface deployed on a plurality of devices, the service interface module writes data into a history data theme corresponding to Kafka according to the request content, the Ceph Client writes the data into a MongoDB history library by subscribing the theme, or directly writes the real-time history data into a MongoDB history library through a Ceph Client interface, feeds back a writing data operation result to the Client through a Restful API interface, and simultaneously supports the modes of transferring the real-time data from the real-time library to the history library through a transfer service, transferring the data to the history library supports full-volume data transfer, transferring the sparse data at a specified time interval and the like.
Reading historical data: the Client sends a reading request to a Restful API unified data interface, a load balancing module realized by Nginx and keepalive distributes the request to data main service interfaces deployed on a plurality of devices, a service interface module reads real-time historical data from a MongoDB historical database through a Ceph Client interface according to the request content, and the data operation result is fed back to the Client through the Restful API interface, wherein the historical data reading supports interpolation, section value and storage value types.
In different embodiments, dynamic and flexible capacity expansion and capacity reduction are facilitated based on open-source distributed components Nginx, Kafka and Ceph, development can be performed based on a java environment, and Linux versions and domestic systems such as kylin and jequirity are supported.
In one embodiment, each component of the system can be installed and deployed in a one-key mode, and real-time operation monitoring is carried out in a B/S mode, so that the difficulty of installation, maintenance and learning of a user is reduced.
The database system provided by the invention combines load balancing, a distributed file system and high-throughput message middleware to establish a real-time data storage architecture, so that the timeliness of time sequence data processing is ensured, the interface access bottleneck under the condition of high concurrency is broken through, the problem that the real-time database cannot be accessed due to single-point faults is avoided, meanwhile, the expandability of the real-time database is ensured, the problem of data integrity due to single-point faults is avoided, the access pressure of the real-time database is relieved, the trigger function of a calculation measuring point is provided, and the second-level time sequence data storage and retrieval capability is possessed.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The present invention has been disclosed in terms of the preferred embodiment, but is not intended to be limited to the embodiment, and all technical solutions obtained by substituting or converting equivalents thereof fall within the scope of the present invention.
Claims (10)
1. A database system, the system comprising:
the external unified interface module provides an access interface of real-time data externally in the form of a Restful API (application program interface); the external unified interface module is also used for load balancing of interface access;
the database main service module is used for forwarding the service request;
the real-time data storage module is used for storing real-time and historical data of time sequence data in a distributed mode and provides a uniform Ceph Client access interface;
the message middleware module is used for subscribing and publishing messages, and the messages are used for triggering memory loading, starting computing service and caching real-time data;
the calculation service module is used for calculating measuring points of a time sequence database;
and the unloading service module is used for unloading the real-time data of the real-time database to the historical database.
2. The system of claim 1, wherein load balancing of the interface accesses is based on Nginx and keepalive implementations.
3. The system of claim 1, wherein the real-time data is stored in a memory database Redis and the historical data is stored in a document database MongoDB, the memory database Redis and the document database MongoDB being deployed on a Ceph cluster.
4. The system of claim 1, wherein the message middleware module is based on kafka message middleware.
5. The system of claim 1, further comprising,
and the measuring point management module is used for recording basic information of the common real-time measuring point and recording the basic information of the calculating measuring point.
6. The system of claim 5, wherein the basic information of the common real-time measuring points comprises the name of the measuring point, the number of the measuring point, the description of the measuring point, the dead zone and whether the history is stored;
the basic information of the measuring points comprises measuring point names, measuring point numbers, measuring point description, common measuring points on which the measuring points depend and a calculation formula.
7. The system of claim 5, wherein basic information storage of common real-time measuring points and calculating measuring points is deployed in a MongoDB database of a Ceph cluster, adding, modifying and deleting operations of the basic information are sent to a data main service interface through an external unified interface module, measuring point information after modification on different devices is synchronized through a message middleware module, a Ceph Client modifies a measuring point information table stored in the MongoDB database by monitoring corresponding measuring point information subjects, and the measuring point information table is loaded into a message middleware module cache queue when a time sequence storage system service is started.
8. The system of claim 1, wherein the means for calculating stations comprises polling means and triggering means.
9. The system of claim 8, wherein the polling comprises periodically calculating according to a calculation cycle of the calculated stations, calculating the station calculation results according to a predetermined calculation model by reading real-time data of the real-time station values, and writing the station calculation results back to the corresponding calculated station values through the unified access interface.
10. The system of claim 8, wherein the triggering mode includes that the computing measuring point triggers the computing service according to the change situation of the real-time value of the dependent measuring point, and the real-time value of the relevant computing measuring point is computed, the triggering is based on the real-time measuring point information which is subscribed by the message middleware module and depended by the data measuring point, and the computing result is written back to the corresponding computing measuring point value through the unified access interface.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010885299.9A CN112035563A (en) | 2020-08-28 | 2020-08-28 | Real-time database system based on shared storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010885299.9A CN112035563A (en) | 2020-08-28 | 2020-08-28 | Real-time database system based on shared storage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112035563A true CN112035563A (en) | 2020-12-04 |
Family
ID=73586964
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010885299.9A Pending CN112035563A (en) | 2020-08-28 | 2020-08-28 | Real-time database system based on shared storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112035563A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395116A (en) * | 2021-01-20 | 2021-02-23 | 北京东方通软件有限公司 | Adjusting and optimizing method and system for message middleware |
| CN112559913A (en) * | 2020-12-11 | 2021-03-26 | 车智互联（北京）科技有限公司 | Data processing method and device, computing equipment and readable storage medium |
| CN113177088A (en) * | 2021-04-02 | 2021-07-27 | 北京科技大学 | Multi-scale simulation big data management system for material irradiation damage |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618484A (en) * | 2015-02-05 | 2015-05-13 | 浪潮电子信息产业股份有限公司 | Key value pair based transverse expansion high-performance storing method |
| CN109815026A (en) * | 2018-12-18 | 2019-05-28 | 国电南京自动化股份有限公司 | Electric power time series database based on distributed component |
| CN110209651A (en) * | 2019-05-16 | 2019-09-06 | 南京华盾电力信息安全测评有限公司 | Time series database system based on MongoDB |
-
2020
- 2020-08-28 CN CN202010885299.9A patent/CN112035563A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104618484A (en) * | 2015-02-05 | 2015-05-13 | 浪潮电子信息产业股份有限公司 | Key value pair based transverse expansion high-performance storing method |
| CN109815026A (en) * | 2018-12-18 | 2019-05-28 | 国电南京自动化股份有限公司 | Electric power time series database based on distributed component |
| CN110209651A (en) * | 2019-05-16 | 2019-09-06 | 南京华盾电力信息安全测评有限公司 | Time series database system based on MongoDB |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559913A (en) * | 2020-12-11 | 2021-03-26 | 车智互联（北京）科技有限公司 | Data processing method and device, computing equipment and readable storage medium |
| CN112559913B (en) * | 2020-12-11 | 2023-10-20 | 车智互联（北京）科技有限公司 | Data processing method, device, computing equipment and readable storage medium |
| CN112395116A (en) * | 2021-01-20 | 2021-02-23 | 北京东方通软件有限公司 | Adjusting and optimizing method and system for message middleware |
| CN113177088A (en) * | 2021-04-02 | 2021-07-27 | 北京科技大学 | Multi-scale simulation big data management system for material irradiation damage |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112035563A (en) | Real-time database system based on shared storage | |
| TWI549080B (en) | The method, system and device for sending information of category information | |
| US11960505B2 (en) | Export data from tables into partitioned folders on an external data lake | |
| CN105205154B (en) | Data migration method and device | |
| CN103095819A (en) | Data information pushing method and data information pushing system | |
| CN113508373B (en) | Distributed metadata-based cluster computation | |
| CN110609813B (en) | Data storage system and method | |
| WO2018035799A1 (en) | Data query method, application and database servers, middleware, and system | |
| CN110837423A (en) | Method and device for automatically acquiring data of guided transport vehicle | |
| US11507277B2 (en) | Key value store using progress verification | |
| US20190228009A1 (en) | Information processing system and information processing method | |
| CN110618860A (en) | Spark-based Kafka consumption concurrent processing method and device | |
| CN113282611A (en) | Method and device for synchronizing stream data, computer equipment and storage medium | |
| CN105550238A (en) | Architecture system of database appliance | |
| CN115292414A (en) | Method for synchronizing service data to data bins | |
| CN112000703B (en) | Data warehousing processing method and device, computer equipment and storage medium | |
| CN113632073B (en) | Scalable streaming over data sources | |
| CN114416868A (en) | Data synchronization method, device, equipment and storage medium | |
| US20190057028A1 (en) | Conflict Resolution and Garbage Collection in Distributed Databases | |
| CN109684611A (en) | A kind of dictionary code-transferring method, device, storage medium and terminal | |
| CN106934044B (en) | Data processing method and device | |
| EL-SAYED et al. | Impact of small files on hadoop performance: literature survey and open points | |
| CN115221116A (en) | Data writing method, device and equipment and readable storage medium | |
| CN114547199A (en) | Database increment synchronous response method and device and computer readable storage medium | |
| CN113360576A (en) | Power grid mass data real-time processing method and device based on Flink Streaming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |