CN105488148A - Big data storage and access system and method - Google Patents
Big data storage and access system and method Download PDFInfo
- Publication number
- CN105488148A CN105488148A CN201510843652.6A CN201510843652A CN105488148A CN 105488148 A CN105488148 A CN 105488148A CN 201510843652 A CN201510843652 A CN 201510843652A CN 105488148 A CN105488148 A CN 105488148A
- Authority
- CN
- China
- Prior art keywords
- data
- real
- service node
- time data
- data service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
- G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a big data storage and access system and method. The system comprises a real-time data service node, a historical data service node and a distributed coordination server; the real-time data service node is used for storing real-time data in a message queue which arrive in real time and transferring the real-time data to the historical data service node when receiving a notification from the distributed coordination server that the real-time data exceed a service period; the historical data service node comprises a distributed file system used for storing the real-time data exceeding the service period and from the real-time data service node as historical data; the distributed coordination server is used for managing the original data information in the real-time data service node and the historical data service node, monitoring the real-time data service node and the historical data service node and notifying the real-time data service node when the real-time data exceed the service period. The big data storage and access system and method in the scheme can improve the big data access instantaneity.
Description
Technical field
The present invention relates to large data technique field, particularly relate to a kind of memory access system and method for large data.
Background technology
Along with the development of Internet of Things, internet-of-things terminal equipment constantly increases, and the data volume thereupon produced is also in explosive increase.If traditional relevant database can not propose the function of horizontal extension, the demand that growing data store, retrieve, calculate can not be met.
Computational frame of the prior art can solve a lot of data pointedly and store and analysis demand, but, in the inquiry and analysis of existing Computational frame in mass data, there is the problem that real-time is lower.
Summary of the invention
The technical problem to be solved in the present invention is to provide a kind of large data memory access system and method, improves the real-time of large data access.
For solving the problems of the technologies described above, the embodiment of the present invention provides a kind of large data memory access system, and described system comprises:
Real-time Data Service node, historical data service node and distributed coordination server;
Described Real-time Data Service node, be suitable for the real time data arrived in real time in storing message queue, receive exceed the notice of viability from the described real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node;
Described historical data service node, comprises distributed file system, and described distributed file system is suitable for the described real time data exceeding the viability from described Real-time Data Service node as history data store;
Described distributed coordination server, be suitable for managing the former data message in described Real-time Data Service node and described historical data service node, monitor described Real-time Data Service node and described historical data service node, and exceed the viability in described real time data and notify described Real-time Data Service node.
Alternatively, described Real-time Data Service node, be suitable for reading the real time data arrived in real time in storing message queue, configuration information according to the tables of data from described distributed coordination server is resolved and index building described real time data, is organized into data block stores according to predefined time granularity.
Alternatively, described Real-time Data Service node is suitable for storing described data block in the internal memory of described Real-time Data Service node by write interface, when internal memory shared by tables of data reaches threshold value or data table entries exceedes setting number, by described real time data unloading in the disk of described Real-time Data Service node, and by distributed coordination server described in the state change notification of described data block.
Alternatively, described distributed coordination server is suitable for the service time of recording described real time data, when reaching the described viability, notify that described real time data writes in the distributed file system of described historical data service node by described Real-time Data Service node service time of described real time data.
Alternatively, described historical data service node also comprises buffer memory, is suitable for storing according to LRU the Query Result be retrieved in the nearest time period.
Alternatively, described distributed file system arranges corresponding storage medium according to storage medium configuration information in metadata table, and described storage medium comprises SSD, DISK and ARCHIVE.
Alternatively, the data in described Real-time Data Service node and described historical data service node are the data that column is expressed.
Alternatively, described large data memory access system, also comprises: data serving node:
Described data serving node, be suitable for the request of access receiving client, according to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data, the request of access of execution cost real time data is to described Real-time Data Service node calculate first intermediate result, the request of access of execution cost historical data is to described historical data service node calculate second intermediate result, merge described first intermediate result and described second intermediate result obtains net result, and return described net result to described client.
Alternatively, described Real-time Data Service node is also suitable for the request of access of receiving real-time data, and the request of access calculating described real time data obtains described first intermediate result, and returns described first intermediate result to described data serving node.
Alternatively, described historical data service node is also suitable for the request of access receiving described historical data, and the request of access calculating described historical data obtains described second intermediate result, and returns described second intermediate result to described data serving node.
The embodiment of the present invention also provides a kind of large data memory access method, and adopt above-mentioned large data memory access system, described method comprises:
By the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, and by the memory location of described distributed coordination server record real time data sheet;
Receive exceed the notice of viability from the described real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node.
Alternatively, described by the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, comprise: read the real time data arrived in real time in storing message queue, configuration information according to the tables of data from described distributed coordination server is resolved and index building described real time data, is organized into data block stores according to predefined time granularity.
Alternatively, described by the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, comprising:
Real-time Data Service node stores described data block in the internal memory of described Real-time Data Service node by write interface;
When internal memory shared by tables of data reaches threshold value or data table entries exceedes setting number, by described real time data unloading in the disk of described Real-time Data Service node, and by distributed coordination server described in the state change notification of described data block.
Alternatively, described large data memory access method also comprises:
Described data serving node receives the request of access of client;
According to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data;
The request of access of execution cost real time data is to described Real-time Data Service node calculate first intermediate result, request of access extremely described historical data service node calculate second intermediate result of execution cost historical data, merges described first intermediate result and described second intermediate result obtains net result;
Return described net result to described client.
Compared with prior art, the technical scheme of the embodiment of the present invention has following beneficial effect:
The technical scheme of the embodiment of the present invention arranges Real-time Data Service node, historical data service node and distributed coordination server, real time data is stored by Real-time Data Service node, and when the described distributed coordination server notice Real-time Data Service time limit arrives, real time data described in unloading is to described history service node, the real time data of viability is exceeded by the storage of described history service node, and there are real time service node described in described distributed coordination server admin and described history service node, when the viability of described real time data arrives, implementation data is write described history service node by described real time service node simultaneously, thus can when system acceptance is to the request of data access, real time data or historical data is provided by corresponding data serving node, thus improve the efficiency of system access real time data, thus improve the real-time of large data access.
Further, by setting data service node, by the request of access from client according to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data, the request of access of execution cost real time data and the request of access of described historical data to corresponding data serving node calculate intermediate result and return to described data serving node again, merge intermediate result by described data serving node and obtain net result to described client, thus can the efficiency of elevator system process request of access, the real-time of elevator system visit data.
Further, by arranging buffer memory in described historical data service node, at historical data service Node configuration local cache to store intermediate result, the efficiency of hotspot query can be accelerated by lru algorithm.
Further, the line data in described Real-time Data Service node and described historical data service node is converted to column and expresses, can by more effective compress technique reduction storage overhead, and provide column data is accessed faster.
Accompanying drawing explanation
Fig. 1 is the structural representation of a kind of large data memory access system in the embodiment of the present invention;
Fig. 2 is the structural representation of a kind of large data memory access system in the embodiment of the present invention;
Fig. 3 is the process flow diagram of the method for a kind of large data memory access in the embodiment of the present invention;
Fig. 4 is the process flow diagram of a kind of large data memory access method in the embodiment of the present invention.
Embodiment
As previously mentioned, Computational frame of the prior art can solve a lot of data pointedly and store and analysis demand, and the inquiry and analysis of existing Computational frame in mass data also exists the lower problem of real-time.
The technical scheme of the embodiment of the present invention arranges Real-time Data Service node, historical data service node and distributed coordination server, real time data is stored by Real-time Data Service node, and when the described distributed coordination server notice Real-time Data Service time limit arrives, real time data described in unloading is to described history service node, the real time data of viability is exceeded by the storage of described history service node, and there are real time service node described in described distributed coordination server admin and described history service node, when the viability of described real time data arrives, implementation data is write described history service node by described real time service node simultaneously, above-mentioned storage scheme can when system acceptance be to the request of data access, real time data or historical data is provided by corresponding data serving node, thus data can be retrieved in real time when can relate to real time data in request of access, just real time data can be retrieved compared to existing technology after medium calculated off-line and arrangement, improve the efficiency of system access to real time data, and then promote the real-time of large data memory access.
Further, when system acceptance is to the request of access of data, by data serving node, the generation time of described request of access according to data slice is split, and turn by corresponding data serving node process, thus can the efficiency of elevator system process request of access, the real-time of elevator system visit data.
For enabling above-mentioned purpose of the present invention, characteristic sum beneficial effect more becomes apparent, and is described in detail specific embodiments of the invention below in conjunction with accompanying drawing.
Fig. 1 is the structural representation of a kind of large data memory access system in the embodiment of the present invention.Fig. 1 is the structural representation of a kind of large data memory access system in the embodiment of the present invention.Described large data memory access system 10 can comprise: Real-time Data Service node 101, historical data service node 102 and distributed coordination server 103.
Described Real-time Data Service node 101, be suitable for the real time data arrived in real time in storing message queue, receive exceed the notice of viability from the described real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node.
Described historical data service node 102, comprises distributed file system, and described distributed file system is suitable for the described real time data exceeding the viability from described Real-time Data Service node as history data store.
Described distributed coordination server 103, be suitable for managing the former data message in described Real-time Data Service node and described historical data service node, monitor described Real-time Data Service node and described historical data service node, and exceed the viability in described real time data and notify described Real-time Data Service node.
Below in conjunction with specific embodiment, described large data memory access system is described.
Fig. 2 is the structural representation of a kind of large data memory access system in the embodiment of the present invention.Described large data memory access system 10 can comprise: Real-time Data Service node 101, historical data service node 102 and distributed coordination server 103.
Described Real-time Data Service node 101, be suitable for the real time data arrived in real time in storing message queue, receive exceed the notice of viability from the described real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node.
In concrete enforcement, node can be a server, or a service processes in logic.
In concrete enforcement, described Real-time Data Service node 101, be suitable for reading the real time data arrived in real time in storing message queue, configuration information according to the tables of data from described distributed coordination server 103 is resolved and index building described real time data, is organized into data block stores according to predefined time granularity.Such as, predefined time granularity can be 10 minutes, 1 hour or 1 day, is organized into one or more storage block according to predefined time granularity, and a storage block is 128MB or 256MB.
In concrete enforcement, described Real-time Data Service node 101 is suitable for storing described data block in the internal memory 1011 of described Real-time Data Service node 101 by write interface, when internal memory shared by tables of data 1011 reaches threshold value or data table entries exceedes setting number, by described real time data unloading in the disk 1012 of described Real-time Data Service node 101, and by distributed coordination server described in the state change notification of described data block.
In concrete enforcement, for real time data increases copy characteristic, multiple copy can be set and be distributed in different described Real-time Data Service nodes 101 respectively, in order to solve when cluster is inquired about and calculating pressure is larger, the problem of the service ability deficiency of single data.
Described historical data service node 102, comprises distributed file system 1021, and described distributed file system 1021 is suitable for the described real time data exceeding the viability from described Real-time Data Service node as history data store.
In concrete enforcement, described historical data service node 102 can also comprise buffer memory 1022, is suitable for storing according to LRU the Query Result be retrieved in the nearest time period, thus accelerates the efficiency of hotspot query.
Described distributed coordination server 103, be suitable for managing the former data message in described Real-time Data Service node and described historical data service node, monitor described Real-time Data Service node and described historical data service node, and exceed the viability in described real time data and notify described Real-time Data Service node.
In concrete enforcement, described distributed coordination server 103 is suitable for the service time of recording described real time data, when reaching the described viability, notify that described real time data writes in the distributed file system of described historical data service node by described Real-time Data Service node service time of described real time data.
In concrete enforcement, described distributed file system 1021 arranges corresponding storage medium according to the storage medium configuration information in metadata table, described storage medium comprises SSD (Solid-StateDriver, solid state hard disc), DISK (ordinary magnetic disc) and ARCHIVE (ArchivalStorage, the storage medium that storage density is higher).
In concrete enforcement, data in described Real-time Data Service node 101 and described historical data service node 102 are the data that column is expressed, line data is converted to column and can reduce storage overhead by more effective compress technique, and provide row are accessed faster.
The technical scheme of the embodiment of the present invention arranges Real-time Data Service node, historical data service node and distributed coordination server, real time data is stored by Real-time Data Service node, and when the described distributed coordination server notice Real-time Data Service time limit arrives, real time data described in unloading is to described history service node, the real time data of viability is exceeded by the storage of described history service node, and there are real time service node described in described distributed coordination server admin and described history service node, when the viability of described real time data arrives, implementation data is write described history service node by described real time service node simultaneously, thus can system when receiving the request of data access, real time data or historical data is provided by corresponding data serving node, need compared to existing technology to access the real time data upgraded after calculated off-line, the technical scheme of the embodiment of the present invention can have access to real time data in real time thus improve the efficiency of system access to real time data.
In concrete enforcement, described large data memory access system 10 can also comprise: data serving node 104.
Described data serving node 104, be suitable for the request of access receiving client, according to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data, request of access to the described Real-time Data Service node 101 of execution cost real time data calculates the first intermediate result, request of access to the described historical data service node 102 of execution cost historical data calculates the second intermediate result, merge described first intermediate result and described second intermediate result obtains net result, and return described net result to described client.
In concrete enforcement, a query analysis statement is through obtaining former data message (SchemaInformation) from described distributed coordination server 103, translate into logic plan (LogicalPlan), then some statistical informations are collected (as load by described distributed coordination server 103, the state of data), further optimization executive plan, generate physical logic plan (PhysicalPlan), and finally convert the combination performing operator (Executor) to, operator is submitted to inquire about to described historical data service node and described Real-time Data Service node, described first intermediate result collected and described second intermediate result are merged into a net result by described data serving node 104, and be back to client.
In concrete enforcement, when there being request of access, described Real-time Data Service node 101 is also suitable for the access receiving described real time data, and the access calculating described real time data obtains described first intermediate result, and returns described first intermediate result to described data serving node 104.
In concrete enforcement, when there being request of access, described historical data service node 102 is also suitable for the access receiving described historical data, and the access calculating described historical data obtains described second intermediate result, and returns described second intermediate result to described data serving node.
The technical scheme of the embodiment of the present invention is by arranging described data serving node, by the request of access from client according to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data, the request of access of execution cost real time data and the request of access of described historical data to corresponding data serving node calculate intermediate result and return to described data serving node again, merge intermediate result by described data serving node and obtain net result to described client, thus can the efficiency of elevator system process request of access, the real-time of elevator system visit data.
Fig. 3 is the process flow diagram of a kind of large data memory access method in the embodiment of the present invention, and described method have employed above-mentioned large data memory access system, can comprise the steps:
Step S301: by the real-time data memory that reaches in real time in message queue in Real-time Data Service node, and by the memory location of described distributed coordination server record real time data sheet;
Step S302: receive exceed the notice of viability from the real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node.
In concrete enforcement, described by the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, can comprise: read the real time data arrived in real time in storing message queue, configuration information according to the tables of data from described distributed coordination server is resolved and index building described real time data, is organized into data block stores according to predefined time granularity.
In concrete enforcement, described real time data sheet is the former data of temporally granularity division.
In concrete enforcement, described by the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, can comprise:
Real-time Data Service node stores described data block in the internal memory of described Real-time Data Service node by write interface;
When internal memory shared by tables of data reaches threshold value or data table entries exceedes setting number, by described real time data unloading in the disk of described Real-time Data Service node, and by distributed coordination server described in the state change notification of described data block.
Fig. 4 is the process flow diagram of a kind of large data memory access method in the embodiment of the present invention.Described method comprises:
Step S401: data serving node receives the request of access of client;
Step S402: according to the generation time relating to data slice in described request of access, splits into the request of access of real time data and the request of access of historical data by described request of access;
Step S403: the request of access of execution cost real time data is to described Real-time Data Service node calculate first intermediate result, request of access extremely described historical data service node calculate second intermediate result of execution cost historical data, merges described first intermediate result and described second intermediate result obtains net result;
Step S404: return described net result to described client.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is that the hardware that can carry out instruction relevant by program has come, this program can be stored in computer-readable recording medium, and storage medium can comprise: ROM, RAM, disk or CD etc.
Although the present invention discloses as above, the present invention is not defined in this.Any those skilled in the art, without departing from the spirit and scope of the present invention, all can make various changes or modifications, and therefore protection scope of the present invention should be as the criterion with claim limited range.
Claims (14)
1. a large data memory access system, is characterized in that, comprising: Real-time Data Service node, historical data service node and distributed coordination server;
Described Real-time Data Service node, be suitable for the real time data arrived in real time in storing message queue, receive exceed the notice of viability from the described real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node;
Described historical data service node, comprises distributed file system, and described distributed file system is suitable for the described real time data exceeding the viability from described Real-time Data Service node as history data store;
Described distributed coordination server, be suitable for managing the former data message in described Real-time Data Service node and described historical data service node, monitor described Real-time Data Service node and described historical data service node, and exceed the viability in described real time data and notify described Real-time Data Service node.
2. large data memory access system according to claim 1, it is characterized in that, described Real-time Data Service node, be suitable for reading the real time data arrived in real time in storing message queue, configuration information according to the tables of data from described distributed coordination server is resolved and index building described real time data, is organized into data block stores according to predefined time granularity.
3. large data memory access system according to claim 1, it is characterized in that, described Real-time Data Service node is suitable for storing described data block in the internal memory of described Real-time Data Service node by write interface, when internal memory shared by tables of data reaches threshold value or data table entries exceedes setting number, by described real time data unloading in the disk of described Real-time Data Service node, and by distributed coordination server described in the state change notification of described data block.
4. large data memory access system according to claim 1, it is characterized in that, described distributed coordination server is suitable for the service time of recording described real time data, when reaching the described viability, notify that described real time data writes in the distributed file system of described historical data service node by described Real-time Data Service node service time of described real time data.
5. large data memory access system according to claim 1, it is characterized in that, described historical data service node also comprises buffer memory, is suitable for storing according to LRU the Query Result be retrieved in the nearest time period.
6. large data memory access system according to claim 1, is characterized in that, described distributed file system arranges corresponding storage medium according to storage medium configuration information in metadata table, and described storage medium comprises SSD, DISK and ARCHIVE.
7. large data memory access system according to claim 1, is characterized in that, the data in described Real-time Data Service node and described historical data service node are the data that column is expressed.
8. large data memory access system according to claim 1, is characterized in that, also comprise: data serving node:
Described data serving node, be suitable for the request of access receiving client, according to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data, the request of access of execution cost real time data is to described Real-time Data Service node calculate first intermediate result, the request of access of execution cost historical data is to described historical data service node calculate second intermediate result, merge described first intermediate result and described second intermediate result obtains net result, and return described net result to described client.
9. large data memory access system according to claim 8, it is characterized in that, described Real-time Data Service node is also suitable for the request of access of receiving real-time data, the request of access calculating described real time data obtains described first intermediate result, and returns described first intermediate result to described data serving node.
10. large data memory access system according to claim 8, it is characterized in that, described historical data service node is also suitable for the request of access receiving described historical data, the request of access calculating described historical data obtains described second intermediate result, and returns described second intermediate result to described data serving node.
11. 1 kinds of large data memory access methods, is characterized in that, adopt the large data memory access system described in any one of claim 1-10, comprising:
By the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, and by the memory location of described distributed coordination server record real time data sheet;
Receive exceed the notice of viability from the described real time data of described distributed coordination server time, real time data described in unloading is to described historical data service node.
12. large data memory access methods according to claim 11, it is characterized in that, described by the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, comprise: read the real time data arrived in real time in storing message queue, configuration information according to the tables of data from described distributed coordination server is resolved and index building described real time data, is organized into data block stores according to predefined time granularity.
13. large data memory access methods according to claim 11, is characterized in that, described by the real-time data memory that reaches in real time in message queue in described Real-time Data Service node, comprising:
Real-time Data Service node stores described data block in the internal memory of described Real-time Data Service node by write interface;
When internal memory shared by tables of data reaches threshold value or data table entries exceedes setting number, by described real time data unloading in the disk of described Real-time Data Service node, and by distributed coordination server described in the state change notification of described data block.
14. large data memory access methods according to claim 11, is characterized in that, also comprise:
Described data serving node receives the request of access of client;
According to the generation time relating to data slice in described request of access, described request of access is split into the request of access of real time data and the request of access of historical data;
The request of access of execution cost real time data is to described Real-time Data Service node calculate first intermediate result, request of access extremely described historical data service node calculate second intermediate result of execution cost historical data, merges described first intermediate result and described second intermediate result obtains net result;
Return described net result to described client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510843652.6A CN105488148A (en) | 2015-11-26 | 2015-11-26 | Big data storage and access system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510843652.6A CN105488148A (en) | 2015-11-26 | 2015-11-26 | Big data storage and access system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105488148A true CN105488148A (en) | 2016-04-13 |
Family
ID=55675123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510843652.6A Pending CN105488148A (en) | 2015-11-26 | 2015-11-26 | Big data storage and access system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105488148A (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106383886A (en) * | 2016-09-21 | 2017-02-08 | 深圳市博瑞得科技有限公司 | Big data distribution type programming framework-based big data pre-statistics system and method |
| CN106528650A (en) * | 2016-10-14 | 2017-03-22 | 努比亚技术有限公司 | Resource query method and terminal |
| CN106776810A (en) * | 2016-11-24 | 2017-05-31 | 广东数果科技有限公司 | The data handling system and method for a kind of big data |
| CN107133128A (en) * | 2017-05-11 | 2017-09-05 | 上海新储集成电路有限公司 | A kind of safe data cloud storage system and method |
| CN109117428A (en) * | 2017-06-26 | 2019-01-01 | 北京嘀嘀无限科技发展有限公司 | Date storage method and its device, data query method and device thereof |
| CN109815026A (en) * | 2018-12-18 | 2019-05-28 | 国电南京自动化股份有限公司 | Electric power time series database based on distributed component |
| CN110309229A (en) * | 2019-05-09 | 2019-10-08 | 北京极数云舟科技有限公司 | The data processing method and distributed system of distributed system |
| CN110417838A (en) * | 2018-04-28 | 2019-11-05 | 华为技术有限公司 | A kind of method of data synchronization and synchronous service equipment |
| CN110502539A (en) * | 2019-07-17 | 2019-11-26 | 苏宁云计算有限公司 | A kind of OLAP method for dynamically caching and device |
| CN115455015A (en) * | 2022-08-08 | 2022-12-09 | 中亿(深圳)信息科技有限公司 | Mass data storage method and device and storage medium |
| WO2024051229A1 (en) * | 2022-09-05 | 2024-03-14 | 华为云计算技术有限公司 | Data storage method and apparatus, and related device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499070A (en) * | 2008-02-02 | 2009-08-05 | 北京城市学院 | History and real-time data access system and method based on open database interface |
| CN202058147U (en) * | 2011-05-23 | 2011-11-30 | 北京六所和瑞科技发展有限公司 | Distribution type real-time database management system |
| CN103605805A (en) * | 2013-12-09 | 2014-02-26 | 冶金自动化研究设计院 | Storage method of massive time series data |
| US20140289200A1 (en) * | 2013-03-21 | 2014-09-25 | Fujitsu Limited | Control method, and information processing system |
| CN104598540A (en) * | 2014-12-31 | 2015-05-06 | 国家电网公司 | Timing data migration device and using method thereof |
-
2015
- 2015-11-26 CN CN201510843652.6A patent/CN105488148A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101499070A (en) * | 2008-02-02 | 2009-08-05 | 北京城市学院 | History and real-time data access system and method based on open database interface |
| CN202058147U (en) * | 2011-05-23 | 2011-11-30 | 北京六所和瑞科技发展有限公司 | Distribution type real-time database management system |
| US20140289200A1 (en) * | 2013-03-21 | 2014-09-25 | Fujitsu Limited | Control method, and information processing system |
| CN103605805A (en) * | 2013-12-09 | 2014-02-26 | 冶金自动化研究设计院 | Storage method of massive time series data |
| CN104598540A (en) * | 2014-12-31 | 2015-05-06 | 国家电网公司 | Timing data migration device and using method thereof |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106383886A (en) * | 2016-09-21 | 2017-02-08 | 深圳市博瑞得科技有限公司 | Big data distribution type programming framework-based big data pre-statistics system and method |
| CN106528650B (en) * | 2016-10-14 | 2019-06-21 | 努比亚技术有限公司 | A kind of resource query method and terminal |
| CN106528650A (en) * | 2016-10-14 | 2017-03-22 | 努比亚技术有限公司 | Resource query method and terminal |
| CN106776810A (en) * | 2016-11-24 | 2017-05-31 | 广东数果科技有限公司 | The data handling system and method for a kind of big data |
| CN106776810B (en) * | 2016-11-24 | 2020-10-16 | 广东数果科技有限公司 | Big data processing system and method |
| CN107133128A (en) * | 2017-05-11 | 2017-09-05 | 上海新储集成电路有限公司 | A kind of safe data cloud storage system and method |
| CN109117428A (en) * | 2017-06-26 | 2019-01-01 | 北京嘀嘀无限科技发展有限公司 | Date storage method and its device, data query method and device thereof |
| CN110417838A (en) * | 2018-04-28 | 2019-11-05 | 华为技术有限公司 | A kind of method of data synchronization and synchronous service equipment |
| US11019145B2 (en) | 2018-04-28 | 2021-05-25 | Huawei Technologies Co., Ltd. | Data synchronization method and synchronization service device |
| CN109815026A (en) * | 2018-12-18 | 2019-05-28 | 国电南京自动化股份有限公司 | Electric power time series database based on distributed component |
| CN110309229A (en) * | 2019-05-09 | 2019-10-08 | 北京极数云舟科技有限公司 | The data processing method and distributed system of distributed system |
| CN110502539A (en) * | 2019-07-17 | 2019-11-26 | 苏宁云计算有限公司 | A kind of OLAP method for dynamically caching and device |
| CN115455015A (en) * | 2022-08-08 | 2022-12-09 | 中亿(深圳)信息科技有限公司 | Mass data storage method and device and storage medium |
| CN115455015B (en) * | 2022-08-08 | 2024-01-26 | 中亿(深圳)信息科技有限公司 | Mass data storage method and device and storage medium |
| WO2024051229A1 (en) * | 2022-09-05 | 2024-03-14 | 华为云计算技术有限公司 | Data storage method and apparatus, and related device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105488148A (en) | Big data storage and access system and method | |
| CN110032567B (en) | Report query method, device, server and storage medium | |
| KR102564170B1 (en) | Method and device for storing data object, and computer readable storage medium having a computer program using the same | |
| CN103353873B (en) | Optimization implementation method and system based on the service of time measure data real-time query | |
| WO2018187229A1 (en) | Database management system using hybrid indexing list and hierarchical query processing architecture | |
| CN106033324B (en) | Data storage method and device | |
| CN102467572B (en) | Data block inquiring method for supporting data de-duplication program | |
| US11372568B2 (en) | System and method for storing and accessing blockchain data | |
| CN103366016A (en) | Electronic file concentrated storing and optimizing method based on HDFS | |
| CN104978324B (en) | Data processing method and device | |
| US10678817B2 (en) | Systems and methods of scalable distributed databases | |
| US20140025899A1 (en) | Efficiently Updating and Deleting Data in a Data Storage System | |
| CN105159845A (en) | Memory reading method | |
| CN109299115A (en) | A kind of date storage method, device, server and storage medium | |
| CN109657009B (en) | Method, device, equipment and storage medium for creating data pre-partition storage periodic table | |
| CN102694828A (en) | Method and apparatus for data access in distributed caching system | |
| CN104182435A (en) | System and method for searching information based on data missing mark | |
| CN107368608A (en) | The HDFS small documents buffer memory management methods of algorithm are replaced based on ARC | |
| US20140258215A1 (en) | Management of updates in a database system | |
| CN112000703B (en) | Data warehousing processing method and device, computer equipment and storage medium | |
| CN110858210A (en) | Data query method and device | |
| CN109165207B (en) | Drinking water mass data storage management method and system based on Hadoop | |
| CN114048186A (en) | Data migration method and system based on mass data | |
| CN102724301B (en) | Cloud database system and method and equipment for reading and writing cloud data | |
| CN112559459B (en) | Cloud computing-based self-adaptive storage layering system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160413 |