CN113626464A - Query support method and system based on stored data in ClickHouse database - Google Patents
Query support method and system based on stored data in ClickHouse database Download PDFInfo
- Publication number
- CN113626464A CN113626464A CN202110879192.8A CN202110879192A CN113626464A CN 113626464 A CN113626464 A CN 113626464A CN 202110879192 A CN202110879192 A CN 202110879192A CN 113626464 A CN113626464 A CN 113626464A
- Authority
- CN
- China
- Prior art keywords
- memory data
- clickhouse
- metadata information
- disk
- memory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000008569 process Effects 0.000 claims abstract description 17
- 230000008520 organization Effects 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000005206 flow analysis Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2425—Iterative querying; Query formulation based on the results of a preceding query
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a query support method and a query support system based on ClickHouse database memory data, which relate to the field of database memory data processing; the method comprises the steps of constructing a block data object, temporarily storing memory data, circularly searching the needed memory data by using an iterator, storing the memory data meeting the conditions into the block data object, carrying out sorting processing on the memory data in the block data object according to a ClickHouse database table building sorting mode, calling a memory data writing interface of a ClickHouse folding tree engine, writing the memory data in the block data object into the memory data object of the ClickHouse, inserting the memory data object information into a list for storing disk metadata information as metadata information through a SQL execution process of the ClickHouse, acquiring the memory data of a disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
Description
Technical Field
The invention discloses a method and a system, which relate to the field of database storage data processing, in particular to a method and a system for query support based on ClickHouse database storage data.
Background
The ClickHouse is a column type storage database with an MPP framework, is mainly used for on-line analytical processing query (OLAP), can use SQL query and generate an analysis data report in real time, and has analysis performance exceeding that of most column type storage databases in the market.
However, when the clickwouse is used for querying the memory data, the data needs to be processed through a series of execution operators, the query process is complicated, the query time is long, and the query result can be finally obtained, otherwise, the data in the memory table cannot be sensed and the data in the memory table cannot be processed.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a query support method and a query support system based on the stored data in the ClickHouse database, which support the writing and reading of the data in the memory, reduce unnecessary data falling in the query process and further improve the query performance.
The specific scheme provided by the invention is as follows:
based on the query support method of the memory data in the ClickHouse database, the data object of the building block is constructed, the memory data is temporarily stored,
using iterator to search needed memory data circularly, storing memory data meeting condition into block data object,
sorting the memory data in the block data object according to a ClickHouse database table-building sorting mode,
calling a memory data writing interface of a ClickHouse folding tree engine, writing the memory data in the block data object into the memory data object of the ClickHouse,
and inserting the memory data object information serving as metadata information into a list for storing the metadata information of the disk through an SQL execution flow of ClickHouse, acquiring the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
Further, the query support method based on the memory data in the ClickHouse database circularly searches the required memory data:
constructing an initial value required by the iterator according to initial information of the memory data required to be inquired by the iterator, searching an initial position of the required memory data according to the initial value,
and circularly searching the next memory data by using the iterator, filtering the memory data which does not accord with the condition, storing the memory data which accords with the condition into the block data object until the memory data which does not accord with the condition is not obtained, and terminating the search.
Further, in the ClickHouse database memory data-based query support method, the ClickHouse SQL execution flow is as follows:
inserting the memory data object information as metadata information into a list for storing the metadata information of the disk,
parsing sql to generate a corresponding abstract syntax tree,
an interpreter for constructing the abstract syntax tree, and acquiring the metadata information of the storage disk through the interpreter,
and calling a reading interface of the memory data object to acquire the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
Further, in the query support method based on the memory data in the ClickHouse database, the interpreter acquires the metadata information of the storage disk:
generating a logical plan according to the interpreter, storing a list storing disk metadata information in the logical plan,
and generating a physical plan according to the logic plan,
and executing the physical plan, and calling a reading interface of the memory data object to acquire the metadata information in the list of the storage disk metadata information.
A query support system based on the memory data in the ClickHouse database, a memory data query organization module and a memory data query processing module,
the memory data query organization module constructs a block data object, temporarily stores memory data, circularly searches the needed memory data by using an iterator, stores the memory data meeting the conditions into the block data object, carries out the sorting processing of the memory data in the block data object according to the table-building sorting mode of a ClickHouse database, calls a memory data writing interface of a ClickHouse folding tree engine, writes the memory data in the block data object into the memory data object of the ClickHouse,
the memory data query processing module inserts the memory data object information as metadata information into a list for storing the metadata information of the disk through an SQL execution flow of the ClickHouse, acquires the memory data of the disk in the list according to the metadata information, and queries and processes the memory data in the disk.
Further, the memory data query organization module in the query support system based on the memory data in the ClickHouse database circularly searches the required memory data:
constructing an initial value required by the iterator according to initial information of the memory data required to be inquired by the iterator, searching an initial position of the required memory data according to the initial value,
and circularly searching the next memory data by using the iterator, filtering the memory data which does not accord with the condition, storing the memory data which accords with the condition into the block data object until the memory data which does not accord with the condition is not obtained, and terminating the search.
Further, the memory data query processing module in the query support system based on the memory data in the clickwouse database executes the SQL procedure of clickwouse:
inserting the memory data object information as metadata information into a list for storing the metadata information of the disk,
parsing sql to generate a corresponding abstract syntax tree,
an interpreter for constructing the abstract syntax tree, and acquiring the metadata information of the storage disk through the interpreter,
and calling a reading interface of the memory data object to acquire the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
Further, the memory data query processing module in the query support system based on the memory data in the ClickHouse database executes the process of acquiring the metadata information of the storage disk by the interpreter in the ClickHouse SQL procedure:
generating a logical plan according to the interpreter, storing a list storing disk metadata information in the logical plan,
and generating a physical plan according to the logic plan,
and executing the physical plan, and calling a reading interface of the memory data object to acquire the metadata information in the list of the storage disk metadata information.
The query supporting device based on the memory data in the ClickHouse database comprises: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine-readable program to execute the query support method based on the data stored in the clickwouse database.
The invention has the advantages that:
the invention provides a query support method based on the memory data in a ClickHouse database, which directly organizes the memory data, converts the memory data into a memory data object which can be processed by the ClickHouse, does not need to fall the memory data, reduces the time consumption caused by falling the memory data during query, reduces the time consumption on IO (input/output) because the memory data does not need to be read in a disk during query, and improves the data reading and querying performance.
Drawings
FIG. 1 is a schematic diagram of a process of performing memory storage by a ClickHouse.
FIG. 2 is a schematic diagram of a memory data reading process of ClickHouse.
FIG. 3 is a schematic diagram of SQL execution flow of the ClickHouse according to the method of the present invention.
FIG. 4 is a schematic diagram of the module composition of the system of the present invention.
Detailed Description
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The invention provides a query support method based on the memory data in a ClickHouse database, which constructs a block data object, temporarily stores the memory data,
using iterator to search needed memory data circularly, storing memory data meeting condition into block data object,
sorting the memory data in the block data object according to a ClickHouse database table-building sorting mode,
calling a memory data writing interface of a ClickHouse folding tree engine, writing the memory data in the block data object into the memory data object of the ClickHouse,
and inserting the memory data object information serving as metadata information into a list for storing the metadata information of the disk through an SQL execution flow of ClickHouse, acquiring the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
The method directly organizes the memory data, converts the memory data into the memory data object which can be processed by the clickwouse, does not need to perform memory data disk dropping, reduces the time consumption caused by the data disk dropping during query, reduces the time consumption on IO (input/output) because the memory data does not need to be read in a disk during query, and improves the data reading and querying performance.
In specific application, in some embodiments of the present invention, unnecessary data dropping in the query process is reduced by supporting the writing and reading of memory data, thereby improving the query performance.
Before the memory data is inquired, the ClickHouse performs the process of writing the memory data.
When memory data is written, the data is written into memTable firstly for memory data storage, and the data format adopts a key-value format.
Log writing is performed before key-value data writing, the atomic consistency of data is guaranteed, and exception recovery can be performed on some data.
And when the data volume reaches a certain threshold value, converting the KV data into a column data format according to the table building information of the ClickHouse, and finally writing the batch of data into a memory of the ClickHouse.
The method of the invention is used for the memory data query preparation process:
s11 constructing a block data object for temporarily storing memory data,
s12, constructing the read iterator of the memory data, constructing the initial value needed by the iterator according to the transmitted initial information of the memory data, finding the initial position of the needed data by the iterator according to the initial value,
s13, circularly searching the next memory data through the iterator, filtering out the memory data which do not accord with the condition in advance according to the transmitted filtering information, storing the data which accord with the condition into the object constructed in the step S11 until the iterator can not obtain the data which accord with the condition, terminating the search,
s14, sorting the objects of the temporary stored data in the step S11, wherein the sorting rule is consistent with the sorting key sorting when the ClickHouse is built, so that the data processed by the ClickHouse is sorted, the data in the memory is also sorted,
s15: and writing the sequenced data into a memory data object supported by the ClickHouse query by calling a memory data writing interface of the ClickHouse folding tree engine, wherein the object is stored in a memory.
Performing memory data query through a ClickHouse SQL execution flow:
s21: inserting the memory data object information as metadata information into a list for storing the metadata information of the disk,
s22: parsing sql to generate a corresponding abstract syntax tree,
s23: constructing an interpreter of the abstract syntax tree, acquiring metadata information of a storage disk through the interpreter, wherein a logic plan is generated according to the interpreter, a list of the metadata information of the storage disk is stored in the logic plan, a physical plan is generated according to the logic plan, the physical plan is executed, a reading interface of a memory data object is called to acquire the metadata information in the list of the metadata information of the storage disk,
s24: and inquiring the memory data in the processing disk according to the metadata information in the list.
According to the process, the ClickHouse execution flow analysis is carried out, the memory data object information is inserted into a list for storing the disk metadata information, the information is transmitted to the step S23 from the beginning of the ClickHouse execution flow, the metadata information list is stored in a logic plan through the processing of a ClickHouse query executor, then, when a physical plan is executed, the memory data object information is acquired by calling a reading interface of the memory data object and is queried together with the data in the disk, and the query support of the ClickHouse memory data is further realized.
In the process, memory data storage is added to the ClickHouse, data organization and writing are effectively carried out on the data in batches, writing one by one is avoided, data writing performance is effectively improved, memory data organization is directly carried out, the memory data organization is converted into a memory data object which can be processed by the ClickHouse, memory data falling is not carried out any more, time consumption caused by data falling during query is reduced, memory data do not need to be read in a disk during query, time consumption on IO is reduced, and data reading and querying performance is improved.
Meanwhile, the invention also provides a query support system based on the memory data in the ClickHouse database, a memory data query organization module and a memory data query processing module,
the memory data query organization module constructs a block data object, temporarily stores memory data, circularly searches the needed memory data by using an iterator, stores the memory data meeting the conditions into the block data object, carries out the sorting processing of the memory data in the block data object according to the table-building sorting mode of a ClickHouse database, calls a memory data writing interface of a ClickHouse folding tree engine, writes the memory data in the block data object into the memory data object of the ClickHouse,
the memory data query processing module inserts the memory data object information as metadata information into a list for storing the metadata information of the disk through an SQL execution flow of the ClickHouse, acquires the memory data of the disk in the list according to the metadata information, and queries and processes the memory data in the disk.
The information interaction, execution process and other contents between the modules in the system are based on the same concept as the method embodiment of the present invention, and specific contents can be referred to the description in the method embodiment of the present invention, and are not described herein again. Similarly, the system can directly organize the memory data, convert the memory data into the memory data object which can be processed by the ClickHouse, do not need to perform memory data falling, reduce the time consumption caused by data falling during query, reduce the time consumption on IO (input/output) because the memory data does not need to be read in a disk during query, and improve the data reading and querying performance.
When the system is applied, the system also comprises a memory data writing module, data is written into a memory table structure of the memory in a key-value format, log management is added, in addition, the memory data can be subjected to disk dropping according to a data volume threshold value, and batch data is written into the ClickHouse so as to be convenient for subsequent data query processing.
The present invention provides a query support device based on the data stored in the ClickHouse database, including: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine-readable program to execute the query support method based on the data stored in the clickwouse database.
The contents of information interaction, readable program process execution and the like of the processor in the device are based on the same concept as the method embodiment of the present invention, and specific contents can be referred to the description in the method embodiment of the present invention, and are not described herein again.
Similarly, the device directly organizes the memory data, converts the memory data into the memory data object which can be processed by the clickwouse, does not need to perform memory data disk dropping, reduces the time consumption caused by the data disk dropping during query, does not need to read the memory data in a disk during query, reduces the time consumption on IO, and improves the data reading and querying performance.
It should be noted that not all steps and modules in the processes and system structures in the preferred embodiments are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (9)
1. A query support method based on the memory data in the ClickHouse database is characterized in that the data objects of building blocks are used for temporarily storing the memory data,
using iterator to search needed memory data circularly, storing memory data meeting condition into block data object,
sorting the memory data in the block data object according to a ClickHouse database table-building sorting mode,
calling a memory data writing interface of a ClickHouse folding tree engine, writing the memory data in the block data object into the memory data object of the ClickHouse,
and inserting the memory data object information serving as metadata information into a list for storing the metadata information of the disk through an SQL execution flow of ClickHouse, acquiring the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
2. The ClickHouse database memory data-based query support method as claimed in claim 1, wherein the loop search for the required memory data is performed by:
constructing an initial value required by the iterator according to initial information of the memory data required to be inquired by the iterator, searching an initial position of the required memory data according to the initial value,
and circularly searching the next memory data by using the iterator, filtering the memory data which does not accord with the condition, storing the memory data which accords with the condition into the block data object until the memory data which does not accord with the condition is not obtained, and terminating the search.
3. The method for supporting query based on the stored data in the ClickHouse database according to claim 1 or 2, wherein the ClickHouse SQL executes a flow:
inserting the memory data object information as metadata information into a list for storing the metadata information of the disk,
parsing sql to generate a corresponding abstract syntax tree,
an interpreter for constructing the abstract syntax tree, and acquiring the metadata information of the storage disk through the interpreter,
and calling a reading interface of the memory data object to acquire the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
4. The ClickHouse database memory data-based query support method as claimed in claim 3, wherein the interpreter obtains storage disk metadata information:
generating a logical plan according to the interpreter, storing a list storing disk metadata information in the logical plan,
and generating a physical plan according to the logic plan,
and executing the physical plan, and calling a reading interface of the memory data object to acquire the metadata information in the list of the storage disk metadata information.
5. The query support system based on the memory data in the ClickHouse database is characterized in that a memory data query organization module and a memory data query processing module,
the memory data query organization module constructs a block data object, temporarily stores memory data, circularly searches the needed memory data by using an iterator, stores the memory data meeting the conditions into the block data object, carries out the sorting processing of the memory data in the block data object according to the table-building sorting mode of a ClickHouse database, calls a memory data writing interface of a ClickHouse folding tree engine, writes the memory data in the block data object into the memory data object of the ClickHouse,
the memory data query processing module inserts the memory data object information as metadata information into a list for storing the metadata information of the disk through an SQL execution flow of the ClickHouse, acquires the memory data of the disk in the list according to the metadata information, and queries and processes the memory data in the disk.
6. The ClickHouse-database-memory-data-based query support system according to claim 5, wherein the memory data query organization module cyclically searches for the required memory data:
constructing an initial value required by the iterator according to initial information of the memory data required to be inquired by the iterator, searching an initial position of the required memory data according to the initial value,
and circularly searching the next memory data by using the iterator, filtering the memory data which does not accord with the condition, storing the memory data which accords with the condition into the block data object until the memory data which does not accord with the condition is not obtained, and terminating the search.
7. The query support system based on the memory data in the ClickHouse database according to claim 5 or 6, wherein the memory data query processing module executes the ClickHouse SQL procedure:
inserting the memory data object information as metadata information into a list for storing the metadata information of the disk,
parsing sql to generate a corresponding abstract syntax tree,
an interpreter for constructing the abstract syntax tree, and acquiring the metadata information of the storage disk through the interpreter,
and calling a reading interface of the memory data object to acquire the memory data of the disk in the list according to the metadata information, and inquiring and processing the memory data in the disk.
8. The system according to claim 7, wherein the memory data query processing module executes a process of acquiring the metadata information of the storage disk by the interpreter in the ClickHouse SQL procedure:
generating a logical plan according to the interpreter, storing a list storing disk metadata information in the logical plan,
and generating a physical plan according to the logic plan,
and executing the physical plan, and calling a reading interface of the memory data object to acquire the metadata information in the list of the storage disk metadata information.
9. The query supporting device based on the memory data in the ClickHouse database is characterized by comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor, configured to invoke the machine readable program to perform the ClickHouse database memory data based query support method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110879192.8A CN113626464B (en) | 2021-08-02 | 2021-08-02 | Query supporting method and system based on ClickHouse database memory data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110879192.8A CN113626464B (en) | 2021-08-02 | 2021-08-02 | Query supporting method and system based on ClickHouse database memory data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113626464A true CN113626464A (en) | 2021-11-09 |
| CN113626464B CN113626464B (en) | 2024-07-12 |
Family
ID=78382127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110879192.8A Active CN113626464B (en) | 2021-08-02 | 2021-08-02 | Query supporting method and system based on ClickHouse database memory data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626464B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114328598A (en) * | 2021-11-29 | 2022-04-12 | 浪潮云信息技术股份公司 | Cache optimization method and system for pipeline based on ClickHouse database |
| CN114356923A (en) * | 2021-12-30 | 2022-04-15 | 山东浪潮科学研究院有限公司 | MVCC multi-version folding tree implementation system and method based on ClickHouse database |
| CN114880322A (en) * | 2022-04-21 | 2022-08-09 | 广州经传多赢投资咨询有限公司 | Financial data column type storage method, system, equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107329814A (en) * | 2017-06-16 | 2017-11-07 | 电子科技大学 | A kind of distributed memory database query engine system based on RDMA |
| CN110362572A (en) * | 2019-06-25 | 2019-10-22 | 浙江邦盛科技有限公司 | A kind of time series database system based on column storage |
| CN111522816A (en) * | 2020-04-16 | 2020-08-11 | 云和恩墨(北京)信息技术有限公司 | Data processing method, device, terminal and medium based on database engine |
| CN112115150A (en) * | 2020-08-03 | 2020-12-22 | 上海金仕达软件科技有限公司 | Data management method, terminal device and medium for embedded memory database |
| CN112269788A (en) * | 2020-11-13 | 2021-01-26 | 中盈优创资讯科技有限公司 | Method and device for improving click House data storage performance |
| CN112347108A (en) * | 2020-11-26 | 2021-02-09 | 上海天旦网络科技发展有限公司 | Data query method and system suitable for hybrid backend |
| CN112783927A (en) * | 2021-01-27 | 2021-05-11 | 浪潮云信息技术股份公司 | Database query method and system |
| CN112905595A (en) * | 2021-03-05 | 2021-06-04 | 腾讯科技（深圳）有限公司 | Data query method and device and computer readable storage medium |
| CN112966020A (en) * | 2021-03-08 | 2021-06-15 | 徐少锋 | Industrial high-frequency big data real-time processing system and method |
| CN113032423A (en) * | 2021-05-31 | 2021-06-25 | 北京谷数科技股份有限公司 | Query method and system based on dynamic loading of multiple data engines |
-
2021
- 2021-08-02 CN CN202110879192.8A patent/CN113626464B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107329814A (en) * | 2017-06-16 | 2017-11-07 | 电子科技大学 | A kind of distributed memory database query engine system based on RDMA |
| CN110362572A (en) * | 2019-06-25 | 2019-10-22 | 浙江邦盛科技有限公司 | A kind of time series database system based on column storage |
| CN111522816A (en) * | 2020-04-16 | 2020-08-11 | 云和恩墨(北京)信息技术有限公司 | Data processing method, device, terminal and medium based on database engine |
| CN112115150A (en) * | 2020-08-03 | 2020-12-22 | 上海金仕达软件科技有限公司 | Data management method, terminal device and medium for embedded memory database |
| CN112269788A (en) * | 2020-11-13 | 2021-01-26 | 中盈优创资讯科技有限公司 | Method and device for improving click House data storage performance |
| CN112347108A (en) * | 2020-11-26 | 2021-02-09 | 上海天旦网络科技发展有限公司 | Data query method and system suitable for hybrid backend |
| CN112783927A (en) * | 2021-01-27 | 2021-05-11 | 浪潮云信息技术股份公司 | Database query method and system |
| CN112905595A (en) * | 2021-03-05 | 2021-06-04 | 腾讯科技（深圳）有限公司 | Data query method and device and computer readable storage medium |
| CN112966020A (en) * | 2021-03-08 | 2021-06-15 | 徐少锋 | Industrial high-frequency big data real-time processing system and method |
| CN113032423A (en) * | 2021-05-31 | 2021-06-25 | 北京谷数科技股份有限公司 | Query method and system based on dynamic loading of multiple data engines |
Non-Patent Citations (1)
| Title |
|---|
| 乖乖猪001: "Clickhouse引擎四：内存引擎", pages 1 - 2, Retrieved from the Internet <URL:https://blog.csdn.net/xiaozhaoshigedasb/article/details/115757765> * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114328598A (en) * | 2021-11-29 | 2022-04-12 | 浪潮云信息技术股份公司 | Cache optimization method and system for pipeline based on ClickHouse database |
| CN114328598B (en) * | 2021-11-29 | 2024-06-11 | 上海沄熹科技有限公司 | Cache optimization method and system for pipeline based on ClickHouse database |
| CN114356923A (en) * | 2021-12-30 | 2022-04-15 | 山东浪潮科学研究院有限公司 | MVCC multi-version folding tree implementation system and method based on ClickHouse database |
| CN114880322A (en) * | 2022-04-21 | 2022-08-09 | 广州经传多赢投资咨询有限公司 | Financial data column type storage method, system, equipment and storage medium |
| CN114880322B (en) * | 2022-04-21 | 2023-02-28 | 广州经传多赢投资咨询有限公司 | Financial data column type storage method, system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113626464B (en) | 2024-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113626464A (en) | Query support method and system based on stored data in ClickHouse database | |
| CN110019218B (en) | Data storage and query method and equipment | |
| CN106202207B (en) | HBase-ORM-based indexing and retrieval system | |
| CN110659282B (en) | Data route construction method, device, computer equipment and storage medium | |
| CN109614432B (en) | System and method for acquiring data blood relationship based on syntactic analysis | |
| CN111400408A (en) | Data synchronization method, device, equipment and storage medium | |
| CN112163025B (en) | Database data export method and device, computer equipment and storage medium | |
| CN111563095B (en) | HBase-based data retrieval device | |
| CN106407360B (en) | Data processing method and device | |
| US20190188302A1 (en) | Group-by-time operations with returned time context | |
| CN112231321B (en) | Oracle secondary index and index real-time synchronization method | |
| CA2363187A1 (en) | Index sampled tablescan | |
| CN107291964A (en) | A kind of method that fuzzy query is realized based on HBase | |
| CN110083617B (en) | DDL statement processing method and device, electronic device and medium | |
| CN101290622A (en) | Database cleaning system and method | |
| CN113704575A (en) | SQL method, device, equipment and storage medium for analyzing XML and Java files | |
| CN113515564A (en) | Data access method, device, equipment and storage medium based on J2EE | |
| US20200301922A1 (en) | Multiform persistence abstraction | |
| CN108388589B (en) | Device for automatically generating sql query statement of database | |
| CN117131230A (en) | Data blood edge analysis method, device, equipment and storage medium | |
| CN116049193A (en) | Data storage method and device | |
| CN112346672B (en) | Log dyeing method, device, equipment and storage medium | |
| CN115687392A (en) | SQL statement optimized execution method and device, electronic equipment and medium | |
| CN115840803A (en) | Document retrieval method, system, computer device and storage medium | |
| US11023469B2 (en) | Value list compression (VLC) aware qualification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20221226 Address after: Room 305-22, Building 2, No. 1158 Zhangdong Road and No. 1059 Dangui Road, China (Shanghai) Pilot Free Trade Zone, Pudong New Area, Shanghai, 200120 Applicant after: Shanghai Yunxi Technology Co.,Ltd. Address before: 250100 No. 1036 Tidal Road, Jinan High-tech Zone, Shandong Province, S01 Building, Tidal Science Park Applicant before: Inspur cloud Information Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |