CN113742359A - Method and device for inquiring presence, electronic equipment and storage medium - Google Patents
Method and device for inquiring presence, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113742359A CN113742359A CN202110069711.4A CN202110069711A CN113742359A CN 113742359 A CN113742359 A CN 113742359A CN 202110069711 A CN202110069711 A CN 202110069711A CN 113742359 A CN113742359 A CN 113742359A
- Authority
- CN
- China
- Prior art keywords
- query
- sql statement
- sql
- hoc
- statement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 230000002776 aggregation Effects 0.000 claims description 41
- 238000004220 aggregation Methods 0.000 claims description 41
- 238000004590 computer program Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 7
- 230000001360 synchronised effect Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000006116 polymerization reaction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24539—Query rewriting; Transformation using cached or materialised query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses an ad hoc query method, an ad hoc query device, electronic equipment and a storage medium. The ad hoc query method comprises the following steps: determining a query engine based on a first Structured Query Language (SQL) statement; SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine; and performing data query in the query engine based on the second SQL statement, and returning a query result.
Description
Technical Field
The present application relates to the field of big data technologies, and in particular, to an ad hoc query method, an ad hoc query device, an electronic device, and a storage medium.
Background
Ad Hoc query (Ad Hoc) is a data query method in which a user flexibly selects a query condition according to the user's needs, and a system generates a corresponding statistical form according to the user's selection. In general, the query flow of ad hoc queries is: and mapping the dimension table and the fact table in the data warehouse to a semantic layer, selecting the table by the semantic layer, establishing association between the dimension table and the fact table by a user, finally generating a query statement, and querying by the query statement. Ad hoc queries generally require query time, and the corresponding query engines are mostly based on memory queries. In the related art, the query efficiency of ad hoc query is low.
Disclosure of Invention
In view of this, embodiments of the present application provide an ad hoc query method, an ad hoc query device, an electronic device, and a storage medium, so as to solve the problem of low ad hoc query efficiency in the related art.
In order to achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:
the embodiment of the application provides an ad hoc query method, which comprises the following steps:
determining a Query engine based on a first Structured Query Language (SQL) statement;
SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine;
and performing data query in the query engine based on the second SQL statement, and returning a query result.
In the above scheme, the query engine is a Presto query engine or a Clickhouse query engine.
In the above scheme, the first SQL statement satisfies a first set condition; the first set condition represents that the first SQL statement is sent out by a routing layer.
In the foregoing solution, the determining a query engine based on a first SQL statement includes:
routing the first SQL statement to an aggregation layer for query;
determining a corresponding query engine based on the first SQL statement if the first SQL statement does not match a corresponding query result at the aggregation layer.
In the foregoing solution, the determining a query engine based on a first SQL statement includes:
determining a query engine corresponding to the first SQL statement based on first information carried in the first SQL statement; wherein,
the first information represents the timeliness requirement corresponding to the first SQL statement.
In the foregoing solution, the routing the first SQL statement to the aggregation layer for querying includes:
under the condition that the first SQL statement does not meet a second set condition, routing the first SQL statement to the aggregation layer, and performing data query on the aggregation layer; wherein,
and the second set condition represents that the first SQL statement is matched with the corresponding query result in the cache region.
In the above scheme, the method further comprises:
and returning a query result under the condition that the first SQL statement meets the second set condition.
In the above solution, the ad hoc query method according to claim 6 or 7, wherein the method further includes:
and under the condition that the grammar of the first SQL statement conforms to the set grammar, performing data query in the cache region.
In the above scheme, the method further comprises:
under the condition that the length of the first SQL statement is larger than a set threshold, splitting the first SQL statement into at least one sub SQL statement;
determining a query result of the first SQL statement based on the query result of the sub-SQL statement;
wherein the length of each sub SQL statement in the at least one sub SQL statement is not greater than the set threshold.
The embodiment of the present application further provides an ad hoc query device, where the device includes:
a determining unit for determining a query engine based on the first SQL statement;
the rewriting unit is used for performing SQL rewriting on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine;
and the query unit is used for performing data query in the query engine based on the second SQL statement and returning a query result.
An embodiment of the present application further provides an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein,
the processor is adapted to perform the steps of any of the above methods when running the computer program.
An embodiment of the present application further provides a storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the steps of any one of the above methods.
In an embodiment of the present application, a query engine is determined by a first Structured Query Language (SQL) based statement; SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine; and performing data query in the query engine based on the second SQL statement, and returning a query result, so that even if the business side inputs a query statement which is not matched with the query engine, the query system can perform corresponding rewriting on the query statement according to the query engine so as to complete data query in the query engine, thereby improving the efficiency of ad hoc query.
Drawings
Fig. 1 is a schematic flow chart illustrating an implementation of an ad hoc query method according to an embodiment of the present application;
fig. 2 is a schematic flow chart illustrating an implementation of an ad hoc query method according to an embodiment of the present application;
fig. 3 is a schematic diagram of an ad hoc query device according to an embodiment of the present application;
fig. 4 is a schematic diagram of a hardware component structure of an electronic device according to an embodiment of the present application.
Detailed Description
The query instruction of the ad hoc query is temporarily produced by the user during use and is a loose type command/query. The query result obtained by the ad hoc query depends on a certain variable, and the query result is different according to different values of the variable when the query command is executed each time.
The ad hoc query is usually queried according to a dynamically programmed SQL statement, that is, the SQL query statement is short-term and is created when the query system is run, and the query system cannot optimize the SQL query statement in advance, so the ad hoc query is an important index for evaluating the running performance of the data warehouse.
Ad hoc queries generally require query time, and the corresponding query engines are mostly based on memory queries. In the related art, ad hoc query has a plurality of query engines, but different query engines have their own characteristics and own unique SQL syntax, so when using these query engines, a business party needs to know and learn the SQL syntax of each query engine, and can complete data query after inputting SQL statements adapted to the query engines, which brings great inconvenience to the business party and also makes ad hoc query inefficient.
Based on this, the embodiment of the application provides an ad hoc query method, an ad hoc query device, electronic equipment and a storage medium, wherein a query engine is determined by a first Structured Query Language (SQL) statement; SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine; and performing data query in the query engine based on the second SQL statement, and returning a query result, so that even if the business side inputs a query statement which is not matched with the query engine, the query system can perform corresponding rewriting on the query statement according to the query engine so as to complete data query in the query engine, thereby improving the efficiency of ad hoc query.
The present application will be described in further detail with reference to the following drawings and examples.
Fig. 1 is a schematic flow chart illustrating an implementation of an ad hoc query method according to an embodiment of the present application. As shown in fig. 1, the method includes:
step 101: a query engine is determined based on a first structured query language, SQL, statement.
In one embodiment, the first SQL statement satisfies a first set condition; the first set condition represents that the first SQL statement is sent out by a routing layer.
Here, when the first SQL statement is issued by the routing layer, it indicates that the query process needs to be completed in the detail table, and the query in the detail table needs to be completed by the query engine, so the query engine is determined based on the first SQL statement. Wherein the detail table comprises a dimension table and a fact table; the dimension table is a collection table of dimension attributes and is an entrance for entering the fact table; the fact table is a basic table of a dimension model and is also a core table in a data warehouse structure, the fact table is tightly designed around the business process, and the business process is expressed by acquiring the measurement describing the business process, wherein the measurement comprises the referenced dimension and the measurement related to the business process.
By determining the query engine when the first SQL statement meets the first set condition, the corresponding query engine can be determined when the detailed table query is needed, and the efficiency of the ad hoc query is improved.
In one embodiment, the query engine is a Presto query engine or a Clickhouse query engine.
Here, the Presto query engine is a distributed query engine based on a memory, in Presto, all query processes and calculation processes are executed in the memory, Presto itself does not store data, but can access multiple data sources, and supports cascading queries across the data sources, so that the analysis and calculation requirements of a large data volume can be met, and the concurrency is strong.
The Clickhouse query engine is a columnar database query engine for Online Analytical Processing (OLAP) that performs a computational process on a memory basis. The click house query performance is strong, multiple query functions are supported, and the query speed is extremely high. The Clickhouse cannot be directly adapted to the existing data source, and external data needs to be imported when the Clickhouse is used.
By providing two query engines of Presto and Clickhouse, more service scenes can be met according to the requirements of service parties, and the cost of calculation and storage is reduced.
In one embodiment, the determining a query engine based on a first SQL statement, the method comprising:
routing the first SQL statement to an aggregation layer for query;
determining a corresponding query engine based on the first SQL statement if the first SQL statement does not match a corresponding query result at the aggregation layer.
Here, the aggregation layer is a summarized data layer, and according to a query scenario of a business party, the query system identifies a common aggregation dimension, aggregates the fact table and the dimension table to a certain extent, and slightly aggregates the table based on a materialized view on the basis of aggregation, so that the tables, such as a weekly report, a monthly report, an annual report, and the like, all belong to an aggregation table. The common polymerization methods include counting, summing and averaging. A materialized view is a database object that includes the results of a query, is a local copy of a remote database, and is used to generate a summary table based on the summation of data tables. The method is realized by a materialized view after connecting some tables which are long in time consumption, and the query efficiency can be improved.
Since the aggregation layer slightly summarizes the data, the first SQL statement is routed to the aggregation layer for query, and when the aggregation layer does not match the corresponding query result, it is indicated that the aggregation layer cannot meet the query request of the first SQL statement, and further query is required.
The first SQL statement is routed to the aggregation layer for query, after the aggregation layer is not matched with the corresponding query result, the corresponding query engine is determined based on the first SQL statement, and query can be performed according to the query engine in time when the aggregation layer cannot meet the query requirement, so that the query efficiency is improved.
In one embodiment, the determining a query engine based on a first SQL statement, the method comprising:
determining a query engine corresponding to the first SQL statement based on first information carried in the first SQL statement; wherein,
the first information represents the timeliness requirement corresponding to the first SQL statement.
Here, when the timeliness requirement in the first information carried in the first SQL statement is low, it indicates that the service party has a low requirement on real-time performance, and the minute-level response is also acceptable, so that the Presto query engine having good concurrency and not needing to consider the data source is preferentially selected for data query.
When the timeliness requirement in the first information carried in the first SQL statement is high, it indicates that the service party has a high requirement on real-time performance and needs a response of second level or even sub-second level, so the Clickhouse query engine with a high query speed is preferentially selected for data query.
By determining the corresponding query engine based on the first information carried in the first SQL statement, the query engine more conforming to the current business scene can be flexibly selected according to the requirement of a business party, and the query efficiency of the ad hoc query is improved.
In an embodiment, the routing the first SQL statement to an aggregation layer for querying includes:
under the condition that the first SQL statement does not meet a second set condition, routing the first SQL statement to the aggregation layer, and performing data query on the aggregation layer; wherein,
and the second set condition represents that the first SQL statement is matched with the corresponding query result in the cache region.
Here, when the first SQL statement does not match the corresponding query result in the cache region, it indicates that the cache region cannot meet the query requirement, and further query is required, and at this time, the first SQL statement is routed to the aggregation layer where the mild summary is performed, and data query is performed in the aggregation layer.
The first SQL statement is routed to the aggregation layer to be queried under the condition that the first SQL statement does not meet the second set condition, if the aggregation layer has the query result corresponding to the first SQL statement, the query result is returned, further query is not needed, the data amount of the query can be reduced, the query steps can be reduced, and the query efficiency of the ad hoc query is improved.
In an embodiment, the method further comprises:
and returning a query result under the condition that the first SQL statement meets the second set condition.
Here, if the first SQL statement matches the corresponding query result in the cache region, it indicates that the cache region can satisfy the query request of this time, and therefore, the corresponding query result matched in the cache region is directly returned.
By directly returning the query result under the condition that the first SQL statement meets the second set condition, further query is not needed, query steps are reduced, and query efficiency is improved.
In an embodiment, the method further comprises:
and under the condition that the grammar of the first SQL statement conforms to the set grammar, performing data query in the cache region.
Here, after receiving the query request of the first SQL statement, the query system first checks whether the syntax of the first SQL statement conforms to the set syntax, if not, it means that the first SQL statement is invalid, and cannot perform data query according to the first SQL statement, in this case, a piece of prompt information is returned to prompt that the query is wrong. Otherwise, if the syntax of the first SQL statement conforms to the set syntax, the next query step is carried out, namely, data query is carried out in the cache region.
By carrying out data query in the cache region under the condition that the grammar of the first SQL sentence conforms to the set grammar, the query system can be ensured to carry out data query based on the correct SQL sentence, and the query efficiency is improved.
Step 102: SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement characterizes an SQL statement recognizable by the query engine.
Here, the first SQL statement is rewritten to obtain a second SQL statement, and after receiving the first SQL statement sent by the routing layer, the first SQL statement may be an SQL statement that meets the standard syntax but does not meet the syntax of the to-be-executed engine, and after determining the to-be-executed query engine, the first SQL statement needs to be rewritten according to the SQL syntax of the to-be-executed query engine, so that the rewritten second SQL statement can be adapted to the to-be-executed query engine.
Step 103: and performing data query in the query engine based on the second SQL statement, and returning a query result.
After the rewriting of the first SQL statement is completed, data query is carried out in the determined query engine to be executed based on a second SQL statement which can be adapted to the query engine to be executed, and a query result is returned.
Fig. 2 is a schematic diagram of an implementation flow of the ad hoc query method provided in the application embodiment of the present application, please refer to fig. 2, after receiving a query request from a service party, check the syntax of an SQL statement first, after the check fails, prompt that the query fails, and perform the next query; after the verification is successful, firstly judging whether a query result matched with the SQL statement exists in the cache region, and directly returning the query result after the matching is successful; under the condition of failed matching, the routing module routes the SQL statement to an aggregation layer, whether a corresponding query result exists in the aggregation layer is matched, and after the matching is successful, the query result is returned; under the condition that the matching of the aggregation layer fails, the routing module acquires a query engine corresponding to the SQL statement; under the condition that the obtained query engine is Presto, rewriting the SQL statement based on Presto, finishing data query in Presto based on the rewritten SQL statement, and returning a query result; and under the condition that the obtained query engine is Clickhouse, rewriting the SQL statement based on the Clickhouse, completing data query in the Clickhouse based on the rewritten SQL statement, and returning a query result.
In an embodiment of the present application, a query engine is determined by a first Structured Query Language (SQL) based statement; SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine; and performing data query in the query engine based on the second SQL statement, and returning a query result, so that even if the business side inputs a query statement which is not matched with the query engine, the query system can perform corresponding rewriting on the query statement according to the query engine so as to complete data query in the query engine, thereby improving the efficiency of ad hoc query.
In an embodiment, the method further comprises:
under the condition that the length of the first SQL statement is larger than a set threshold, splitting the first SQL statement into at least one sub SQL statement;
determining a query result of the first SQL statement based on the query result of the sub-SQL statement;
wherein the length of each sub SQL statement in the at least one sub SQL statement is not greater than the set threshold.
Here, the set threshold is the maximum length of the SQL statements that can be supported by the query engine, and when the length of the first SQL statement is greater than the maximum length of the query engine, the query engine does not support the query of the first SQL statement, which may result in a query failure, in this case, the query system splits the first SQL statement into at least one sub-SQL statement according to the time sequence of the received line number of the first SQL statement, where the length of each sub-SQL statement is not greater than the set threshold. And summarizing the query result of the first SQL statement based on the query results of all the sub SQL statements.
By splitting the first SQL statement with the length larger than the set threshold, the length limitation of single query of the query engine is avoided, and more query scenes of a business side can be met.
It should be noted that, in the embodiment of the present application, the query system is open to the service party through a Remote Procedure Call (RFC) interface and a hypertext Transfer Protocol (HTTP) interface, and the service party only needs to input an SQL statement that conforms to a standard syntax in one of the two interfaces, so that data query can be started by using the query system. The RFC interface is an interface for remotely requesting one node to provide services for another node.
In the embodiment of the present application, the syntax of the first SQL statement is checked, and the first SQL statement is rewritten, which are all processed based on the call of the SQL processor, and the call processor can provide a whole set of processing flow: parsing (Parser) - > validation (valid) - > optimization (Optimize) - > execution (Execute), in the above embodiment, the query system mainly utilizes validation and optimization functions of the call processor, on one hand, verifying the accuracy of the syntax of the SQL statement, and on the other hand, performing different adaptations and optimizations for different query engines.
To implement the method according to the embodiment of the present application, an ad hoc query device is further provided in the embodiment of the present application, fig. 3 is a schematic diagram of the ad hoc query device provided in the embodiment of the present application, please refer to fig. 3, where the device includes:
a determining unit 301, configured to determine a query engine based on the first SQL statement.
A rewriting unit 302, configured to perform SQL rewriting on the first SQL statement to obtain a second SQL statement; the second SQL statement characterizes an SQL statement recognizable by the query engine.
A query unit 303, configured to perform data query in the query engine based on the second SQL statement, and return a query result.
In one embodiment, the query engine is a Presto query engine or a Clickhouse query engine.
In one embodiment, the first SQL statement satisfies a first set condition; the first set condition represents that the first SQL statement is sent out by a routing layer.
In an embodiment, the determining unit 301 is further configured to route the first SQL statement to an aggregation layer for querying;
determining a corresponding query engine based on the first SQL statement if the first SQL statement does not match a corresponding query result at the aggregation layer.
In an embodiment, the determining unit 301 is further configured to determine, based on first information carried in the first SQL statement, a query engine corresponding to the first SQL statement; wherein,
the first information represents the timeliness requirement corresponding to the first SQL statement.
In an embodiment, the querying unit 303 is further configured to route the first SQL statement to the aggregation layer and perform data query in the aggregation layer when the first SQL statement does not satisfy a second set condition; wherein,
and the second set condition represents that the first SQL statement is matched with the corresponding query result in the cache region.
In an embodiment, the querying unit 303 is further configured to return a query result when the first SQL statement satisfies the second set condition.
In an embodiment, the querying unit 303 is further configured to query the data in the cache area when the syntax of the first SQL statement conforms to a set syntax.
In an embodiment, the apparatus further includes a splitting unit, configured to split the first SQL statement into at least one sub-SQL statement when the length of the first SQL statement is greater than a set threshold;
determining a query result of the first SQL statement based on the query result of the sub-SQL statement;
wherein the length of each sub SQL statement in the at least one sub SQL statement is not greater than the set threshold.
In practical applications, the determining Unit 301, the rewriting Unit 302, the querying Unit 303, and the splitting Unit may be implemented by a Processor in a terminal, such as a Central Processing Unit (CPU), a Digital Signal Processor (DSP), a Micro Control Unit (MCU), or a Programmable Gate Array (FPGA).
It should be noted that: the ad hoc query device provided in the above embodiment is only exemplified by the division of the program modules when displaying information, and in practical applications, the processing may be distributed to different program modules as needed, that is, the internal structure of the device may be divided into different program modules to complete all or part of the processing described above. In addition, the ad hoc query device provided by the above embodiment and the ad hoc query method embodiment belong to the same concept, and specific implementation processes thereof are described in the method embodiment and are not described herein again.
Based on the hardware implementation of the program module, in order to implement the method of the embodiment of the present application, an embodiment of the present application further provides an electronic device. Fig. 4 is a schematic diagram of a hardware component structure of an electronic device according to an embodiment of the present application, and as shown in fig. 4, the electronic device includes:
a communication interface 401 capable of performing information interaction with other devices such as network devices and the like;
and the processor 402 is connected with the communication interface 401 to implement information interaction with other devices, and is used for executing the method provided by one or more technical schemes of the terminal side when running a computer program. And the computer program is stored on the memory 403.
In particular, the processor 402 is configured to determine a query engine based on a first structured query language, SQL, statement; SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine; and performing data query in the query engine based on the second SQL statement, and returning a query result.
In one embodiment, the query engine is a Presto query engine or a Clickhouse query engine.
In one embodiment, the first SQL statement satisfies a first set condition; the first set condition represents that the first SQL statement is sent out by a routing layer.
In an embodiment, the processor 402 is further configured to route the first SQL statement to an aggregation layer for querying; determining a corresponding query engine based on the first SQL statement if the first SQL statement does not match a corresponding query result at the aggregation layer.
In an embodiment, the processor 402 is further configured to determine, based on first information carried in the first SQL statement, a query engine corresponding to the first SQL statement; and the first information represents the aging requirement corresponding to the first SQL statement.
In an embodiment, the processor 402 is further configured to route the first SQL statement to the aggregation layer and perform data query in the aggregation layer when the first SQL statement does not satisfy a second set condition; and the second set condition represents that the first SQL statement is matched with the corresponding query result in the cache region.
In an embodiment, the processor 402 is further configured to return a query result if the first SQL statement satisfies the second set condition.
In an embodiment, the processor 402 is further configured to perform data query in the cache region if the syntax of the first SQL statement conforms to a set syntax.
In an embodiment, the processor 402 is further configured to split the first SQL statement into at least one sub-SQL statement if the length of the first SQL statement is greater than a set threshold; determining a query result of the first SQL statement based on the query result of the sub-SQL statement; wherein the length of each sub SQL statement in the at least one sub SQL statement is not greater than the set threshold.
Of course, in practice, the various components in the electronic device are coupled together by the bus system 404. It is understood that the bus system 404 is used to enable communications among the components. The bus system 404 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 404 in FIG. 4.
The memory 403 in the embodiments of the present application is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program for operating on an electronic device.
It will be appreciated that the memory 403 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. Among them, the nonvolatile Memory may be a Read Only Memory (ROM), a Programmable Read Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a magnetic random access Memory (FRAM), a Flash Memory (Flash Memory), a magnetic surface Memory, an optical disk, or a Compact Disc Read-Only Memory (CD-ROM); the magnetic surface storage may be disk storage or tape storage. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Enhanced Synchronous Dynamic Random Access Memory (Enhanced DRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory (DRmb Access), and Random Access Memory (DRAM). The memory 403 described in embodiments herein is intended to comprise, without being limited to, these and any other suitable types of memory.
The method disclosed in the embodiments of the present application may be applied to the processor 402 or implemented by the processor 402. The processor 402 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 402. The processor 402 described above may be a general purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. Processor 402 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in the memory 403, and the processor 402 reads the program in the memory 403 and performs the steps of the aforementioned method in conjunction with its hardware.
The processor 402 executes the program to implement the corresponding flow in the methods of the embodiments of the present application.
In an exemplary embodiment, the present application further provides a storage medium, specifically a computer-readable storage medium, for example, a memory 403 storing a computer program, which can be executed by the processor 402 to perform the steps of the foregoing method. The computer readable storage medium may be Memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface Memory, optical disk, or CD-ROM.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus, terminal and method may be implemented in other manners. The above-described device embodiments are only illustrative, for example, the division of the unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof that contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling an electronic device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (12)
1. An ad hoc query method, comprising:
determining a query engine based on a first Structured Query Language (SQL) statement;
SQL rewriting is carried out on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine;
and performing data query in the query engine based on the second SQL statement, and returning a query result.
2. The ad hoc query method according to claim 1,
the query engine is a Presto query engine or a Clickhouse query engine.
3. The ad hoc query method according to claim 1,
the first SQL statement meets a first set condition; the first set condition represents that the first SQL statement is sent out by a routing layer.
4. The ad hoc query method according to claim 1 or 2, wherein the query engine is determined based on a first SQL statement, the method comprising:
routing the first SQL statement to an aggregation layer for query;
determining a corresponding query engine based on the first SQL statement if the first SQL statement does not match a corresponding query result at the aggregation layer.
5. The ad hoc query method of claim 1, wherein said determining a query engine based on a first SQL statement, the method comprising:
determining a query engine corresponding to the first SQL statement based on first information carried in the first SQL statement; wherein,
the first information represents the timeliness requirement corresponding to the first SQL statement.
6. The ad hoc query method of claim 4, wherein said routing the first SQL statement to an aggregation layer for querying comprises:
under the condition that the first SQL statement does not meet a second set condition, routing the first SQL statement to the aggregation layer, and performing data query on the aggregation layer; wherein,
and the second set condition represents that the first SQL statement is matched with the corresponding query result in the cache region.
7. The ad hoc query method of claim 6, wherein the method further comprises:
and returning a query result under the condition that the first SQL statement meets the second set condition.
8. The ad hoc query method according to claim 6 or 7, wherein the method further comprises:
and under the condition that the grammar of the first SQL statement conforms to the set grammar, performing data query in the cache region.
9. The ad hoc query method of claim 1, wherein the method further comprises:
under the condition that the length of the first SQL statement is larger than a set threshold, splitting the first SQL statement into at least one sub SQL statement;
determining a query result of the first SQL statement based on the query result of the sub-SQL statement;
wherein the length of each sub SQL statement in the at least one sub SQL statement is not greater than the set threshold.
10. An ad hoc query device, comprising:
a determining unit for determining a query engine based on the first SQL statement;
the rewriting unit is used for performing SQL rewriting on the first SQL statement to obtain a second SQL statement; the second SQL statement represents an SQL statement recognizable by the query engine;
and the query unit is used for performing data query in the query engine based on the second SQL statement and returning a query result.
11. An electronic device, comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein,
the processor is adapted to perform the steps of the method of any one of claims 1-9 when running the computer program.
12. A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the steps of the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110069711.4A CN113742359A (en) | 2021-01-19 | 2021-01-19 | Method and device for inquiring presence, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110069711.4A CN113742359A (en) | 2021-01-19 | 2021-01-19 | Method and device for inquiring presence, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113742359A true CN113742359A (en) | 2021-12-03 |
Family
ID=78728225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110069711.4A Pending CN113742359A (en) | 2021-01-19 | 2021-01-19 | Method and device for inquiring presence, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113742359A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114547086A (en) * | 2022-04-22 | 2022-05-27 | 网易（杭州）网络有限公司 | Data processing method, device, equipment and computer readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040215604A1 (en) * | 2003-04-23 | 2004-10-28 | International Business Machines Corporation | System and method for querying a data source |
| CN101706810A (en) * | 2009-11-23 | 2010-05-12 | 北京中创信测科技股份有限公司 | Method and device for querying databases |
| US10157234B1 (en) * | 2015-02-27 | 2018-12-18 | Quest Software Inc. | Systems and methods for transforming datasets |
| CN110969024A (en) * | 2018-09-30 | 2020-04-07 | 北京奇虎科技有限公司 | Query statement rewriting method and device |
| CN112000688A (en) * | 2020-08-14 | 2020-11-27 | 杭州数云信息技术有限公司 | Query method and query system based on universal query language |
-
2021
- 2021-01-19 CN CN202110069711.4A patent/CN113742359A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040215604A1 (en) * | 2003-04-23 | 2004-10-28 | International Business Machines Corporation | System and method for querying a data source |
| CN101706810A (en) * | 2009-11-23 | 2010-05-12 | 北京中创信测科技股份有限公司 | Method and device for querying databases |
| US10157234B1 (en) * | 2015-02-27 | 2018-12-18 | Quest Software Inc. | Systems and methods for transforming datasets |
| CN110969024A (en) * | 2018-09-30 | 2020-04-07 | 北京奇虎科技有限公司 | Query statement rewriting method and device |
| CN112000688A (en) * | 2020-08-14 | 2020-11-27 | 杭州数云信息技术有限公司 | Query method and query system based on universal query language |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114547086A (en) * | 2022-04-22 | 2022-05-27 | 网易（杭州）网络有限公司 | Data processing method, device, equipment and computer readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109617646B (en) | Message conversion method and device, computer equipment and computer readable storage medium | |
| WO2019062010A1 (en) | Semantic recognition method, electronic device and computer readable storage medium | |
| WO2020192141A1 (en) | Method and device for simulating interface, computer apparatus, and storage medium | |
| CN113360519B (en) | Data processing method, device, equipment and storage medium | |
| US11263542B2 (en) | Technologies for auto discover and connect to a rest interface | |
| CN112905620B (en) | Data query method and device, electronic equipment and storage medium | |
| CN113901083A (en) | Heterogeneous data source operation resource analysis positioning method and equipment based on multiple analyzers | |
| CN116166700A (en) | Query method, device, system, electronic equipment and storage medium | |
| CN108959294B (en) | Method and device for accessing search engine | |
| CN113626558B (en) | Intelligent recommendation-based field standardization method and system | |
| CN109241157B (en) | Data calling method and device, communication equipment and storage medium | |
| CN113742359A (en) | Method and device for inquiring presence, electronic equipment and storage medium | |
| CN113190236B (en) | HQL script verification method and device | |
| CN114358596A (en) | Index calculation method and device | |
| CN109471849B (en) | Model processing method and device, storage medium and electronic equipment | |
| CN114020789A (en) | Bill data query method, system, device and storage medium | |
| CN114090514A (en) | Log retrieval method and device for distributed system | |
| CN114020446A (en) | Cross-multi-engine routing processing method, device, equipment and storage medium | |
| CN113792055A (en) | Data processing method, electronic device and storage medium | |
| CN113032836A (en) | Data desensitization method and apparatus | |
| CN115437906A (en) | Test method and device | |
| CN112597119A (en) | Method and device for generating processing log and storage medium | |
| WO2019095582A1 (en) | Method and device for navigating to target location, storage medium and terminal | |
| CN115186674B (en) | Aviation failure case management method, device, equipment and storage medium | |
| CN117251472B (en) | Cross-source data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |