KR20180104498A - Method and apparatus for processing query - Google Patents
Method and apparatus for processing query Download PDFInfo
- Publication number
- KR20180104498A KR20180104498A KR1020170031361A KR20170031361A KR20180104498A KR 20180104498 A KR20180104498 A KR 20180104498A KR 1020170031361 A KR1020170031361 A KR 1020170031361A KR 20170031361 A KR20170031361 A KR 20170031361A KR 20180104498 A KR20180104498 A KR 20180104498A
- Authority
- KR
- South Korea
- Prior art keywords
- operator
- query
- llvm
- executor
- special
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24537—Query rewriting; Transformation of operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
Abstract
A query execution method of a query executor is provided. The query executor receives a physical query plan tree. The query executor determines whether the first operator among the operators included in the physical query plan tree is a special operator processed in a JIT (just in time) compilation method. The query executor determines whether there is a first low level virtual machine (LLVM) intermediate representation operator corresponding to the first operator when the first operator is the special operator. The query executor converts the first LLVM IR operator into a machine code and executes the first LLVM IR operator when the first LLVM IR operator exists.
Description
The present invention relates to a method and apparatus for query processing in the database field.
In a conventional database management system (DBMS), a query processing method generally includes parsing a query, checking errors, performing optimization, and executing an optimized query plan through a query executor Follow the process.
The process that the query executor finally executes among these query processing processes traditionally uses an interpreter type query executor. At this time, the interpreter-based query execution unit performs a conventional volcano-based iterator model query processing that circles each plan in the query execution plan tree and executes each query plan.
Such a volcano method calls a lot of function calls and has a disadvantage that the execution speed is slow. In order to improve the conventional iterative query execution method, there is a need for a query execution technique based on JIT (Just-In-Time) compilation. The JIT compilation-based query execution method is a compiler method, not an interpreter method.
If this JIT compilation method is used, the query execution speed can be improved. However, the JIT compilation method requires a separate compile time because the runtime compilation process is performed during the execution of the query. This approach may only be appropriate for OLAP (online analytical processing) queries because it may take longer to perform all execution plans in a JIT compilation manner in the case of an OLTP (online transaction processing) query.
However, when the OLTP query is performed, even if the relational operator with a low query processing load is processed through the existing interpreter method, there is little problem in performance. However, performance problems may arise if relational operators have complicated operations and specific operators (eg, join operators, sorting operators) that perform a lot of comparison operations are processed through the existing interpreter method. In particular, when a join operator is processed, many accesses to a table may be performed according to a join condition, and many functions may be called.
Even in the case of OLTP queries, the performance can be improved if the JIT compilation method is partially used, rather than the interpreter method, for such operators with a high computational load.
SUMMARY OF THE INVENTION The present invention provides a method and an apparatus for solving performance degradation according to an interpreter-based query processing method in a method of performing a query processing in a DBMS (for example, a method of executing an execution plan) will be.
Another object of the present invention is to provide a method and apparatus for reducing compilation time and improving overall performance.
Another object of the present invention is to provide a method and apparatus for improving the performance of a query processing using a JIT compilation method rather than an interpreter method for specific operators having a large computation load.
Another object of the present invention is to provide a method and an apparatus for executing a query execution plan in a general interpreter method for a general operator with a small load.
According to an embodiment of the present invention, a method for processing a query executor query is provided. The query processing method of a query executor includes: receiving a physical query plan tree; Determining whether the first operator among the operators included in the physical query plan tree is a special operator that is processed in a JIT (just in time) compilation method; If the first operator is the special operator, determining whether a first low level virtual machine (LLVM) intermediate representation (LLVM) operator corresponding to the first operator exists; And converting the first LLVM IR operator into a machine code and executing the first LLVM IR operator if the first LLVM IR operator exists.
The special operator may have a larger computational load than a general operator that is processed by an interpreter method.
The step of determining whether the first operator is the special operator may include determining the first operator as the special operator when the first operator exists in the pre-stored special operator list.
The list of special operators can be modified.
The special operator list can be updated based on the operation execution time of each operator executed for query processing.
The method of
The converting and executing the first LLVM IR operator into a machine code may include optimizing the first LLVM IR operator; And converting the optimized first LLVM IR operator to machine code.
The query processing method of the query executor may further include executing the first operator in the interpreter manner when the first operator is the general operator.
The query executor may be included in a query processor of a database management system (DBMS) server.
The physical query plan tree may be generated through optimization of a logical query plan configured for the query sentence.
According to another embodiment of the present invention, a query executor is provided. The query executor processes the first operator in a JIT (just in time) compilation method among the operators included in the physical query plan tree when a physical query plan tree is input. An operator manager for determining whether the operator is a special operator or an ordinary operator processed in an interpreter manner; An LLVM IR manager that verifies whether a first low level virtual machine (LLVM) intermediate representation operator corresponding to the first operator exists if the first operator is the special operator; An LLVM IR generator for converting the first operator into the first LLVM IR operator and transmitting the first operator to the LLVM IR manager when the first LLVM IR operator does not exist; And an LLVM IR cache manager for transferring the previously stored first LLVM IR operator to the LLVM IR manager when the first LLVM IR operator exists.
The query executor may further include an LLVM IR optimizer that optimizes the first LLVM IR operator when the first LLVM IR operator is received from the LLVM IR manager.
The query executor may further include an LLVM IR operator executor that converts the optimized first LLVM IR operator into machine code and executes the machine code.
The query executor may further include an interpreter operator executor that executes the first operator in the interpreter manner when the first operator is the general operator.
The operator manager can determine the first operator as the special operator when the first operator exists in the previously stored special operator list.
The operator manager can update the special operator list based on the execution time of each operator executed for query processing.
The operator manager can modify the special operator list according to a user's request.
According to the embodiment of the present invention, in the case where the query execution unit performs query processing on the execution plan tree in the query processing process, the performance improvement can be improved by applying the JIT compilation process to the operator with a heavy computation load.
Also, according to the embodiment of the present invention, by applying the interpreter method to a general operator having a small computational load, an efficient method can be performed according to operator characteristics. Through this, the overall performance can be improved.
1 is a diagram showing a general database system.
2 is a diagram showing a query processing process in the database system.
FIG. 3 is a diagram illustrating a query processing procedure and a detailed configuration necessary for the query processing according to an embodiment of the present invention.
4 is a diagram illustrating a process in which a query processor performs a query process according to an embodiment of the present invention.
5 is a diagram of a computing device, in accordance with an embodiment of the present invention.
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.
In the present specification, duplicate descriptions are omitted for the same constituent elements.
Also, in this specification, when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, May be present. On the other hand, in the present specification, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that no other element exists in between.
Furthermore, terms used herein are used only to describe specific embodiments and are not intended to be limiting of the present invention.
Also, in this specification, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.
Also, in this specification, the terms " comprise ", or " have ", and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, or combinations thereof.
Also, in this specification, the term 'and / or' includes any combination of the listed items or any of the plurality of listed items. In this specification, 'A or B' may include 'A', 'B', or 'both A and B'.
Hereinafter, a method and apparatus for maximizing query processing performance by strategically using an interpreter method and a JIT compilation method in a query processing method for a DBMS will be described. Specifically, a method and apparatus for efficient query processing based on JIT compilation for a special operator (eg, time-consuming and data-intensive operator) having a heavy query processing load will be described.
In particular, in the case of the OLTP query, since the execution time is generally shorter than that of the OLAP query, the query is mostly executed through the interpreter method. In this case, there is no big problem for the general operator, but the operation cost of a specific operator (for example, a join operator, a sort operator, etc.) which is complicated and performs a lot of comparison operations may occur. If the query for performing such an operation is executed through the interpreter method, there is a disadvantage that the overall query processing speed is rapidly deteriorated.
In order to overcome the drawbacks of performance degradation, a method of differentiating a general operator having a small operation cost and a specific operator having a large operation cost differently can be used in consideration of the operation cost of the operator in the query processing execution method. Concretely, a method of providing the function of selecting the optimum processing method according to the type of operator by using the interpreter method and the JIT compilation method in parallel can be used. In this way, optimal query performance can be improved. For example, in a query processing method for an OLTP query, a different query execution method can be used for a general operator and a specific operator (for example, a high-load operator such as a join operator, a sorting operator, etc.) A query processing method of a compilation method other than an interpreter method may be applied.
Meanwhile, the method according to the embodiment of the present invention is not limited to a specific DBMS system. For example, the method according to an embodiment of the present invention can be widely applied not only to a disk-based DBMS but also to an in-memory-based DBMS.
1 is a diagram showing a general database system.
Specifically, FIG. 1 illustrates an overall configuration of a general database system as a basis for describing a query processing method according to an embodiment of the present invention.
1, the
The
The
The
Next, the detailed module (or detailed configuration) included in each of the
1, the
The
The
The
The
1, the
The
The
The
The
The
1, the
2 is a diagram showing a query processing process in the database system.
Specifically, FIG. 2 illustrates a general query processing process performed by the
As illustrated in FIG. 2, when a query sentence (e.g., SQL query) inquired by the user is input (S201), the
The
The
After all the operators are executed, the query results generated are passed to the user.
FIG. 3 is a diagram illustrating a query processing procedure and a detailed configuration necessary for the query processing according to an embodiment of the present invention.
Specifically, FIG. 3 illustrates a process and module (or configuration) for performing a JIT compilation for a particular operator.
In FIG. 3, the
In order to perform the query processing, a
Depending on the type of operator in the physical query plan, computing costs for operator processing are different. In particular, in the case of an operator such as a join, a cost for query processing may be increased depending on a join condition. In addition, even in the case of an operator such as sorting, since a lot of comparison operations are performed among a plurality of data, a computation query processing tendency tends to occur. In the case of these special operators (for example, join operator, sorting operator, group-by operator, etc.), the processing cost is higher than that of general operators. When the conventional interpreter method is applied to such a large-load operator (special operator), a continuous function call occurs, which causes a lot of operation costs and the processing speed is remarkably slow.
In order to overcome this problem, a method of performing an interpreter-based query processing for a general operator and a query processing of a JIT compilation method for a special expensive operator will be described.
The
The physical query plan optimized by the
The
The
When the
Basically, at the time of system startup, a function of selecting, registering and providing special operators to be subjected to JIT compilation processing among the operators analyzed in advance may be provided. Alternatively, when the system is executed, a function may be provided in which a user adds another operator to the special operators to be provided in the existing JIT compilation process, or the operator deletes the operator in the special operators to be JIT compiled. These functions may be provided by the
As a result of the determination by the
The
If the
If the
The
In order to improve the performance, the LLVM IR optimizer 109a-5 performs an operator optimization process on the LLVM IR operator to be performed (S311). At this time, the LLVM IR optimizer 109a-5 can perform various optimizations by configuring various optimization algorithms through the optimization function provided by the LLVM framework.
The LLVM
If the
4 is a diagram illustrating a process in which a query processor performs a query process according to an embodiment of the present invention.
Specifically, FIG. 4 illustrates a process in which the
In Fig. 4, the
The
The
In step S404, the
The
If the LLVM IR operator cache does not exist, the
The
The
If it is determined in operation S404 that the corresponding operator is not a special operator that is a JIT compilation processing object but a general operator, the
The
The
If there is no next operator to be executed, the
5 is a diagram of a computing device, in accordance with an embodiment of the present invention. The computing device TN100 of FIG. 5 may correspond to the
In the embodiment of FIG. 5, the computing device TN100 may include at least one processor (TN 110), a transceiver (TN 120) in communication with the network, and a memory (TN 130). Further, the computing device TN100 may further include a storage device TN140, an input interface device TN150, an output interface device TN160, and the like. The components included in the computing device TN100 may be connected by a bus (TN170) and communicate with each other.
The processor TN110 may execute a program command stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may refer to a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. The processor TN110 may be configured to implement the procedures, functions, and methods described in connection with the embodiments of the present invention. The processor TN110 may control each component of the computing device TN100.
Each of the memory TN130 and the storage device TN140 may store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may be constituted of at least one of a volatile storage medium and a nonvolatile storage medium. For example, the memory TN130 may be configured with at least one of a read only memory (ROM) and a random access memory (RAM).
The transceiver apparatus TN120 can transmit or receive a wired signal or a wireless signal.
On the other hand, the embodiments of the present invention are not only implemented by the apparatuses and / or methods described so far, but may also be realized through a program realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium on which the program is recorded And such an embodiment can be easily implemented by those skilled in the art from the description of the embodiments described above.
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, It belongs to the scope of right.
Claims (17)
Determining whether the first operator among the operators included in the physical query plan tree is a special operator that is processed in a JIT (just in time) compilation method;
If the first operator is the special operator, determining whether a first low level virtual machine (LLVM) intermediate representation (LLVM) operator corresponding to the first operator exists; And
Converting the first LLVM IR operator into a machine code and executing the first LLVM IR operator if the first LLVM IR operator exists
And a query executor having a query executor.
The special operator
Compared to general operators that are processed in an interpreter manner,
Query processing method of query executor.
Wherein the step of determining whether the first operator is the special operator comprises:
Determining the first operator as the special operator when the first operator exists in the pre-stored special operator list,
The list of special operators
Query processing method of query executor.
In the special operator list,
Is updated based on the operation execution time of each operator executed for query processing
Query processing method of query executor.
Converting the first operator into the first LLVM IR operator and converting the first LLVM IR operator into machine code and executing the first LLVM IR operator if the first LLVM IR operator does not exist
Wherein the query executor further comprises a query executor.
The step of converting the first LLVM IR operator into a machine code and executing the first LLVM IR operator,
Optimizing the first LLVM IR operator; And
And converting the optimized first LLVM IR operator to machine code
Query processing method of query executor.
Executing the first operator in the interpreter manner if the first operator is the general operator;
Wherein the query executor further comprises a query executor.
The query execution unit,
A database management system (DBMS) server includes a query processor
Query processing method of query executor.
The physical query plan tree comprises:
Generated by optimization of the logical query plan that is constructed for the query sentence
Query processing method of query executor.
An LLVM IR manager that verifies whether a first low level virtual machine (LLVM) intermediate representation operator corresponding to the first operator exists if the first operator is the special operator;
An LLVM IR generator for converting the first operator into the first LLVM IR operator and transmitting the first operator to the LLVM IR manager when the first LLVM IR operator does not exist; And
An LLVM IR cache manager for transferring the previously stored first LLVM IR operator to the LLVM IR manager when the first LLVM IR operator exists;
A query executor that includes a query executor.
An LLVM IR optimizer for optimizing the first LLVM IR operator when receiving the first LLVM IR operator from the LLVM IR manager;
A query executor further comprising:
An LLVM IR operator executor that converts the optimized first LLVM IR operator into machine code and executes it
A query executor further comprising:
An interpreter operator executor for executing the first operator in the interpreter manner when the first operator is the general operator;
A query executor further comprising:
The operator manager,
Determining that the first operator is the special operator when the first operator exists in the pre-stored special operator list,
The operator manager,
The list of special operators is updated based on the execution time of each operator executed for query processing
The query executor.
The operator manager,
The special operator list is modified according to the user's request
The query executor.
The physical query plan tree comprises:
Generated by optimization of the logical query plan that is constructed for the query sentence
The query executor.
Wherein the operator manager, the LLVM IR manager, the LLVM IR generator, and the LLVM IR cache manager,
Used for a query processor in a database management system (DBMS) server.
The query executor.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020170031361A KR20180104498A (en) | 2017-03-13 | 2017-03-13 | Method and apparatus for processing query |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020170031361A KR20180104498A (en) | 2017-03-13 | 2017-03-13 | Method and apparatus for processing query |
Publications (1)
Publication Number | Publication Date |
---|---|
KR20180104498A true KR20180104498A (en) | 2018-09-21 |
Family
ID=63720829
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
KR1020170031361A KR20180104498A (en) | 2017-03-13 | 2017-03-13 | Method and apparatus for processing query |
Country Status (1)
Country | Link |
---|---|
KR (1) | KR20180104498A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20240078233A1 (en) * | 2022-09-02 | 2024-03-07 | Oracle International Corporation | Automatic error mitigation in database statements using alternate plans |
WO2024063817A1 (en) * | 2022-09-20 | 2024-03-28 | Oracle International Corporation | Producing natively compiled query plans by recompiling existing c code through partial evaluation |
-
2017
- 2017-03-13 KR KR1020170031361A patent/KR20180104498A/en unknown
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20240078233A1 (en) * | 2022-09-02 | 2024-03-07 | Oracle International Corporation | Automatic error mitigation in database statements using alternate plans |
WO2024063817A1 (en) * | 2022-09-20 | 2024-03-28 | Oracle International Corporation | Producing natively compiled query plans by recompiling existing c code through partial evaluation |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107247808B (en) | Distributed NewSQL database system and picture data query method | |
US11423001B2 (en) | Technique of efficiently, comprehensively and autonomously support native JSON datatype in RDBMS for both OLTP and OLAP | |
US20190303379A1 (en) | Method and system for transparent interoperability between applications and data management systems | |
US7933894B2 (en) | Parameter-sensitive plans for structural scenarios | |
US7676453B2 (en) | Partial query caching | |
EP3254191B1 (en) | Apparatus and method for using parameterized intermediate representation for just-in-time compilation in database query execution engine | |
CN105718593A (en) | Database query optimization method and system | |
CN108369591B (en) | System and method for caching and parameterizing IR | |
US11176133B2 (en) | Filter evaluation for table fragments | |
US8180791B2 (en) | Combining streaming and navigation for evaluating XML queries | |
US8073843B2 (en) | Mechanism for deferred rewrite of multiple XPath evaluations over binary XML | |
US10635666B2 (en) | System, method and device for index compilation to optimize index usage | |
CN102760143A (en) | Method and device for dynamically integrating executing structures in database system | |
CN110019314B (en) | Dynamic data packaging method based on data item analysis, client and server | |
CN111078961A (en) | Multi-data source query driving system, method, device and storage medium | |
US11645281B1 (en) | Caching query plans in database systems | |
US10664475B2 (en) | Generating a native access plan for semi join operators | |
US20180218039A1 (en) | Query planning and execution with reusable memory stack | |
KR20180104498A (en) | Method and apparatus for processing query | |
US10558661B2 (en) | Query plan generation based on table adapter | |
US11016973B2 (en) | Query plan execution engine | |
US8832157B1 (en) | System, method, and computer-readable medium that facilitates efficient processing of distinct counts on several columns in a parallel processing system | |
US9135302B2 (en) | Query rewrite with a nested materialized view | |
US20240045868A1 (en) | Method and device for accelerating database operation | |
US20230141902A1 (en) | Columnar Cache Query Using Hybrid Query Execution Plan |