KR20210007665A - Apparatus and method for a query processing for in-memomry data in numa-based hardware - Google Patents

Abstract

Provided is a query processing method for in-memory data in a multiprocessor system environment of the NUMA structure, which comprises the steps of: configuring a query operation group based on a dependency relationship between query operations and the number of threads allocated to query processing; determining at least one candidate NUMA node in which a thread to perform query processing of the query operation group is to be executed based on a distribution rate for each NUMA node of query operation group input data; and determining a NUMA node in which a thread to perform query processing of the query operation group is executed in consideration of the candidate NUMA node and the NUMA node in which the thread is located.

Description

Query processing method and device for in-memory data in hardware with non-uniform memory access (NUMA) structure {APPARATUS AND METHOD FOR A QUERY PROCESSING FOR IN-MEMOMRY DATA IN NUMA-BASED HARDWARE}

The present invention relates to a query processing method and apparatus, and more particularly, to a query processing method and apparatus for in-memory data in a hardware environment having a non-uniform memory access (NUMA) structure in which memory device access speed is non-uniform. .

A multiprocessor system with a NUMA structure consists of a number of NUMA nodes, and a number of CPUs, caches of multiple layers, memory controllers, and links between CPUs are packaged in one NUMA node, and several NUMA nodes are interconnected by link between nodes. Connected. The NUMA node network has a fully connected structure in which all NUMA nodes can access one hop distance or a partial connection structure that can access several hop distances.

In a multiprocessor architecture, NUMA characteristics have different access delay speeds between local memory that can be accessed directly through a memory controller and remote memory that can be accessed through a link between NUMA nodes when accessing memory.

In an in-memory database system mounted on a multiprocessor system with a NUMA structure, the data access speed has a large effect on the query processing performance, so it is necessary to schedule queries in consideration of the data loading location.

Therefore, it is necessary to minimize remote memory access by scheduling queries so that query processing is performed in the CPU of the NUMA node where data is stored. In addition, in order to place a thread running on another NUMA node on a specific NUMA node, thread transfer costs are incurred, so the cost of transferring a thread is compared with the benefits of reducing remote memory access. There is a need.

In other words, the data locality should be considered so that the memory in which the data is stored and the CPU to perform the query processing are located in the same NUMA node, and the benefit of performing the query processing in a specific NUMA node is higher than the cost of transferring a thread to a specific NUMA node. Should be scheduled.

Also, since query scheduling provided by the operating system is not efficient in a specific environment such as a database system, it is necessary to schedule a query in consideration of the database service environment and database system structure.

The requirements of an environment that dynamically allocates resources like a cloud service environment are different from the case where system resources for database services are already secured and operated, and the optimal query scheduling method is also different depending on the data management and query processing method of the database system. . That is, the optimal query scheduling method differs according to the goal to be achieved, such as improving query processing performance or reducing necessary resources, and the structure of the database system.

Therefore, in a system where resources are pre-allocated for database service, to improve query processing performance, data is partitioned and loaded into in-memory for each partition, and query processing is performed only on the CPU belonging to the NUMA node where the partition accessed by the query is stored. A method of controlling has been proposed. However, in this method, remote memory access is minimized, but it may cause performance degradation due to a specific CPU overload, and since queries that access partitions stored in different NUMA nodes are not considered, partitions must be reconfigured and relocated periodically.

In addition, for an environment in which data is dynamically loaded into in-memory and operated, a method of dynamically controlling a CPU set in which a query processing thread is to be operated is proposed in consideration of memory allocation history and CPU utilization rate for each database server. However, in this method, CPU overload is prevented, but performance may be degraded depending on the storage location of the data accessed in the query operation.

An object of the present invention to solve the above problems is to provide a NUMA-aware query scheduling method to improve query processing performance in an environment in which data is loaded into in-memory for each partition.

Another object of the present invention for solving the above problems is to provide query scheduling that minimizes remote memory access and can cope with resource overload while providing a response to queries that access data partitions stored in several NUMA nodes. have.

Another object of the present invention for solving the above problems is to reflect the distribution ratio of NUMA nodes of access data referenced in the query operation, the number of threads allocated for query processing, and the resource status of the NUMA node, and the number of threads to perform query processing is It is to provide a way to control the NUMA node to be executed.

A method for processing a query for in-memory data in a non-uniform memory access (NUMA) multiprocessor system environment according to an embodiment of the present invention for achieving the above object, comprising: a dependency relationship between query operations and Configuring a query operation group based on the number of threads allocated to perform query processing; Determining at least one candidate NUMA node in which a thread to perform query processing of the query operation group is to be executed based on a distribution ratio of the query operation group input data for each NUMA node; And determining a NUMA node in which a thread to perform query processing of a query operation group is executed in consideration of the candidate NUMA node and the NUMA node in which the thread is located.

According to an embodiment of the present invention, it has an advantage of improving query processing performance for in-memory data by minimizing remote memory access in a hardware environment of a NUMA structure.

1 is a block diagram of a database service system to which the present invention is applied.
2 is a block diagram of a database server to which the present invention is applied.
3 is a flowchart of a method of selecting a NUMA node based on access data for each NUMA node according to an embodiment of the present invention.
4 is a conceptual diagram illustrating a method of configuring an operation group according to an embodiment of the present invention.
5 is a conceptual diagram illustrating NUMA-aware query scheduling according to a query processing environment according to an embodiment of the present invention.
6 shows a query scheduling process of the NUMA-aware query scheduler according to an embodiment of the present invention.
7 is a flowchart illustrating a query processing method according to an embodiment of the present invention.
8 is a hardware block diagram of a query processing apparatus according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. The term "and/or" includes a combination of a plurality of related stated items or any of a plurality of related stated items.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a database service system to which the present invention is applied.

The database service system may include a database server 1000, an in-memory database 2000, and a persistent database 3000.

Referring to FIG. 1, the database server 1000 may be operated on a multiprocessor system composed of two or more NUMA nodes 2100, and the in-memory database 2000 includes one or more data partitions in the memory of each NUMA node. 2200 may be loaded, and the database server 1000 may process a query processing request for the in-memory database 2000. The query may be a query that targets only the data loaded in one NUMA node, or it may be a query that targets data loaded in several NUMA nodes. The in-memory database 2000 may fetch data from the persistent DB 3000 or store the data in the persistent DB when necessary for continuous storage and management of data.

2 is a block diagram of a database server to which the present invention is applied.

The database server 1000 stores data in an in-memory DB and a persistent DB, a data storage 1200 in charge of concurrency control and transaction management, and a query processing device in charge of query parsing, query optimization, and query processing execution functions. It may be configured to include (1100). The query processing apparatus may include a query parser 1110, a query optimizer 1120, a query execution NUMA node selector 1130, a NUMA-aware query scheduler 1140, and a query executer 1150.

The query parser 1110 may parse the query to generate a parse tree, and the query optimizer 1120 may generate a query processing execution plan by converting from the parse tree to a refined query and performing cost-based optimization. The query execution NUMA node selector 1130 may divide into a query operation group and select a candidate NUMA node in which a thread to process a query of the query operation group will be executed based on a query processing execution plan. The NUMA-aware query scheduler 1140 may be in charge of controlling a query processing thread to be executed in a selected candidate NUMA node by minimizing thread transfer in consideration of the state of the candidate NUMA node. The query executer 1150 may play a role of actually performing query processing.

3 is a flowchart of a method of selecting a NUMA node based on access data for each NUMA node according to an embodiment of the present invention.

In selecting a NUMA node for allocating a thread to perform query processing, first, the total access data size is calculated from the access data size for each NUMA node (S301), and a ratio of the total access data size for each NUMA node is calculated (S302). Subsequently, it is analyzed whether the ratio of the total access data size to each NUMA node is greater than or equal to the reference value (S303), and if it is greater than or equal to the reference value, the candidate NUMA node list is added (S304). Next, it is checked whether the analysis of all NUMA nodes has been completed (S305), and if the analysis is completed, the final NUMA node is selected (S306) and is terminated. The final NUMA node can be one or several nodes.

It may be beneficial to perform query processing in a NUMA node where access data is biased based on the size of access data for each NUMA node of data accessed in each query operation. In addition, when the cost of transferring a thread is less than the cost of accessing a remote memory in another NUMA node, it may be beneficial to perform query processing by transferring a thread to a specific NUMA node based on the size of access data for each NUMA node. For example, in an in-memory DB consisting of 4 NUMA nodes, if the access data size is 100 and the access data size for each NUMA node is (50, 25, 25, 0), the access data size for each NUMA node is ( 28, 25, 25, 22), it may be more beneficial to transfer the thread to node 0 to perform query processing.

Therefore, the selection of candidate NUMA nodes to perform query processing can be selected when the access data distribution ratio of the NUMA node is greater than or equal to the reference value. That is, when there is a NUMA node with an access data size that exceeds the reference value compared to the total access data size, it can be selected as a candidate NUMA node to perform query processing. The reference value may be selected in consideration of the number of NUMA nodes in the operating system environment.

가 기준치 이상인 노드로 선정될 수 있다. The candidate NUMA node may be selected using Equation 1. Candidate NUMA nodes are

May be selected as a node that is equal to or greater than the reference value.

는 NUMA 노드 i의 접근 데이터 규모, n은 NUMA 노드의 수,

는 전체 접근 데이터 규모 대비 NUMA 노드 i의 접근 데이터 규모 비율을 의미한다.here,

Is the size of the access data of NUMA node i, n is the number of NUMA nodes,

Is the ratio of the size of the access data of NUMA node i to the size of the total access data.

For example, when the size of the access data for each NUMA node is (45, 40, 10, 5), the difference in gains between performing query processing in node 0 and performing query processing in node 1 may be small. Therefore, only candidate NUMA nodes with the highest access data size may be selected as the final NUMA node, or the final NUMA node depending on the resource situation during the actual query processing by selecting all candidate NUMA nodes whose access data size ratio exceeds the standard value. You can also select. If the final NUMA node is selected according to the resource situation during actual query processing, the selection of candidate NUMA nodes for the query after the query processing execution result is used must be dynamically executed immediately before the query processing is executed.

The size of the accessed data can be predicted by considering the size of the input data and the type of query operation that refers to the input data. That is, if the input data is the result of processing the preceding query operation, the size of the data accessed may be the size of the predicted data for the data to be generated as a result of processing the preceding query operation, and if the input data is persistent data, consider the number of records and data type. The size of the access data can be estimated. In addition, the set of accessed data may be the entire input data or part of the input data according to the type of query operation referring to the input data. For example, if a query operation with a specific value is an index-based query operation, the size of the input data can be predicted by accessing only a few index records and data designated by the index instead of accessing all data belonging to the accessed data set.

Estimating the size of the access data based on the result of the query operation processing can be calculated using the same criteria as the selectivity factor, and the size of the access data based on the result of the query operation processing can be predicted by the query optimizer 1120. In the present invention, it is assumed that the size of the access data is already known.

4 is a conceptual diagram illustrating a method of configuring a query operation group according to an embodiment of the present invention.

In the present invention, a NUMA node to perform query processing may be selected for each query operation group by configuring a query operation group. In other words, if a query operation that uses only the result of the preceding query operation according to the dependency relationship between query operations, the query processing can be performed in the same NUMA node as the preceding query operation, so a query operation group can be formed together with the preceding query operation.

Dividing query operations into query operation groups consists of query operations that are optimal to perform query processing in the NUMA node, such as the preceding query operation, and can be performed as follows. First, when calling a separately configured query block, such as calling a stored procedure, a query operation group can be divided for each query block. Then, the relationship between the query operations is based on a query block composed of DAG (Directed Acyclic Graph), and the query operation that accesses persistent data constitutes a new group, and the query operation that only accesses the result of the previous query operation is immediately previous. A query operation that composes the same group as the query operation and considers the processing results of several preceding query operations constitutes a new group.

For example, as shown in the conceptual diagram of FIG. 4, when a query block has a graph dependency relationship between 10 query operations consisting of an e1 query operation to an e10 query operation, it may be composed of four query operation groups. Since the e1 query operation or the e8 query operation is a query operation that accesses persistent data, it can be composed of operation group 1 and operation group 2, respectively. Since query operations e2 to e7 are query operations that consider only the result of the previous query operation processing, they can be composed of operation group 1. Since the e9 query operation and the e10 query operation consider the processing results of several preceding query operations, they can be composed of operation group 3 and operation group 4, respectively.

The NUMA node that performs query processing on the operation group may be selected according to the location of the input data among data referenced within the operation group. Accordingly, a NUMA node that performs query processing for an operation group can be selected based on the distribution ratio of the operation group input data by NUMA node.

5 is a conceptual diagram illustrating NUMA-aware query scheduling according to a query processing environment according to an embodiment of the present invention.

In the present invention, NUMA-aware query scheduling may be performed based on a dependency relationship between a NUMA node performing query processing of an operation group and an operation group. Query scheduling can be performed in consideration of the sequential processing environment and the parallel processing environment.

In the sequential processing environment, the next operation group that the thread will perform query processing is selected by searching for the next operation group in the dependency relationship graph between operation groups in DFS (Depth First Search) method, so the operation group can be selected and assigned to the thread in the order in which the query will be processed. have. In addition, in a parallel processing environment, a thread and an operation group having a high locality among the next candidate operation groups having a dependency relationship may be preferentially selected and assigned to the thread as the next operation group to be processed by the thread. That is, the next operation group in which the thread performs query processing may be selected in the order of the nearest NUMA node in consideration of whether the query processing NUMA node and the NUMA node in which the thread is located are the same, or in consideration of the topology between NUMA nodes.

If the NUMA node and the NUMA node where the thread is located are different from each other, the selected NUMA node determines the resource status based on the selected NUMA node information before performing the query processing, and if it is not overloaded, the thread to perform the query processing is selected. The environment can be set so that query processing is performed within the selected NUMA node after moving to. When the resource of the selected NUMA node is overloaded, query processing is performed according to the scheduling policy of the operating system.

For example, when the query processing execution plan consisting of operation group 1 to operation group 8 in which the NUMA node is selected for query processing is sequentially processed by referring to the conceptual diagram of FIG. 5, a depth first search method is used. One thread is allocated according to the dependency relationship between them and query processing is performed in the order of operation group 1-operation group 2-operation group 4-operation group 3-operation group 5-operation group 7-operation group 6-operation group 8. In addition, a thread transfer occurs to node 2 to perform query processing of operation group 2 after performing query processing of operation group 1, and query processing of operation group 3 is performed after query processing of operation group 4 is performed. For this, a thread transfer to node 1 occurs.

In the case of parallel processing of the query processing execution plan, since there is no dependency relationship between the operation groups in the group in the four bundles (510 520, 530, 540) that group the operation groups, the next operation immediately after the query processing of the preceding operation group is completed. Group query processing can be performed. For example, when three threads are allocated, a thread located at nodes 0, 2, and 1, respectively, may be in charge of operation group 1, operation group 2, and operation group 3 to perform query processing at the same time. In other words, operation group 4 can perform query processing when operation group 1 and operation group 2 query processing is completed. In operation group 4, since node 2 is the optimal query processing node, node 2 performs query processing of operation group 2 Allocates threads so that the executing thread can take care of it. In order for the three operation groups in bundle 2 520 to perform query processing at the same time, the optimal execution node of operation group 5 and operation group 6 is 1, so the thread in node 0 is assigned to perform query processing, and the thread is assigned to node 1. Is transferred to perform query processing.

6 shows a query scheduling process of the NUMA-aware query scheduler according to an embodiment of the present invention.

The NUMA-aware query scheduler according to an embodiment of the present invention selects a query operation group to be scheduled by a thread based on the dependency relationship between the query operation groups, the query processing candidate NUMA node of the query operation group, and the NUMA node where the thread is located ( S601), it is checked whether the candidate NUMA node of the computation group and the NUMA node in which the thread is located match (S602).

Then, if the candidate NUMA node of the computation group and the NUMA node where the thread is located do not match, the candidate NUMA node is checked for overload (S603), and if it is not overload, the operating environment of the thread currently processing the query is selected. It is set as a node (S604). The thread operating environment setting is limited to the CPU on which the thread will be executed to the selected candidate NUMA node, and memory is preferentially allocated to the selected candidate NUMA node, and if memory is insufficient in the selected candidate NUMA node, memory can be allocated to other NUMA nodes. Set to If the candidate NUMA node is overloaded, the operating environment of the thread currently executing the query processing is reset to all NUMA nodes (S605), and the query processing is assigned to the query scheduling policy of the operating system. However, the location of the expected result data may be different from the result data generated by the query processing of the operation group by the operating system query scheduling and the query processing of the operation group in the selected candidate NUMA node. Therefore, since the NUMA node previously selected for the next operation group may not be the optimal node for performing query processing, a dynamic NUMA node selection policy may be applied.

After setting the thread operating environment, the query processor 1150 performs query processing of the query operation group (S606). Query scheduling is performed until query processing of all query operation groups constituting the query processing execution plan is completed (S607), and when query processing of all query operation groups is completed, the thread operating environment is set to all NUMAs for the next query processing. The node is reset (S608).

If the number of operation groups configured based on the dependency relationship between query operations is too large and the number of allocated threads is small, query scheduling based on the operation group may incur excessive thread transfer costs. In this case, it is possible to reduce the number of query scheduling target operation groups through the composition of the integrated operation group by integrating the operation groups.

The integrated operation group reconstructs the operation group in consideration of the relationship between the operation groups again based on the integration of the two terminal operation groups, and may be repeatedly performed until an appropriate number of operation groups is formed.

That is, the reconstruction of the operation group may combine the unified terminal operation group and the subsequent operation group when the subsequent operation group using the unified terminal operation group uses only the result of one operation group.

In addition, the selection of an integration target among the terminal operation groups may be performed in the following priority order. First, the end operation groups having the same query processing NUMA node are first integrated. Then, if there is no end-operation group having the same query processing NUMA node, operation groups with high redundancy of the subsequent operation group using the end-operation group are first integrated.

7 is a flowchart illustrating a query processing method according to an embodiment of the present invention.

In a query processing method according to an embodiment of the present invention, first, a query operation group is configured in consideration of a dependency relationship between query operations and the number of threads allocated to perform query processing (S710). Then, a candidate NUMA node is selected based on the ratio of the query operation group input data by NUMA node (S720), and the NUMA node to perform query processing of the query operation group is selected by comparing the candidate NUMA node and the NUMA node in which the thread is located. It is determined (S730).

8 is a hardware block diagram of a query processing apparatus according to an embodiment of the present invention.

The query processing apparatus 1100 according to an embodiment of the present invention includes a processor 1102 and a memory 1101 for storing at least one instruction executed through the processor and a result of the instruction execution, and a communication module 1103 can do.

Here, the at least one instruction includes: an instruction for causing the processor to configure a query operation group based on a dependency relationship between query operations and the number of threads allocated to perform query processing; An instruction for determining a candidate NUMA node to be executed by a thread that will process a query of the query operation group based on a distribution ratio of the query operation group input data for each NUMA node; An instruction for determining a NUMA node in which a thread to perform query processing of a query operation group is to be executed in consideration of the candidate NUMA node and the NUMA node in which the thread is located; It may include.

In addition, the communication module 1103 may collect data through a connection network and transmit the data to the processor under the control of the processor. The communication module may be a device including hardware and software necessary for transmitting and receiving signals such as control signals or data signals through wired or wireless connection with other network devices.

The operation of the method according to an embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. In addition, the computer-readable recording medium may be distributed over a computer system connected through a network to store and execute a computer-readable program or code in a distributed manner.

In addition, the computer-readable recording medium may include a hardware device specially configured to store and execute program commands, such as ROM, RAM, and flash memory. The program instructions may include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

While some aspects of the invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, where a block or apparatus corresponds to a method step or characteristic of a method step. Similarly, aspects described in the context of a method can also be represented by a corresponding block or item or a feature of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, a programmable computer or electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

Although the above has been described with reference to the preferred embodiments of the present invention, those skilled in the art will be able to variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can.

Claims (1)

As a query processing method for in-memory data in a non-uniform memory access (NUMA) multiprocessor system environment,
Configuring a query operation group based on a dependency relationship between query operations and the number of threads allocated for performing query processing;
Determining at least one candidate NUMA node in which a thread to perform query processing of the query operation group is to be executed based on a distribution ratio of the query operation group input data for each NUMA node; And
And determining a NUMA node in which a thread to perform query processing of a query operation group is to be executed in consideration of the candidate NUMA node and the NUMA node in which the thread is located.

Images