WO2018083751A1

WO2018083751A1 - Intelligent storage subsystem

Info

Publication number: WO2018083751A1
Application number: PCT/JP2016/082558
Authority: WO
Inventors: 浩平海外
Original assignee: 浩平海外
Priority date: 2016-11-02
Filing date: 2016-11-02
Publication date: 2018-05-11

Abstract

[Problem] To provide a storage subsystem of an SSD base with which it is possible to efficiently process access to an RDBMS in line format or column format, while minimizing storage of duplicate data. [Solution] An SSD command transmitted to a storage subsystem is expanded and information pertaining to a RDBMS column is included. On the basis of the column information, conversion from a line format to a column format is dynamically executed on the storage subsystem side. In addition, unnecessary blocks are not read on the basis of the index, whereby further efficiency is achieved.

Description

Intelligent storage subsystem

The present invention relates to a storage subsystem that improves the performance of an RDBMS, and more particularly to a storage subsystem that improves performance by offloading database processing and converting from a row format to a column format.

RDBMS (relational database system) has become a common technology in today's information and communication technology field. In most information systems, RDBMS-related processing accounts for a large proportion of the total processing, so improving the efficiency leads to improvement in the performance of the entire system, and provides great value.

RDBMS is characterized by storing data in a table format (table) consisting of rows (rows) and columns (columns). RDBMS data configuration methods are broadly divided into row-oriented and column-oriented types. The row-oriented type is a more general method and stores data in units of rows. The efficiency of processing for reading and writing the entire row as in online transaction processing is high, but the efficiency is low in analysis processing for extracting only a small number of columns of data from a long row. The column-oriented type has the opposite characteristics. For this reason, in many RDBMS, measures are taken such as adopting one of the data construction methods for narrowing the application, or reorganizing column-oriented data from row-oriented data and storing redundant data ( For example, Patent Document 1). However, the former method has a problem in that performance in a poor application is deteriorated, and the latter method has a problem of a time lag caused by data reorganization or an increase in storage requirement due to redundant data.

Patent Publication Gazette Special Table 2010-539616

To provide a storage subsystem that can speed up RDBMS access without storing redundant data.

The present invention is a storage system that includes a DB offload processing execution unit and a storage device, and performs relational database read processing,
The DB offload processing execution unit converts the block read from the storage device into a format consisting only of the data in the relational database based on the received command including information on the relational database column, and transmits the converted block. The above problem is solved by providing a storage system.

The invention of the present application is a storage system that includes a DB offload processing execution unit and a storage device, and performs a read process of a relational database,
Provided is a storage system in which the DB offload processing execution unit reads and transmits only blocks in a range specified by the index from the storage device based on the received command including information on the index of the relational database. To solve the above problem.

The present invention solves the above problem by providing the storage system according to Paragraph 0006 or Paragraph 0007, in which the transmission destination is a main memory of a graphic processing unit.

The invention of the present application is a device driver program that is executed on a host computer connected to a storage system that includes a DB offload processing execution unit and a storage device and performs a read process of a relational database,
An instruction for transmitting a command including information on a column to be read in the relational database to the DB offload processing execution unit;
The above problem is solved by providing a device driver program including a command for receiving data converted into a column format based on information on the column to be read from the DB offload processing execution unit.

The invention of the present application is a device driver program that is executed on a host computer connected to a storage system that includes a DB offload processing execution unit and a storage device and performs a read process of a relational database,
A command including information related to the index of the relational database, and a command for transmitting the command to the DB offload processing execution unit;
The above-mentioned problem is solved by providing a device driver program including an instruction for receiving, from the DB offload processing execution unit, data consisting only of blocks in the range specified by the index transmitted from the DB offload processing execution unit To do.

The present invention further includes a command for directly writing the received block to the main memory of the graphic processing unit.
The above problem is solved by providing the storage system according to paragraph 0009 or paragraph 0010.

The present invention also includes a DB offload processing execution unit and a storage device, and is a computer-implemented method for acquiring data from a storage system that performs a relational database read process,
Sending a command including information about a column to be read of the relational database;
Converting data read from the storage device into column format data based on information about the column to be read;
The above-mentioned problem is solved by providing a method including the step of transmitting the column format data.

The present invention also includes a DB offload processing execution unit and a storage device, and is a computer-implemented method for acquiring data from a storage system that performs a relational database read process,
Sending a command including information about an index of the relational database;
Reading only blocks in the range specified by the index;
The above problem is solved by providing a method including the step of transmitting the read block.

The present invention further includes a step of directly writing the transmitted block into a main memory of a graphic processing unit.
The above problem is solved by providing the method described in paragraph 0012 or paragraph 0013.

RDBMS access can be accelerated without storing redundant data.

It is a figure showing the whole structure of the 1st Example of the storage subsystem which concerns on this invention. It is a figure showing the outline of the flow of the data of the 1st Example of the storage subsystem which concerns on this invention, and a command. It is a figure showing the example of the extended SSD command used in the 1st Example of the storage subsystem which concerns on this invention. It is a figure showing the detail of the flow of a process in the 1st Example of the storage subsystem which concerns on this invention. It is a figure showing the flow of the data of the 2nd Example of the storage subsystem concerning this invention, and a command flow. It is a figure for demonstrating the index relevant to the 3rd Example of the storage subsystem which concerns on this invention. It is a figure showing the example of the extended SSD command used in the 3rd Example of the storage subsystem which concerns on this invention.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the overall structure of a first embodiment of the storage subsystem according to the present invention. The host computer (101) is a general computer server machine or a server machine dedicated to storage processing called NAS (Network-Attached Storage), and has an RDBMS (relational database management system) (102), driver software (103 ), An operating system (not shown), and an application program (not shown). The driver software (103) may include a file system function. A DMA buffer (105) is provided on the main memory (104) of the host computer (101) to temporarily store data used for storage input / output.

The storage subsystem (121) is typically a storage device using SSD (solid-state drive), and it is integrated with the host computer (101) even if it is independent of the host computer (101). It may be a device. The storage subsystem (121) has a command / data transmission / reception unit (122), an SSD control unit (123), and a storage / read / write unit for controlling storage according to a command (command) transmitted by the driver software (103). The database processing offload execution unit (124) is provided. The DB processing offload execution unit (124) is a component of the SSD control unit (123), and performs optimization in the database access processing according to the present invention, thereby improving the efficiency of processing of the RDBMS (102). The command / data transmission / reception unit (122), the SSD control unit (123), and the DB processing offload execution unit (124) may be realized by a dedicated processor that generally runs a program called microcode or firmware. The DB processing offload execution unit (124) is preferably realized by dedicated hardware such as FPGA (Field Programmable Grid Array) to ensure the maximum speed. The SSD storage block (125) is a means for storing data in units of blocks, and is not limited to SSD, but can use any random access nonvolatile memory technology such as flash memory or DRAM with power backup. May be used. Further, it may be a conventional HDD (Hard Disk Disk Drive). The host computer (101) and the storage subsystem (121) may be connected via a storage network such as Fiber Channel or Gigabit Ethernet (Ethernet is a registered trademark), but the host computer (101) and the storage subsystem ( 121) are preferably connected in the high-speed I / O bus (130) in the case.

FIG. 2 shows the flow of data and commands in the first embodiment of the storage subsystem according to the present invention. (S201) The driver software (103) converts the database access request of the RDBMS (102) into a command for accessing the SSD storage block (125), and the command / data transmission / reception unit (122 ). Here, the SSD access command is extended from the standard specification, and includes information about which column the database access request of the RDBMS (102) targets (details will be described later).

(S202) The SSD control unit (123) reads the SSD storage block (125) based on the SSD access command passed from the command / data transmission / reception unit (122). (S203) The read data is passed to the DB processing offload execution unit (124), and the DB processing offload execution unit (124) is required based on the database column information included in the SSD access command. Extract column information only. (S204) The command / data transmission / reception unit (122) receives data consisting only of columns required for the RDBMS (102) from the DB processing offload execution unit (124), and transmits it to the driver software (203). The driver software (103) places the data in the DMA buffer and completes the RDBMS (102) database access request (when the host computer (101) and the storage subsystem are connected via a high-speed I / O bus, This processing is preferably performed directly using the DMA mechanism without using the driver software (103).

Especially in analytical applications, RDBMS (102) database access requests often require only a small number of columns from very long rows of data. In such a case, it is inefficient to send the entire row data from the storage subsystem (121) to the host computer (101) via the high-speed I / O bus (130). As the processing speed of computers increases, the movement of data between subsystems is becoming a bottleneck for the entire process. This bottleneck can be eliminated by sending only the necessary columns.

FIG. 3 shows an example of a command (SSD extension command) transmitted from the driver software (103) to the command / data transmission / reception unit (122). This command is preferably based on industry standard specifications such as NVME (Non-Volatile Memory Express). The standard command only contains block information to be read and written, it does not contain information about database columns, but it is expanded to include information (metadata) about the columns to be read. Thus, conversion from the row format to the column format on the storage subsystem (121) side becomes possible.

The SSD control unit (123) that has received the extended command reads data from the SSD storage block (125) designated by the read source, and passes it to the DB processing offload execution unit (124). Next, the DB processing offload execution unit (124) expands each column included in the read block based on the column number to be requested included in the expanded SSD command and the specification of the table definition information (column The information is stored as metadata in the SSD storage block (125) instead of the command body, and pointer information (storage block number or the like) to the metadata may be included in the expanded SSD command). Note that this processing is independent for each block and can be easily parallelized, so that the benefits of parallel hardware can be easily received. Further, the DB processing offload execution unit (124) converts the data into a column format and places the data in the DMA buffer (105) via the command / data transmission / reception unit (122).

FIG. 4 illustrates the above example. FIG. 4 is equivalent to FIG. 1 although illustration of some elements is omitted for simplification. SQL_SCAN defined as an extended SSD command is transmitted from the host computer (101) to the storage subsystem (121). This extended SSD command is the same as the command for a normal SSD. In addition to the block number to be read, the number of blocks, the address and size of the DMA buffer (105) to which the result is written back, the column information to be read (in this example, column information called col_x and col_z). The SSD control unit (123) reads the blocks specified in the extended SSD command (in this example, block 5, block 6, and block 7) from the SSD storage block (121), and the DB processing offload execution unit (124) To pass. The DB processing offload execution unit (124) (implemented by hardware in this example) extracts only the data related to col_x and col_z from the read block based on the column information specified in the extended SSD command, Convert to format data (array). This column format data is written back to the DMA buffer (105) of the host computer (101).

FIG. 5 shows a second embodiment of the present invention. The block read according to the specification of the extended SSD command is converted into the column format by the DB processing offload execution unit (124), which is the same as in the first embodiment. In the second embodiment, not the main memory (104) of the host computer (101) but the GPU internal memory (502) which is a component of the GPU (Graphic Processing Unit) (501) provided in the host computer (101). Transfer data directly to the DMA buffer (105) in the above GPU (in this example, the host computer (101) and storage subsystem (121) are connected via an I / O bus, not a storage network) ). In general, the GPU (501) can be used not only as a graphic-related process but also as a general-purpose parallel processing processor. By arranging data in a regular sequence in advance, there is no extra processing. Data parallel processing can be executed more efficiently. For example, data subjected to parallel processing such as sorting by the GPU (501) is transferred to the DMA buffer (105) on the main memory (104), and the database access processing by the RDBMS (102) is completed.

The third embodiment of the present invention will be described below. This embodiment is characterized in that the DB processing offload execution unit (124) performs index processing in addition to the conversion from the row format to the column format in the first embodiment and the second embodiment. . The DB processing offload execution unit (124) determines blocks that do not match the search conditions based on the index information, and does not send those blocks to the host computer (101), thereby reducing the amount of transferred data and processing efficiency. To improve.

This example shows a processing efficiency improvement method using an index called BRIN (Block-Range INdex), which is a function of PostgreSQL, a popular open source RDBMS. BRIN is a type of index that bundles a plurality of data blocks and holds the maximum value / minimum value as a summary. Fig. 6 shows an example of BRIN. For example, if it is determined by BRIN set in a certain column (for example, Col_X) that the minimum value of the column is 100 and the maximum value is 654 in all records included in block 0 to block 2, col_X ≧ It can be seen that there is no need to read block 2 from block 0 and send it to the host computer (101) for a query with 1000 as a search condition.

FIG. 7 shows an example of an extended SSD command used in the third embodiment of the present invention. In the extended SSD command used in the present embodiment, the block number including the index, the number of blocks, the number of the column in which the index (search key) is set, and the minimum value and maximum value of the value are included. included. These values can be set by the driver software (103) based on information obtained from the RDBMS (102). First, based on the contents of the extended SSD command, the SSD control unit (123) reads data corresponding to the index from the block including the index itself and passes it to the DB processing offload execution unit (124). Next, the DB processing offload execution unit (124) specifies whether or not the block specified as the reading target is included in the block range that matches the search condition. If it is specified that the minimum value is 600 as a search key for the index of FIG. 6, it is understood that it is not necessary to read block 2 and block 5. Next, the DB processing offload execution unit (124) instructs the SSD control unit (123) to read blocks other than block 2 and block 5. In the subsequent processing, as in the first embodiment, it is desirable to read only the blocks corresponding to the necessary columns, convert them to the column format, and return them to the host computer (101). Further, as shown in the second embodiment, instead of the main memory of the host computer (101), the data composed of the read blocks may be directly written to the DMA buffer on the GPU main memory via the DMA. In addition, since the index processing is also a routine processing, it is desirable to increase the hardware speed by means such as FPGA.

(Technologically significant effect of the present invention)
The technically significant effects provided by the present invention will be summarized again. First, in the past, the data format used by the database system was selectively used according to the workload characteristics of write center / read center, but according to the present invention, unified management of data, that is, maintenance of duplicate data becomes unnecessary. . Second, the present invention reduces the amount of data transferred between the host computer and the storage subsystem, and makes it possible to efficiently use the bandwidth of the I / O bus and storage network, which are valuable resources. Third, write-centric transaction processing uses row-format data, while read-intensive workloads such as data warehouses use column-format data to provide modern parallelism such as GPUs and vector processors. It becomes possible to easily draw out the capacity of the processor.

Claims

A storage system that includes a DB offload processing execution unit and a storage device, and performs a read process of a relational database,
Storage in which the DB offload processing execution unit converts the block read from the storage into a format consisting only of the data of the relational database based on the received command including information on the relational database column and transmits the storage system.
A storage system that includes a DB offload processing execution unit and a storage device, and performs a read process of a relational database,
The storage system in which the DB offload processing execution unit reads and transmits only the blocks in the range specified by the index based on the received command including information on the index of the relational database.
The storage system according to claim 1 or 2, wherein the transmission destination is a main memory of a graphic processing unit.
A device driver program that is executed on a host computer connected to a storage system that includes a DB offload processing execution unit and a storage device, and that reads a relational database;
An instruction for transmitting a command including information on a column to be read in the relational database to the DB offload processing execution unit;
A device driver program including an instruction for receiving data converted into a column format based on information on the column to be read from a DB offload processing execution unit.
A device driver program that is executed on a host computer connected to a storage system that includes a DB offload processing execution unit and a storage device, and that reads a relational database;
A command including information related to the index of the relational database, and a command for transmitting the command to the DB offload processing execution unit;
A device driver program including an instruction for receiving, from the DB offload processing execution unit, only blocks in a range specified by the index from the DB offload processing execution unit.
Further comprising an instruction to directly write the received data or block to the main memory of the graphics processing unit;
The storage system according to claim 4 or 5.
A computer-executed method for acquiring data from a storage system that includes a DB offload processing execution unit and a storage device and that performs a read process of a relational database,
Sending a command including information about a column to be read of the relational database;
Converting data read from the storage device into column format data based on information about the column to be read;
Transmitting the column format data.
A method of acquiring data from a storage system that includes a DB offload processing execution unit and a storage device and performs a read process of a relational database,
Sending a command including information about an index of the relational database;
Reading only blocks in the range specified by the index;
Transmitting the read block.
Writing the transmitted data or block directly into the main memory of the graphics processing unit;
9. A method according to claim 7 or claim 8.