JP6316503B2 - Computer system, accelerator, and database processing method - Google Patents

Description

  The present invention relates to a data processing method, a computer system, and a storage system using a storage device having a data storage unit and a database aggregation processing unit.

  In data processing systems such as database search processing, a part of the processing performed on the data processing server is offloaded to a hardware accelerator located near the mass storage medium (storage) for the purpose of speeding up the data processing. The structure which performs is proposed (for example, patent document 1).

  By offloading the filtering processing, projection processing, grouping processing, or aggregation calculation processing that was performed by the conventional data processing server to the hardware accelerator, the load on the data processing server is reduced and processing such as search time is performed. A technique for significantly reducing the time is known (Non-Patent Document 1).

  Furthermore, in the data processing server, it has been studied to offload join processing and merge sort processing to a hardware accelerator (Patent Document 2). As a technique for speeding up database processing, it is known to perform pipeline processing at the function level and parallelization of functions (Patent Documents 1 and 2).

Japanese Patent Laid-Open No. 5-128164 US Patent Application Publication No. 2012/0047126

Louis Woods, Zsolt Isvan, Gustavo Alonso: “Ibex-An Intelligent Storage Engine with support for advanced SQL Off-loading”, Proc. VLDB Endowment (PVLDB), 2014

  However, in Non-Patent Document 1, the database processing unit is “row”, and it is difficult to improve the database processing performance because the processing unit is small. Furthermore, in Non-Patent Document 1, data is directly input to a hardware accelerator from an SSD (Solid State Drive) that stores a database, so the database processing performance is subject to processing restrictions due to the SSD reading performance. Therefore, there has been a problem that the speed of data supply to the hardware accelerator becomes a bottleneck.

  On the other hand, in the above grouping process, an example using a hash operation is known. When grouping is performed using a hash operation, different groups may have the same hash value, and hash value collision (synonym) may occur.

  When a synonym occurs, if the hardware accelerator recalculates the hash value, the calculation of the next line (data) is awaited, and the processing performance of the hardware accelerator is reduced.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide hardware that improves the performance of database aggregation operations in cooperation with a server.

  The present invention is a computer system having a server including a processor, a memory, an accelerator connected to the server for database processing, and a storage device connected to the accelerator for storing a database. The server receives a query, generates a database command, determines a database range to be processed and a unit size to be processed by one database command by dividing the database range, and instructs the accelerator A command processing unit, and a re-aggregation unit that aggregates the output of the accelerator and generates a processing result for the query, and the accelerator has the unit size based on a command from the server command processing unit. Database processing from the storage device The processing target data is divided into predetermined processing units, database processing including grouping processing, stacking processing, and aggregation processing is executed for each predetermined processing unit, and an aggregation result is output. Having a database processing unit, the re-aggregation unit, when receiving an aggregation result for the range of the database to be processed from the accelerator, aggregates the aggregation result and generates it as a processing result for the query.

  According to the present invention, the server and the accelerator execute database processing in cooperation, thereby reducing the load on the server and improving the database processing performance of the accelerator. In addition, the accelerator executes the aggregation process in units of database commands, and the server can improve the processing performance of the database processing system by performing the aggregation process of the aggregation results for a plurality of database commands.

It is a block diagram which shows a 1st Example of this invention and shows an example of a structure of a database processing system. 1 is a block diagram illustrating an example of a configuration of an FPGA according to a first embodiment of this invention. FIG. It is a figure which shows the 1st Example of this invention and shows an example of the query with respect to a database, and a processing content. It is a figure which shows a 1st Example of this invention and shows an example of the database command with respect to a database. It is a figure which shows the 1st Example of this invention and shows an example of command issuing of DB server, and command processing of FPGA. It is a figure which shows 1st Example of this invention and shows an example of the page format of a database. It is a block diagram which shows the 1st Example of this invention and shows an example of a structure of the database process part of FPGA. It is a timing chart which shows a 1st Example of this invention and shows an example of the pipeline process performed by FPGA. It is a figure which shows the 1st Example of this invention and shows an example of the output data amount in each process of FPGA. It is a figure which shows a 1st Example of this invention and shows an example of a process of FPGA and a process of a server. It is a figure which shows the 1st Example of this invention and shows an example of the method of grouping of a grouping row | line | column. It is a flowchart which shows a 1st Example of this invention and shows an example of a grouping process. It is a figure which shows the 1st Example of this invention and shows an example of the storage method of fixed point data by a stacking calculation. FIG. 3 is a block diagram illustrating an example of a stacking calculation register according to the first embodiment of this invention. It is a flowchart which shows a 1st Example of this invention and shows an example of a stacking process. It is a figure which shows the 1st Example of this invention and shows an example of a stacking calculation. It is a figure which shows a 1st Example of this invention and shows an example of the command used by a stacking calculation. It is a figure which shows the 1st Example of this invention and shows an example of an aggregation result. It is a timing chart which shows a 1st Example of this invention and shows an example of an aggregation process. It is a figure which shows the 1st Example of this invention and shows an example of the format of a synonym. It is a flowchart which shows a 1st Example of this invention and shows an example of the process performed by a database server. It is a figure which shows the 1st Example of this invention and shows an example of the re-aggregation process by DB server. It is a figure which shows 1st Example of this invention and shows an example of a hash table. It is a figure which shows 1st Example of this invention and shows an example of a grouping row | line | column table. It is a figure which shows 1st Example of this invention and shows an example of a group hash table. It is a figure which shows the 2nd Example of this invention and shows the example which reduces the data size at the time of synonym frequent occurrence.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

  1 shows a storage device 3 for storing a field programmable grid array (FPGA) 2 having a database processing function according to the present invention and a database (hereinafter referred to as DB) 30, and a database management system (hereinafter referred to as a DBMS) 20. 1 is a block diagram showing an example of a database system including a database server (hereinafter, referred to as a DB server) 1 on which is operated.

  The DB server 1 includes a host CPU 11 that executes arithmetic operations, a host memory 12 that stores programs and data, and a host interface 13 that is connected to the FPGA 2 as a hardware accelerator and the storage device 3 via the PCI switch 4. Including. The host interface 13 is an interface conforming to PCI or PCI express.

  In the DB server 1, the DBMS 20 loaded in the host memory 12 is executed by the host CPU 11 and issues a database command to the FPGA 2 in response to an access request (query) from a client computer (not shown).

  The FPGA 2 mounted on the storage device 3 includes a CPU 126 in addition to the SRAM 200, and includes a database processing unit 250 that executes database processing (filter processing, grouping processing, projection processing, aggregation processing, etc.) based on database commands. Functions as an accelerator. Note that the FPGA 2 and the storage device 3 may be configured independently, and each may be connected to the PCI switch 4.

  The storage device 3 includes an SSD 137 and a DRAM 136 as storage media for storing the DB 30, and a control unit 129 that controls these storage media includes an SSD interface 130 and a DRAM interface 131.

  The storage device 3 and the FPGA 2 are connected by an SSD interface 130 and a DRAM interface 131.

  In the storage apparatus 3 according to the first embodiment, the DB 30 is stored in the SSD 137 of the nonvolatile storage medium, and the control unit 129 reads the range of the DB 30 to be processed from the SSD 137 into the DRAM 136.

  The FPGA 2 reads data from the DRAM 136, which has a higher reading speed than the SSD 137, by a predetermined processing size (for example, several megabytes), so that the processing from the DRAM 136 to the DB 30 is not limited by the reading speed of the SSD 137. The target data can be read at high speed.

  However, in the present invention, the size of the processing target portion of the DB 30 is larger than the unit in which the FPGA 2 performs arithmetic processing. That is, after a portion to be processed in the DB 30 is once read into the DRAM 136, data is input from the DRAM 136 to the FPGA 2 by a predetermined size.

<DBMS>
Next, the DBMS 20 will be described. The DBMS 20 operating on the DB server 1 includes a result storage area 115, a message storage area 116, a synonym storage area 117, a re-aggregation module 118, a grouping and aggregation module 120, a command generation unit 103, and a command storage unit. 123, a request command queue 121, and a completion command queue 122. The grouping and aggregation module 120 includes a grouping and aggregation module for re-grouping the DB 30 in which a synonym is generated in the FPGA 2 aggregation processing.

  The DBMS 20 divides the DB 30 of several TB or several GB into several MBs, and instructs the FPGA 2 to perform aggregation processing of the DB 30. For this reason, the DB server 1 generates a database command from the range of data targeted for aggregation processing in the DB 30 and the contents of the aggregation processing.

  The DBMS 20 generates a database command from the access request received from the client computer by the command generation unit 103 and stores it in the command storage unit 123. Further, the command generation unit 103 sets a command pointer indicating the storage position of the database command stored in the command storage unit 123 in the request command queue 121. The DBMS 20 sequentially inputs database commands corresponding to the command pointers input to the request command queue 121 to the FPGA 2. These command generation unit 103, command storage unit 123, and request command queue 121 constitute a server command processing module 1030.

  The DBMS 20 accepts the database command aggregation result received from the FPGA 2 and stores it in the result storage area 115. The DBMS 20 accepts, as a message, the number of groups aggregated by the database command executed by the FPGA 2, the number of synonyms generated, information on operation overflow, and the like, and stores them in the message storage area 116. When the DBMS 20 receives the information (synonym 114) in which the synonym is generated by executing the database command of the FPGA 2, the DBMS 20 stores it in the synonym storage area 117. Further, the DBMS 20 stores the database command that has received the notification of execution completion from the FPGA 2 in the completion command queue 122.

<FPGA>
FIG. 2 is a block diagram illustrating an example of the configuration of the FPGA 2. The FPGA 2 includes a CPU 126 that controls database processing based on a database command, and hardware functional blocks that perform database processing.

  The functional blocks constituting the database processing unit 250 of the FPGA 2 include a data reading unit (hereinafter, DATA I / F) 105 that reads processing target data from the DRAM 136 in the DB 30, and a filter processing unit that performs filtering processing on the processing target data ( Hereinafter, Filter) 106-0 and 106-1, filter processing units (hereinafter referred to as Filter) 106-0 and 106-1 that perform filter processing on the processing target data, and projection processing that performs projection processing on the processing target data. Sections (hereinafter referred to as “Projections”) 107-0 and 107-1, grouping processing sections (hereinafter referred to as “Grouping”) 108-0 and 108-1 that perform data grouping (grouping) on the results of the filter processing and the projection processing, and filters Processing and projection Stack processing unit about the results performs calculation (hereinafter, Stacking) includes a 109-0,109-1, aggregation processing unit that aggregates the results of grouping and operation (hereinafter, Aggregation) 111, and.

  The CPU 126 of the FPGA 2 acquires a command pointer from the command queue 124, activates DMA (not shown), and transfers the database command from the command storage unit 123 of the DB server 1 to the command register 104.

  Next, the CPU 126 of the FPGA 2 transfers the data of the processing target portion of the DB 30 designated by the database command from the SSD 137 to the DRAM 136. When page data, which is a processing unit of DB 30, is stored in the DRAM 136, the CPU 126 writes an aggregation processing start signal in the register 104.

  Next, the CPU 126 starts data transfer of the DRAM 136 from the DRAM interface 131 to the DATA interface 105. The DRAM interface 131 stores the page data of the DB 30 in the SRAMs D0 to D3 (201 to 204) divided into four blocks of the data interface 105, respectively. Note that the SRAMs D0 to D3 (201 to 204) are assigned with predetermined areas of the SRAM 200 of the FPGA 2 shown in FIG. It should be noted that other SRAMs P0, P1, and G0 described below are assigned predetermined areas of the SRAM 200.

  The page data of the DB 30 stored in the DATA interface 105 is input on a page-by-page basis to pipeline processing that performs filtering processing and projection processing. This page-by-page pipeline processing includes the pipelines of Filter # 0 (106-0) and Projection # 0 (107-0), Filter # 1 (106-1), and Projection # 1 (107-1). An example of a two-stage pipeline is shown. The generic name of Filter # 0 (106-0) and Filter # 1 (106-1) is shown as Filter 106. The same applies to other components.

  The result of page-by-page pipeline processing is input to parallel processing of grouping processing and stacking processing. In this parallel processing, row data output by Projection # 0 (107-0) is processed in parallel by Grouping # 0 (108-0) and Stacking # 0 (109-0), and Projection # 1 (107-1) The row data to be output are processed in parallel by Grouping # 1 (108-1) and Stacking # 1 (109-1).

  The Arbiter 110 sequentially receives the results of Grouping # 0 (108-0) and Stacking # 0 (109-0), and the results of Grouping # 1 (108-1) and Stacking # 1 (109-1), Input to Aggregation (111).

  The Aggregation 111 executes the aggregation process set by the database command with the result of the parallel processing of the grouping process and the stacking process as an input, and executes the aggregation process 112, the message 113, the synonym 114, and the completion command (1 command process completion notification). 138 is output. The output of the aggregation result 112, the message 113, the synonym 114, and the completion command 138 is transmitted to the DB server 1 via the PCI switch 4.

  The DBMS 20 of the DB server 1 performs re-aggregation by the re-aggregation module 118 using the completion command 138 stored in the completion command queue 122, the aggregation result of the result storage area 115, and the synonym storage area 117, as will be described later. Then, grouping and aggregation processing by the grouping and aggregation module 120 is performed on the data in which the synonym is generated. The DB server 1 processes all the DBs 30 by repeating this processing a plurality of times (database size / data size processed by the FPGA 2 with one command = page unit).

  The grouping / aggregation module 120 executes processing when synonym information is written in the synonym storage area 117, and outputs the grouped and aggregated data to the re-aggregation module 118.

  The re-aggregation module 118 aggregates the aggregation results for each grouping column based on the aggregation result in the result storage area 115 and the output of the grouping and aggregation module 120, and responds to the query result. As will be described later, the grouping column is data that is a hash value calculation target by combining a plurality of columns of processing target data in the row direction.

  FIG. 3 is a diagram illustrating an example of a query and processing content for the DB 30. In the example of FIG. 3, the benchmark TPC-H Query 3 of the DB 30 is shown as an example of the query received by the DB server 1.

  A select statement appearing in the query of FIG. 3 is the projection process 301, which describes a data string to be extracted by the query. In the FPGA 2 of FIG. 2, Projection (projection processing unit) # 0 (107-0) and Projection # 1 (107-1) perform the Projection process 301.

  The WHERE statement that appears in the query of FIG. 3 is the filtering process 302, and a line that matches the condition of the WHERE statement is extracted. In the FPGA 2 of FIG. 2, Filter (filter processing unit) # 0 (106-0) and Filter # 1 (106-1) perform the filtering process 302.

  A Group by statement appearing in the query shown in FIG. 3 is a grouping process 303, and grouping is performed by a column designated by the grouping. In the FPGA 2 of FIG. 2, the Grouping (grouping processing unit) # 0 (108-0) and the Grouping # 1 (108-1) perform the Grouping process 303.

  The Sum statement that appears in the query of FIG. 3 is the aggregation process 305, and performs an aggregation operation. In the FPGA 2 of FIG. 2, the aggregation calculation processing 305 performed by the aggregation 111 includes a total, a maximum value, a minimum value, a count, and the like.

  The inside of () of the Sum that appears in the query of FIG. In the FPGA 2 of FIG. 2, Stacking (stacking processing unit) # 0 (109-0) and Stacking # 1 (109-1) perform the Stacking process 306. Numerical operations performed by Stacking # 0 (109-0) and Stacking # 1 (109-1) include addition, subtraction, and multiplication. Note that Order by for rearranging rows is the ordering process 304, and this process is executed by the DB server 1.

  FIG. 4 is a diagram illustrating an example of a database command for the FPGA 2. The information of the storage destination SSD 137 of the DB 30 includes a read start address 401 and a read data size 402. Although not shown, the size (unit size: 8 MB) of the DB 30 processed by the FPGA 2 may be specified with one command.

  The setting information of the filtering 106 includes a filter target column 403 and a filter condition 404 in the DB 30. The setting information of the Projection 107 includes an extraction column 405. The grouping column 406 is included as the setting information of the Grouping 108.

  The setting information of Stacking 109 includes a calculation target column 407, an operator 408, and a direct value 409. The setting information of the aggregation (aggregation processing unit) 111 includes an operator 410.

  FIG. 5 is a time chart showing the relationship between the processing of the server command processing module 1030 and the aggregation processing (DATA I / F (105) to Aggregation 111) of FPGA2.

  The server command processing module 1030 generates a plurality (or one or more) of database commands (501 to 504) to be processed by the FPGA 2 from the query received by the DB server 1, and stores the database commands in the command storage unit 123. When the storage of the database command is completed, the server command processing module 1030 writes the pointer of the area storing the command in the request command queue 121 and stacks the database command on the CPU 126 of the FPGA 2 via the doorbell register (not shown). Is notified (time 505 to 508 in the figure).

  The CPU 126 activates a DMA (not shown) and transfers the database command in the command storage unit 123 indicated by the pointer of the request command queue 121 from the DB server 1 to the register 104.

  In the FPGA 2, in parallel with the acquisition of the database command, the CPU 126 transfers the processing target data in the DB 30 from the SSD 137 to the DRAM 136. When the page data of the DB30 processing unit is stored in the DRAM 136, the CPU 126 writes an aggregation processing start signal to the register 104, starts data transfer from the DRAM 136 to the DATA I / F 105, and stores the data of the DB30 in the four SRAMs 0 to 3. Data (page data) is stored in units of pages.

  A plurality of page data written in the DATA I / F 105 is sequentially input to the filter 106, and operations are performed in the processing units of the projection 107, the grouping 108, the stacking 109, and the aggregation 111.

  When each operation of the FPGA 2 is completed, the CPU 126 issues a completion command 138 to the DB server 1 to notify the completion of one command processing (512, 513, and 514 in the figure). The DB server 1 that has received the one-command processing completion notification registers the database command indicated by the completion notification in the completion command queue 122.

  The re-aggregation module of the DB server 1 receives the completion notification (512, 513, 514) for each command, generates the group hash table 119 using the grouping column of the result storage area 115, and performs grouping processing and re-aggregation. Processing is performed as described below.

  FIG. 6 shows an example of the page format of the DB 30. The DB 30 is configured in units of pages, and a page header 601 is stored at the top of the page data. Following the page header 601, row data 602 is stored in order from the first row to the Mth row, and a row pointer 603 indicating the head address of the row is stored in order from the end of the page data.

  The Filter 106 and the Projection 107 of the FPGA 2 acquire necessary column data using the row pointer 603 and the database command setting information (403, 405) shown in FIG.

  FIG. 7 is a block diagram illustrating a detailed example of the circuit configuration of the database processing unit 250 of the FPGA 2. The page data of the DB 30 read from the DRAM 136 is stored in units of pages in the SRAMs 201 to 204 (SRAMs D0 to D3) via the DATA I / F 105.

  The SRAM 201 is composed of a set of SRAMs, holds the same data in two SRAMs (for example, D1_0, D1-1), and supplies the same data to the filter 106 and the projection 107 that perform parallel processing. The same applies to the other SRAMs 202 to 204.

  Data supplied from the SRAM 201 to 204 to the filter 106 and the projection 107 is selected by selectors 707, 708, 709, and 710. The processing result of the filter 106 is input to the project 107. Here, the filter 106 executes processing based on the filter information in the filter target column 403 and the filter condition 404 in the database command (FIG. 4) set in the register 104. Similarly, the projection 107 executes processing based on the projection information set in the register 104.

  The results processed by the Projection 107 are stored in the SRAM P0 (205) and the SRAM P1 (206) in units of one page in the DB 30. Using the projection results of SRAM P0 (205) and SRAM P1 (206), Grouping 108-0 and 108-1 and Stacking 109-0 and 109-1 execute parallel processing, respectively.

  In the Grouping 108-0 and 108-1, the calculation is performed using the grouping column 406 of the database command, the hash tables 728_0 and 728_1, and the grouping column tables 729_0 and 729_1. In the hash table 728, hash values of grouping target data are stored. The grouping column table 729 stores grouped column data. These hash table 728 and grouping column table 729 are set in predetermined areas in the SRAM 200 of the FPGA 2.

  In Stacking 109-0 and 109-1, the calculation is performed based on the calculation target column 407, the operator 408, and the direct value 409 of the database command.

  The outputs of these Grouping 108 and Stacking 109 are sequentially received by the Arbiter 110 and written to the SRAM G0 (207). The contents of the SRAM G0 (207) are rewritten by the result of the aggregation calculation of the aggregation 111.

  When the data processing for one command is completed, the aggregation 111 outputs the aggregation result 112, the message 113, and the synonym 114 to the DB server 1. The message 113 stores the number of aggregated groups, the number of occurrences of synonyms, and operation overflow information.

  FIG. 8 shows a timing chart of pipeline processing performed in the FPGA 2. At time T0 (801), the DATA I / F 105 writes the data of page 1 (P-1 in the figure) to the SRAM D0.

  At time T1 (802), the DATA I / F 105 writes the data of page 2 (P-2) to the SRAM D1, and the filters 106-0 and 106-1 perform the filtering process using the data of the SRAM D0.

  At time T2 (803), the DATA I / F 105 writes the data of page 3 (P-3) to the SRAM D2, and the filters 106-0 and 106-1 perform the filtering process using the data (P-2) of the SRAM D1. The Projections 107-0 and 107-1 perform the Projection processing using the data of the SRAM D0 (P-1). The Filtering process and the Projection process are executed for each line of page data.

  Further, at time T21 when data in the first row of the projection processing is written into the SRAM P0, Grouping 108-0 and 108-1 and Stacking 109-0 and 109-1 are started. Thereby, in parallel with the projecting process, the grouping process and the stacking process are executed in parallel based on the result of the projection process written in the SRAM P0. Other Grouping processing and Stacking processing also wait until one row of data is written in the SRAMs P0 and P1.

  When data is written to the SRAMs P0 and P1, the aggregation 111 performs an aggregation process and writes the result of the aggregation process to the SRAM G0.

  At time T3 (804), the DATA I / F 105 writes the data of page 4 (P-4) to the SRAM D3, and the Filter (106-0, 106-1) uses the data of the SRAM D2 (P-3). Filtering processing is performed. In addition, the Projections 107-0 and 107-1 perform the Projection process using the data (P-2) of the SRAM D1.

  The Grouping 108-0 and 108-1, and the Stacking 109-0 and 109-1 execute a grouping process and a stacking process in parallel with the Projection process. Then, the aggregation 111 performs an aggregation process based on the result of the grouping process and the result of the stacking process, and writes the aggregation result to the SRAM G0. At the timing described above, the pipeline processing is executed in the FPGA 2.

  FIG. 9 is a diagram illustrating a change in output data amount in each process in units of commands. The processing target data of the DB 30 processed by a single database command is indicated by reference numeral 901 in the figure.

  When the filtering processing is performed on the processing target data 901 of the DB 30, row data that matches the filter condition is extracted, and two row data 902 in the figure are output as processing results.

  The data amount of the row data 902 remaining in the process of this filtering process is about 1/10 of the data amount of the processing target data 901. Projection processing is performed using the row data 902 as input, and data in the Grouping column 903 necessary for the Grouping processing and the Stacking column 904 necessary for the Stacking processing are extracted.

  The data amount of the column data (903, 904) calculated in the process of the projection processing is about 1/10 of the data amount of the row data 902 after the filtering processing. Data 905 is obtained by performing the Grouping process and the Stacking process using the Grouping column 903 and the data of the Stacking 904. The data amount of the data 905 remaining in this process is about 1/10 of the data amount of the column data (903, 904) after the projection processing. Data 905 obtained by the Grouping process and the Stacking process is subjected to an aggregation process, and a final aggregation result 906 is obtained.

  In the FPGA 2 of this embodiment, the data amount of the aggregation result 906 to be output is about 1/1000 of the processing target data 901 to be input.

  FIG. 10 is a diagram illustrating a relationship between the aggregate calculation local process performed by the FPGA 2 and the aggregate calculation global process performed by the DB server 1.

  The DB server 1 uses the server command processing module 103 to determine that the size of the processing target data to be processed by one database command out of the data 1000 of the DB 30 of 2 TB size stored in the SSD 137 is 8 MB. Then, the DB server 1 designates in the database command that the FPGA 2 processes the processing target data 901 of 8 MB with one database command (1004).

  The FPGA 2 accepts one database command and processes the processing target data 901 of the DB 30 divided into 8 MB. The FPGA 2 processes the 8 MB processing target data 901 in units of 8 KB pages (1005).

  The FPGA 2 reads the processing target data 901 from the DRAM 136 in units of pages (8 KB) for the processing target data 901 for 8 MB with one database command, and executes grouping processing and stacking processing. Then, the FPGA 2 performs aggregation processing based on the results of looping processing and stacking processing (1006).

  When the aggregation process is completed, the FPGA 2 returns the grouping column and the aggregation result to the DB server 1 in a state where the order of the group of the aggregation result is not the same as shown in FIG. 21 (1008).

  Upon receiving the grouping column and the aggregation result, the DB server 1 generates a group hash table 119 based on the grouping column by the re-aggregation module 118, arranges the data in the order of hash values, and sets the global (all all database commands). (Processing target data) is aggregated.

  FIG. 11 is a diagram illustrating an example of a grouping method for grouping columns. Grouping processing is performed in the FPGA 108 Grouping 108-0 and 108-1. In this grouping process, columns to be grouped included in one row of data are linked as grouped columns and stored in the grouping column table 729.

  In the illustrated example, grouping data 1102 is generated by concatenating grouping columns 1, 2, 3 (1101) of row data of one row, and stored in the grouping column table 729. Each row of the grouping data 1102 is data of a grouping column that is a hash value calculation target.

  Then, the Grouping 108 calculates a hash value and a group ID for each grouping column of the grouping data 1102 to identify each group. Note that the hash value and group ID calculated in the grouping 108 are stored in the hash table 728.

  FIG. 22 is a diagram illustrating an example of the hash table 728 in the FPGA 2. The hash table 728 shows an example in which the address 2301 of the SRAM 200 is a hash value, 1-bit data 2302 is a flag bit, and 10-bit data 2303 is a group ID corresponding to the hash value.

  The hash table 728 stores the result of the grouping process, and “1” (0b1) is set in the flag bit (2302) of the address 2301 to which the group ID (2303) is assigned. The group ID (2303) is added every time a new group appears.

  FIG. 23 is a diagram illustrating an example of the grouping column table 729 in the FPGA 2. The grouping column table 729 shows an example in which an address 2401 (10 bits) of the SRAM 200 is used as a group ID and 64-bit data 2402 is used as a grouping column. The grouping column table 729 holds a grouping column (2402) corresponding to the group ID (2401).

  In the first embodiment, an example capable of supporting up to 256 Kbyte grouping columns is shown, and for example, 32 8-byte wide SRAMs are mounted in the FPGA 2. The correspondence between the hash value and the group ID (2401) is defined by the hash table 728 described above. In the first embodiment, the maximum value of the group ID is 1024, and this maximum value is the number that can be assigned a hash value.

  The calculation of the hash value and the group ID is not limited to the above, and a publicly known or known method may be applied, and the hash value of the processing target data 901 and the group ID to which the data belongs are determined. Just do it.

  FIG. 12 is a flowchart illustrating an example of the grouping process. In this process, the Grouping 108 of the FPGA 2 is activated when the operation result of the row is written in the SRAMs P0 and P1.

  The Grouping 108 calculates (1201) the hash value of the grouping column (2402). The Grouping 108 determines whether or not the calculated hash value is registered in the hash table 728 (1202).

  If the calculated hash value is not registered in the hash table 728, the grouping 108 proceeds to step 1205. In step 1205, the Grouping 108 registers the calculated hash value in the hash table 728. In this case, since no synonym has occurred, the process proceeds to step 1206 and is written from the Arbiter 110 to the SRAM G0 (207).

  On the other hand, if the calculated hash value is already registered in the hash table 728, the process proceeds to step 1203. In step 1203, the Grouping 108 acquires data 2402 (grouping column) of the grouping column table 729 corresponding to the address 2301 having the same hash value in the hash table 728. If the original grouping sequence of the calculated hash value is different from the acquired grouping sequence, the grouping 108 proceeds to step 1204 and determines a synonym (hash value collision).

  In the case of a synonym, the aggregation processing in the aggregation 111 is not performed, and the synonym information is transmitted to the DB server 1 as a synonym 114 as described later. As the synonym information, a hash value or a grouping sequence can be used.

  On the other hand, in the case of a non-synonym, aggregation processing is performed in the aggregation 111 based on the result of the Grouping 108 written in the SRAM G0 (207) and the result of the Stacking 109 described later.

  FIG. 13 is a diagram illustrating an example of a fixed-point data storage method in the stacking calculation. The storage format in the DB 30 is an integer so that the Stacking 109 can easily calculate a fixed point.

  In FIG. 13, there are two columns for stacking calculation, and data 1031 is stored in the first column of the Nth row by multiplying 0.08 by 100. In the second column of the Nth row, data 1302 in which 0.5 is multiplied by 10 to be 5 is stored.

  The Stacking 109 performs an operation as an integer without regard to the fixed decimal point, and the DB server 1 restores the digit of the value of the aggregation result stored in the result storage area 115 to obtain the final aggregation result. Note that the position of the fixed point is preset by the DBMS 20.

  FIG. 14 is a diagram illustrating a configuration of the stacking calculation register and the SRAM. This configuration shows the hardware configuration of Stacking 109-0 and 109-1.

  As will be described later, the Stacking 109 constituting the stacking processing unit accumulates values and operators, and executes an operation on the two most recent values when the operators appear. Therefore, the configuration of the circuit that holds the values is assumed to be two registers REG0 (1401) and REG1 (1402) and one SRAM (1403). The SRAM (1403) indicates a predetermined area in the SRAM 200 shown in FIG.

  FIG. 15 is a flowchart illustrating an example of the stacking calculation executed in the Stacking 109 of the FPGA 2.

  The Stacking 109-0 and 109-1 start processing when the Projection 107 writes the operation result in the SRAMs P0 and P1 (205 and 206) (1501).

  When the stacking operation is started (1501), the Stacking 109 receives one stack operation command from the database command from the register 104 (1502), and determines whether the content of the command is a numerical value or an operator (1503).

  If it is determined in step 1503 that the content of the command is the output of the projection 107 or a direct value, the process proceeds to step 1504 for determining the storage location of the numerical value. In step 1504, the Stacking 109 writes the data of REG0 (1401) to the SRAM 1403 and writes the value of REG1 (1402) to REG0 (1401) if the data has already been stored in the stack registers 1401 and 1402 of FIG. The input data of step 1503 is written into REG1 (1402).

  If it is determined in step 1504 that the data has not been stored in the stack registers 1401 and 1402, the process proceeds to step 1509, and Stacking 109 writes the data of REG1 (1402) to REG0 (1401), and the input data of step 1503 is REG1. Write to (1402).

  If it is determined in step 1503 that the content of the stack operation command is an operator, the Stacking 109 operates the outputs of REG0 (1401) and REG1 (1402) using the stack operation command, and writes the operation result back to REG1 (1402). .

  Furthermore, if data exists in the SRAM (1403), the Stacking 109 writes data from the SRAM (1403) to the REG0 (1401). When the data writing is completed, the Stacking 109 refers to the register 104 and determines whether or not there is a next stack operation command (1505).

  In step 1505, if the content of the stack operation command is not an end command, the stacking 109 returns to step 1502 and repeats the above processing. On the other hand, if the next stack operation command is an end command in step 1505, the Stacking 109 ends the stacking operation (1506).

  When the result of the projection processing is written to the SRAMs 205 and 206 by the above processing, the stacking processing is executed by the Stacking 109 and written to the SRAM G0 (207).

  FIG. 16A is a diagram illustrating an example of a stacking calculation. FIG. 16B is a diagram illustrating an example of commands used in the stacking calculation. FIG. 16A shows the states of the stack operation commands 1602 and 1605 to 1608 and the registers 1401 and 1402 input to the stacking processing unit 109 with the horizontal axis as time. The contents of the stack operation commands 1605 to 1608 correspond to the code 1612 in FIG. 16B. The stack operation command 1601 of FIG. 16B includes NO1610, a command 1611 executed in Stacking 109, a code 1612, and a command meaning 1613.

  As shown in FIG. 16B, a code 1612 is defined for each command 1611, and the Stacking 109 receives the code 1612 corresponding to the command 1611 and executes a stacking operation.

  At time T0 in FIG. 16A, the Stacking 109 receives the value 1 of the Projection column 1 of 0X81 (1605) as a command and stores the value of the Projection column 1 in the stack register REG1 (1402). Here, the code 1612 of the stack operation command 1605 is “0x81”, and the meaning 1613 in FIG. 16B is “stack the projector output row 1 on the stack”.

  That is, the first stack operation command stores the value of the output string 1 of the Projection 107 in the register REG1 (1402). Note that “64 ′” in (64′d1) indicates that the data has 64 bits and the data is d1. The stack in FIG. 16B shows a stack register REG0 (1401) and a stack register REG1 (1402).

  At time T1 in FIG. 16A, as a stack operation command, Stacking 109 receives the value of direct value 0 of 0X10 (1606), writes the value of stack register REG1 (1402) to REG0 (1401), and sets the value of direct value 0. Write to REG1 (1402).

  The value of direct value 0 is set in a separate register, and is set to “2” here. At time T2, the Stacking 109 receives the 0X01 (1607) sum operator as a stack operation command, adds the values of the stack registers 1401 and 1402, and writes the result to the REG1 (1402).

  At time T3, the Stacking 109 receives the stack operation end of 0X7F (1608) as a command, and outputs the final stack operation result.

  As described above, the Stacking 109 executes an operation according to the stack operation command using the stack registers 1401 and 1402.

  FIG. 17 is a diagram illustrating an example of the aggregation result 112 that is the output of the aggregation 111. In the aggregation result 112, grouping columns 0 (1701), an aggregation result 0 (1702), and an aggregation result 1 (1703) are stored in the order of the grouping column and the aggregation result, and the grouping indicates a group. Data is stored in the same format as the number of data columns. In the aggregation result 112, a portion 1704 where no data exists is filled with a zero value up to a predetermined area.

  In the example of FIG. 17, the example in which the FPGA 2 outputs two aggregation results 1702 and 1703 for one grouping column is shown, but the present invention is not limited to this, and the number of aggregation results depends on the setting of the FPGA 2. It can be changed accordingly.

  FIG. 18 is a timing chart illustrating an example of aggregation processing performed in the FPGA 2. In the FPGA 2, the filtering process by the filter 106 and the projection process by the projection 107 are performed in units of pages (8 KB).

  From the time T0 to the FPGA, the filtering process FLT_0 (1801) is executed. When the filtering process is completed, the projection process PRJ_0 (1802) is started at time T1, using the result of the filtering process FLT_0 (1801).

  Using the result of the projection process PRJ_0, in the Grouping 108 and the Stacking 109, the grouping process GRP_0 (1803) and the stacking process STK_0 (1804) are executed in parallel. The grouping process GRP_0 (1803) and the stacking process STK_0 (1804) are started from time T1A when the row data is output in the projection process PRJ_0 (1802).

  When a synonym occurs in the grouping process GRP_0 (1803), the synonym information is notified to the aggregation process together with the result of the grouping process.

  When the grouping process GRP_0 (1803) and the stacking process STK_0 (1804) are completed, the aggregation process A_0 (1805) is executed in the aggregation 111. Note that the aggregation 111 performs synonym processing S_0 (1806) when there is synonym information. The synonym process S_0 (1806) is a process of writing a grouping column and a stack column to a synonym 114 described later.

  In the FPGA 2, the filtering process FLT_N (1807), the projection process PRJ_N (1808), the grouping process GRP_N (1809), the stacking process STK_N (1810), and the aggregation process A_N, which are the processes of the last page of the processing target data 901, are performed. When (1811) and the synonym S_N (1812) are finished, the aggregation result 112, the synonym 114, and the message 113 shown in FIG. 17 are transferred to the DB server 1. Further, the FPGA 2 notifies the DB server 1 of the completion of execution of the completed database command. Note that the message 113 includes the number of groups aggregated by the database command executed by the FPGA 2, the number of synonyms generated, information on operation overflow, and the like.

  The DB server 1 stores the aggregation result 112 received from the FPGA 2 in the result storage area 115, stores the received synonym 114 in the synonym storage area 117, and stores the message 113 in the message storage area 116.

  When synonyms occur frequently, the synonyms 114 may be transferred to the synonym storage area 117 of the DB server 1 when the data of the synonyms 114 is filled.

<Cooperation between FPGA and server>
Next, an aggregation process when a synonym occurs in the grouping process of FPGA 2 will be described. In FIG. 12, when it is determined as a synonym (1204) and the same hash value is assigned to different grouping columns, the grouping column and the stack column are transferred to the DB server 1 as shown in FIG. The DB server 1 performs grouping and aggregation operations in the grouping and aggregation module 120.

  FIG. 19 is a diagram illustrating an example of the format of the synonym 114 that is the output of the aggregation 111.

  The synonym 114 that stores the synonym result in the grouping process is stored in the order of the grouping column and the stack column like the grouping column 0 (2201), the stack column 0 (2202), and the stack column 1 (2203). Data is stored in the same format as the number of (grouping columns). In the synonym 114, a portion 2204 where no data exists is filled with zeros up to a predetermined area (2204).

  The grouping column 2201 stores grouping data 1102 in which synonyms have occurred. The stack column stores the result of stacking processing of the group.

  FIG. 20 is a flowchart illustrating an example of processing performed in the DB server 1. The DBMS 20 of the DB server 1 generates a database command based on a query received from a computer (not shown) and requests the FPGA 2 to process, and the FPGA 2 responds to the DB server 1 with the result of the database processing performed in a predetermined page unit. To do. The DBMS 20 of the DB server 1 receives a plurality of aggregation results grouped in units of pages, and sums up when receiving the aggregation results for all the data specified by the database command, to the computer (not shown) that sends the query. Send back.

  First, the DBMS 20 receives a query for the DB 30 from another computer (not shown) (1901). Based on the accepted query, the DBMS 20 generates a database command for the DB 30 of the storage apparatus 3 as shown in FIG. 4, and issues the database command to the FPGA 2 (1902). In other words, the DBMS 20 determines the processing range of the DB 30 to be processed by the query and the unit size (for example, 8 MB) that the FPGA 2 processes with one database command by dividing the processing range of the DB 30 to process each database processing. Is determined and commanded to the FPGA 2.

  The DBMS 20 determines whether or not a completion command for the database command has been received from the FPGA 2 (1903). If not received, it waits for the completion command. On the other hand, if a completion command has been received, the process advances to step 1904 to determine whether or not the completion command has been received for all data in the range to be processed by the query specified by the database command.

  Since the FPGA 2 responds with the result of the database processing in units of predetermined pages, the DBMS 20 waits until all the processing target data corresponding to the database command is processed. If the database processing has been completed for all the data, the process proceeds to step 1905. If not, the process returns to step 1903 to wait for a completion command.

  In step 1905, the DBMS 20 acquires the aggregation result for each command stored in the result storage area 115. The result storage area 115 stores the aggregation result 112 shown in FIG. 19 in units of commands.

  The DBMS 20 calculates a hash value of the grouping column from the aggregation result read in Step 1905 and generates a group hash table 119. The generation of the hash value is the same as the Grouping 108 of the FPGA 2, and the hash value of the grouping column of the aggregation result 112 shown in FIG. 17 is calculated. Then, the DBMS 20 associates the group ID of the grouping column with the hash value and stores them in the group hash table 119. Note that the hash value calculation and group ID determination may be performed in the same manner as in the FPGA 2 Grouping 108.

  FIG. 24 is a diagram illustrating an example of the group hash table 119. The group hash table 119 has the same configuration as the hash table 728 of the FPGA 2 shown in FIG.

  The group hash table 119 includes one entry including a hash value 1191, a flag 1192 indicating whether the hash value is used, and a group ID 1193 corresponding to the hash value.

  The hash value 1191 stores the hash value calculated from the grouping column 1701 (FIG. 17) of the aggregation result read in Step 1905. The flag 1192 is set to “1” if the hash value 1191 of the entry is used, and is set to “0” if it is not used.

  Next, in step 1907 of FIG. 20, the DBMS 20 sorts the hash values 1191 of the group hash table 119. The DBMS 20 rearranges the order of the grouping column 1701 of the aggregation result 112 according to the group ID 1193 of the group hash table 119 after sorting, and aligns the relationship between the grouping column and the aggregation result for the aggregation result of each page.

  In step 1908, the DBMS 20 aggregates the aggregation result of data in which the relationship between the grouping column and the aggregation result is aligned by the processing up to step 1907. As a result of this processing, the aggregation results that are aggregated in command units and in which the order of the grouping columns is not the same are aggregated in the processing target range of the DB 30 specified by the query, and the processing results for the query are generated. At this point, the processing result for the query is a total result that does not include the synonym 114.

  Next, the DBMS 20 reads the data in the synonym storage area 117 (1909), and determines whether or not a synonym has occurred in the database processing of the FPGA 2 (1910). If the synonym 114 has been written in the synonym storage area 117, the process proceeds to step 1911. On the other hand, if the information in the synonym storage area 117 is not written, the process proceeds to step 1913, where the DBMS 20 responds to the transmission result of the query and ends the process.

  In step 1911, the DBMS 20 performs the grouping process on the data of the synonym 114 again. That is, the DBMS 20 obtains a grouping column (2201) of the synonyms 114 stored in the synonym storage area 117 and calculates a hash value. The DBMS 20 searches the hash value 1191 of the group hash table 119 for an entry that matches the hash value of the calculation result. If there is no corresponding entry, the DBMS 20 adds the hash value as a new hash value 1191 to the group hash table 119.

  In step 1912, the stack string (2202, 2203) included in the data of the synonym 114 for which the DBMS 20 has calculated the hash value is re-added in addition to the aggregation result corresponding to the group ID 1193 of the group hash table 119 with which the hash value matches or is added. calculate. In step 1913, the DBMS 20 responds to the aggregation result recalculated with the data of the synonym 114 to the query transmission source and ends the process.

  Through the above processing, the processing results of the DB 30 aggregated in a predetermined processing unit (page unit) by the FPGA 2 are aggregated with the synonym 114 added and transmitted to the query transmission source. Further, the processing of steps 1905 to 1908 corresponds to the processing of the re-aggregation module 118 of the DBMS 20, and the processing of steps 1909 to 1912 corresponds to the processing of the grouping and aggregation module 120.

  FIG. 21 is a diagram illustrating an example of re-aggregation processing of the aggregation result 112 by the DB server 1. This processing corresponds to the processing in steps 1905 to 1908 in FIG. In the illustrated example, the database command instructed to the FPGA 2 is composed of commands 1 to N, and N aggregation results 2101-1 to 2101-N are output to the DB server 1.

  The aggregation result of the command 1 processed by the FPGA 2 is 2101-1, and the aggregation result of the command N is 2101-N. The order of appearance of the grouping columns differs between the aggregation results 2101-1 and 2101-N. For this reason, the re-aggregation module 118 of the DB server 1 cannot re-aggregate as it is using these results.

  Therefore, the DB server 1 generates a group hash table 119 for the aggregation results of N commands received from the FPGA 2, performs grouping and rearrangement, and calculates new aggregation results 2102-1 to 2102-N. To do.

  In the aggregation results 2102-1 and 2102-N, the order of appearance of the grouping columns is the same, and the DB server 1 can calculate the total or maximum value of the aggregation results of each command in the row direction. The re-aggregation module 118 of the DB server 1 performs the re-aggregation calculation using the aggregation results 2102-1 and 2102-N, and finally calculates the illustrated aggregation result 2103.

  As described above, according to the first embodiment, when the DB server 1 divides the DB 30 and requests processing from the hardware accelerator FPGA 2 in units of several MB, the bandwidth of the DRAM 136 is larger than the bandwidth of the SSD 137. Therefore, reading of the DB 30 from the SSD 137 and reading of data from the DRAM 136 can be performed in parallel. After the control unit 129 replicates the data of the DB 30 to be processed from the SSD 137 to the DRAM 136, the FPGA 2 inputs data of a predetermined processing size (for example, page) from the DRAM 136, which has a faster reading speed than the SSD 137. Performance can be improved.

  Then, the FPGA 2 executes aggregation processing in units of database commands, and the DB server 1 performs re-aggregation processing of aggregation results for a plurality of database commands, and the DB server 1 and FPGA 2 execute database processing in cooperation with each other. The processing performance of the database processing system can be improved.

  Further, in the FPGA 2, the database processing unit in the filtering process and the projection process is set to the page unit of the DB 30, so that pipeline processing can be realized and the performance of the hardware accelerator can be further improved.

  In the second embodiment, processing of the DB server 1 when the synonym occurrence frequency is high will be described. In this embodiment, when the occurrence frequency of synonyms exceeds a predetermined threshold, the DB server 1 reduces the data size processed by the FPGA 2 with one command and suppresses the occurrence of synonyms in the grouping process. . Other configurations are the same as those of the first embodiment.

  As the synonym occurrence frequency, for example, the number of grouping columns included in the synonym 114 or the ratio between the number of grouping columns included in the synonym 114 and the processing target data 901 can be used.

  FIG. 25 is a diagram showing processing for reducing the data size processed by the FPGA 2 when synonyms occur frequently. 1004 to 1008 are the same as in FIG. 10 of the first embodiment, and the DB server 1 sets the data size that the FPGA 2 processes with one command to 8 MB.

  When the grouping 108 performs grouping, the frequency of synonyms is high, the load on the DB server 1 that executes the re-aggregation module 118 increases, and the DB processing performance as a computer system decreases.

  Therefore, if the DB server 1 determines that the synonym occurrence frequency is high, the DB server 1 reduces the size of the processing unit of the DB 30 per command processed by the FPGA 2. Before the size reduction, the data size per command is 8 MB (1004), but after the size reduction, the data size per command is reduced to 4 MB (2006).

  The FPGA 2 reads 4 MB (2006) data from the DRAM 136 in units of one page (8 KB), and executes grouping processing, stacking processing (numerical calculation), aggregation calculation, and the like, similar to the first embodiment.

  When the aggregation process for 8 MB per command is completed, the FPGA 2 returns the grouping column and the aggregation result to the DB server 1 in the same group order as in the first embodiment. The DB server 1 that has received the grouping column and the aggregation result generates a group hash table 119 using the grouping column in the re-aggregation module 118 in the same manner as in the first embodiment, and displays the aggregation result in the order of the hash values. Rearrange and execute global aggregation processing for all commands.

  Thereby, in the second embodiment, in addition to the effects of the first embodiment, the frequency at which the synonym 114 is generated can be reduced, and the degradation of the processing performance of the DB server 1 can be suppressed.

<Summary>
In the first and second embodiments, one storage apparatus 3 having the FPGA 2 is shown as an example. However, a plurality of storage apparatuses 3 can be connected to the PCI switch 4. Further, the FPGA 2 and the storage device 3 may be independent, and a plurality of storage devices 3 can be connected to each FPGA 2.

  In addition, although the storage apparatus 3 is configured with the SSD 137 that stores the DB 30 and the DRAM 136 that once reads data from the SSD 137 and then reads the data into the FPGA 2, the storage apparatus 3 is not limited thereto. The storage device 3 is composed of, for example, a non-volatile semiconductor storage medium that stores the DB 30 and a semiconductor storage medium that is faster in reading speed than the first storage section and stores data in the FPGA 2. A second storage unit to be read may be included.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments are described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. In addition, any of the additions, deletions, or substitutions of other configurations can be applied to a part of the configuration of each embodiment, either alone or in combination.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. In addition, each of the above-described configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files for realizing each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

Claims (15)

A server including a processor and memory;
An accelerator connected to the server for database processing;
A storage system connected to the accelerator for storing a database, and a computer system comprising:
The server
A server command processing unit that receives a query, generates a database command, determines a database range to be processed, a unit size to be processed by one database command by dividing the database range, and instructs the accelerator; ,
A re-aggregation unit that aggregates the output of the accelerator and generates a processing result for the query,
The accelerator is
Based on a command from the server command processing unit, the processing target data of the database is read from the storage device in the unit size, the processing target data is divided into predetermined processing units, and grouped for each predetermined processing unit. A database processing unit that executes database processing including processing, stacking processing, and aggregation processing and outputs an aggregation result;
The re-aggregation unit
When a result of aggregation for the range of the database to be processed is received from the accelerator, the aggregation result is aggregated and generated as a processing result for the query. The computer system according to claim 1,
The database processing unit
A computer system, wherein the grouping process and the stacking process are executed in parallel in the predetermined processing unit. The computer system according to claim 1,
The database processing unit
A grouping processing unit that performs a grouping process on the data string of the predetermined processing unit;
The grouping processing unit
A hash value and group information are calculated for the data in the data string, the hash value and the group information are paired and stored in hash information, and if the hash information includes different group information with the same hash value, the hash value A computer system characterized by detecting a collision. The computer system according to claim 3,
The database processing unit
When the grouping processing unit detects a collision of the hash values, the computer system outputs the group information and a numerical value that is a result of the stacking processing. The computer system according to claim 1,
The storage device
A first storage unit for storing the database;
A second storage unit configured by a semiconductor storage medium having a higher reading speed than the first storage unit and causing the accelerator to read data;
The accelerator is
A computer system, wherein the range of the database to be processed is copied from the first storage unit to the second storage unit and read from the second storage unit for each unit size. The computer system according to claim 1,
The database processing unit
In the grouping process, a hash value and group information are calculated for data in the data string of the predetermined processing unit,
In the stacking process, a numerical value is calculated for data in the data string of the predetermined processing unit,
In the aggregation process, numerical information obtained by aggregating the numerical values for each group information is calculated, and the group information and the numerical information are included in the aggregation result,
The re-aggregation unit
A hash value is calculated from the group information, stored in the group hash information, the aggregation result for the range of the database to be processed is rearranged based on the hash value, and then the numerical information is aggregated, to the query A computer system characterized by generating a processing result. A computer system according to claim 6, wherein
The database processing unit
In the grouping process, when the hash value and the group information are paired and stored in the hash information, and the hash information includes different group information with the same hash value, it is detected that the hash value collides, Output information and numerical values that are the result of the stacking process,
The re-aggregation unit
A computer system that refers to the group hash information, re-aggregates the group information with which the hash values collide, and the numerical value of the stacking process, and recalculates the processing result for the query. The computer system according to claim 3,
The server command processing unit
When the frequency of occurrence of hash value collisions received by the re-aggregation unit exceeds a preset threshold, the database range is divided to reduce the unit size processed by one database command. Computer system to do. An accelerator connected to a storage device for storing a database and receiving a database command to perform database processing,
Receiving a database range to be processed and a unit size to be processed by a single database command by dividing the database range, and reading the database processing target data from the storage device in the unit size, the processing target A database processing unit that divides data into predetermined processing units, executes database processing including grouping processing, stacking processing, and aggregation processing for each predetermined processing unit and outputs an aggregation result;
The database processing unit
An accelerator, wherein the grouping process and the stacking process are executed in parallel in the predetermined processing unit. The accelerator according to claim 9, comprising:
The database processing unit
A grouping processing unit that performs a grouping process on the data string of the predetermined processing unit;
The grouping processing unit
A hash value and group information are calculated for the data in the data string, the hash value and the group information are paired and stored in hash information, and if the hash information includes different group information with the same hash value, the hash value Accelerator characterized by detecting the collision. The accelerator according to claim 10, wherein
The database processing unit
When the grouping processing unit detects a collision of the hash values, the grouping processing unit outputs the group information and a numerical value that is a result of the stacking processing. The accelerator according to claim 9, comprising:
The storage device
A first storage unit for storing the database;
A second storage unit configured by a semiconductor storage medium having a higher reading speed than the first storage unit and causing the accelerator to read data;
The database processing unit
An accelerator, wherein the range of the database to be processed is copied from the first storage unit to the second storage unit and read from the second storage unit for each unit size. A server including a processor and a memory is a database processing method for causing an accelerator connected to the server to perform database processing to process a database of a storage device connected to the accelerator,
The server receives a query, generates a database, determines a database range to be processed and a unit size to be processed by one database command by dividing the database range, and instructs the accelerator And the steps
Based on the command from the server, the accelerator reads data to be processed in the database from the storage device in the unit size, and divides the data to be processed into predetermined processing units, for each predetermined processing unit. A second step of executing a database process including a grouping process, a stacking process, and an aggregation process to output an aggregation result;
A third step in which the server aggregates the output of the accelerator and generates a processing result for the query;
The third step includes
A database processing method, wherein when an aggregation result for the range of the database to be processed is received from the accelerator, the aggregation result is aggregated and generated as a processing result for the query. The database processing method according to claim 13, comprising:
The second step includes
In the grouping process, a hash value and group information are calculated for the data string of the predetermined processing unit, and in the stacking process, a numerical value is calculated for the data string of the predetermined processing unit. , Calculating numerical information obtained by aggregating the numerical values for each group information, and including the group information and numerical information in the aggregation result,
The third step includes
A hash value is calculated from the group information, stored in the group hash information, the aggregation result for the range of the database to be processed is rearranged based on the hash value, and then the numerical information is aggregated, to the query A database processing method characterized by generating a processing result. The database processing method according to claim 14, comprising:
The second step includes
In the grouping process, when the hash value and the group information are paired and stored in the hash information, and the hash information includes different group information with the same hash value, it is detected that the hash value collides, Output information and numerical values that are the result of the stacking process,
The third step includes
A database processing method characterized by referring to the group hash information, re-aggregating the group information with which the hash values collide with numerical values of the stacking process, and recalculating the processing result for the query.

Images